2024-03-21T13:04:34.733 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-21T13:04:34.733 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --archive /home/teuthworker/archive/gabrioux-2024-03-21_12:04:17-orch:cephadm-wip-guits-testing-2024-03-21-0818-distro-default-smithi/7614989 --description orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} --owner scheduled_gabrioux@teuthology --name gabrioux-2024-03-21_12:04:17-orch:cephadm-wip-guits-testing-2024-03-21-0818-distro-default-smithi -- /home/teuthworker/archive/gabrioux-2024-03-21_12:04:17-orch:cephadm-wip-guits-testing-2024-03-21-0818-distro-default-smithi/7614989/orig.config.yaml 2024-03-21T13:04:34.765 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T13:04:34.872 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/gabrioux-2024-03-21_12:04:17-orch:cephadm-wip-guits-testing-2024-03-21-0818-distro-default-smithi/7614989 branch: wip-guits-testing-2024-03-21-0818 description: orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} email: gabrioux@ibm.com first_in_suite: false job_id: '7614989' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: gabrioux-2024-03-21_12:04:17-orch:cephadm-wip-guits-testing-2024-03-21-0818-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-guits-testing-2024-03-21-0818 ceph: conf: client: debug ms: 1 global: mon election default strategy: 1 ms bind msgr1: false ms bind msgr2: true ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon warn on pool no app: false osd: debug ms: 1 debug osd: 20 osd class default list: '*' osd class load list: '*' osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - reached quota - but it is still running - overall HEALTH_ - \(POOL_FULL\) - \(SMALLER_PGP_NUM\) - \(CACHE_POOL_NO_HIT_SET\) - \(CACHE_POOL_NEAR_FULL\) - \(POOL_APP_NOT_ENABLED\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) sha1: 960fbbe4645c2becc8084669be194c95183c7753 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: cephadm_mode: cephadm-package install: ceph: flavor: default sha1: 960fbbe4645c2becc8084669be194c95183c7753 extra_packages: - cephadm selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: wip-guits-testing-2024-03-21-0818 sha1: 960fbbe4645c2becc8084669be194c95183c7753 owner: scheduled_gabrioux@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 187 sha1: 960fbbe4645c2becc8084669be194c95183c7753 sleep_before_teardown: 0 subset: 1/8 suite: orch:cephadm suite_branch: wip-guits-testing-2024-03-21-0818 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_960fbbe4645c2becc8084669be194c95183c7753/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 960fbbe4645c2becc8084669be194c95183c7753 targets: smithi097.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN/oDPzQhVaF9hfNE3tZavEfktdnClngzT1j2vsNv2ywHYOQiuzoOiVPq/+1imK/u71NhX4vTPxergJdOjhjgZs= smithi144.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNsHy2xxQJP5PcBlwnWObdXJc8jXGCO4Z5IAjWxkBkRtQIelMLzm/vvLeNagsA7HWY6wQ7+KeEYbg12jhJDWUYY= tasks: - pexec: all: - sudo dnf install runc -y - sudo sed -i 's/^#runtime = "crun"/runtime = "runc"/g' /usr/share/containers/containers.conf - sudo sed -i 's/runtime = "crun"/#runtime = "crun"/g' /usr/share/containers/containers.conf - install: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - workunit: clients: client.0: - rados/test.sh - rados/test_pool_quota.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-21_12:04:17 tube: smithi user: gabrioux verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-21T13:04:34.872 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_960fbbe4645c2becc8084669be194c95183c7753/qa; will attempt to use it 2024-03-21T13:04:34.873 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_960fbbe4645c2becc8084669be194c95183c7753/qa/tasks 2024-03-21T13:04:34.873 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-21T13:04:34.874 INFO:teuthology.task.internal:Checking packages... 2024-03-21T13:04:34.899 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '960fbbe4645c2becc8084669be194c95183c7753' 2024-03-21T13:04:34.900 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-21T13:04:34.900 INFO:teuthology.packaging:ref: None 2024-03-21T13:04:34.900 INFO:teuthology.packaging:tag: None 2024-03-21T13:04:34.900 INFO:teuthology.packaging:branch: wip-guits-testing-2024-03-21-0818 2024-03-21T13:04:34.900 INFO:teuthology.packaging:sha1: 960fbbe4645c2becc8084669be194c95183c7753 2024-03-21T13:04:34.900 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-guits-testing-2024-03-21-0818 2024-03-21T13:04:35.045 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2267.g960fbbe4 2024-03-21T13:04:35.047 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-21T13:04:35.057 INFO:teuthology.task.internal:no buildpackages task found 2024-03-21T13:04:35.058 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-21T13:04:35.071 INFO:teuthology.task.internal:Saving configuration 2024-03-21T13:04:35.082 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-21T13:04:35.090 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-21T13:04:35.115 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi097.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-03-21_12:04:17-orch:cephadm-wip-guits-testing-2024-03-21-0818-distro-default-smithi/7614989', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-21 12:58:11.414860', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN/oDPzQhVaF9hfNE3tZavEfktdnClngzT1j2vsNv2ywHYOQiuzoOiVPq/+1imK/u71NhX4vTPxergJdOjhjgZs='} 2024-03-21T13:04:35.138 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi144.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-03-21_12:04:17-orch:cephadm-wip-guits-testing-2024-03-21-0818-distro-default-smithi/7614989', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-21 12:58:11.416562', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNsHy2xxQJP5PcBlwnWObdXJc8jXGCO4Z5IAjWxkBkRtQIelMLzm/vvLeNagsA7HWY6wQ7+KeEYbg12jhJDWUYY='} 2024-03-21T13:04:35.138 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-21T13:04:35.149 INFO:teuthology.task.internal:roles: ubuntu@smithi097.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-03-21T13:04:35.149 INFO:teuthology.task.internal:roles: ubuntu@smithi144.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-03-21T13:04:35.149 INFO:teuthology.run_tasks:Running task console_log... 2024-03-21T13:04:35.225 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f696c8950d0>, signals=[15]) 2024-03-21T13:04:35.226 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-21T13:04:35.233 INFO:teuthology.task.internal:Opening connections... 2024-03-21T13:04:35.233 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi097.front.sepia.ceph.com 2024-03-21T13:04:35.235 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T13:04:35.325 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi144.front.sepia.ceph.com 2024-03-21T13:04:35.326 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T13:04:35.416 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-21T13:04:35.422 DEBUG:teuthology.orchestra.run.smithi097:> uname -m 2024-03-21T13:04:35.440 INFO:teuthology.orchestra.run.smithi097.stdout:x86_64 2024-03-21T13:04:35.440 DEBUG:teuthology.orchestra.run.smithi097:> cat /etc/os-release 2024-03-21T13:04:35.496 INFO:teuthology.orchestra.run.smithi097.stdout:NAME="CentOS Stream" 2024-03-21T13:04:35.497 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION="9" 2024-03-21T13:04:35.497 INFO:teuthology.orchestra.run.smithi097.stdout:ID="centos" 2024-03-21T13:04:35.497 INFO:teuthology.orchestra.run.smithi097.stdout:ID_LIKE="rhel fedora" 2024-03-21T13:04:35.497 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION_ID="9" 2024-03-21T13:04:35.497 INFO:teuthology.orchestra.run.smithi097.stdout:PLATFORM_ID="platform:el9" 2024-03-21T13:04:35.497 INFO:teuthology.orchestra.run.smithi097.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-21T13:04:35.497 INFO:teuthology.orchestra.run.smithi097.stdout:ANSI_COLOR="0;31" 2024-03-21T13:04:35.497 INFO:teuthology.orchestra.run.smithi097.stdout:LOGO="fedora-logo-icon" 2024-03-21T13:04:35.497 INFO:teuthology.orchestra.run.smithi097.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-21T13:04:35.497 INFO:teuthology.orchestra.run.smithi097.stdout:HOME_URL="https://centos.org/" 2024-03-21T13:04:35.497 INFO:teuthology.orchestra.run.smithi097.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-21T13:04:35.497 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-21T13:04:35.497 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-21T13:04:35.498 INFO:teuthology.lock.ops:Updating smithi097.front.sepia.ceph.com on lock server 2024-03-21T13:04:35.518 DEBUG:teuthology.orchestra.run.smithi144:> uname -m 2024-03-21T13:04:35.536 INFO:teuthology.orchestra.run.smithi144.stdout:x86_64 2024-03-21T13:04:35.536 DEBUG:teuthology.orchestra.run.smithi144:> cat /etc/os-release 2024-03-21T13:04:35.592 INFO:teuthology.orchestra.run.smithi144.stdout:NAME="CentOS Stream" 2024-03-21T13:04:35.592 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION="9" 2024-03-21T13:04:35.592 INFO:teuthology.orchestra.run.smithi144.stdout:ID="centos" 2024-03-21T13:04:35.592 INFO:teuthology.orchestra.run.smithi144.stdout:ID_LIKE="rhel fedora" 2024-03-21T13:04:35.592 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION_ID="9" 2024-03-21T13:04:35.592 INFO:teuthology.orchestra.run.smithi144.stdout:PLATFORM_ID="platform:el9" 2024-03-21T13:04:35.592 INFO:teuthology.orchestra.run.smithi144.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-21T13:04:35.593 INFO:teuthology.orchestra.run.smithi144.stdout:ANSI_COLOR="0;31" 2024-03-21T13:04:35.593 INFO:teuthology.orchestra.run.smithi144.stdout:LOGO="fedora-logo-icon" 2024-03-21T13:04:35.593 INFO:teuthology.orchestra.run.smithi144.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-21T13:04:35.593 INFO:teuthology.orchestra.run.smithi144.stdout:HOME_URL="https://centos.org/" 2024-03-21T13:04:35.593 INFO:teuthology.orchestra.run.smithi144.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-21T13:04:35.593 INFO:teuthology.orchestra.run.smithi144.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-21T13:04:35.593 INFO:teuthology.orchestra.run.smithi144.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-21T13:04:35.593 INFO:teuthology.lock.ops:Updating smithi144.front.sepia.ceph.com on lock server 2024-03-21T13:04:35.618 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-21T13:04:35.628 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-21T13:04:35.636 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-21T13:04:35.637 DEBUG:teuthology.orchestra.run.smithi097:> test '!' -e /home/ubuntu/cephtest 2024-03-21T13:04:35.639 DEBUG:teuthology.orchestra.run.smithi144:> test '!' -e /home/ubuntu/cephtest 2024-03-21T13:04:35.654 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-21T13:04:35.669 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-21T13:04:35.669 DEBUG:teuthology.orchestra.run.smithi097:> test -z $(ls -A /var/lib/ceph) 2024-03-21T13:04:35.695 DEBUG:teuthology.orchestra.run.smithi144:> test -z $(ls -A /var/lib/ceph) 2024-03-21T13:04:35.811 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-21T13:04:35.868 INFO:teuthology.run_tasks:Running task kernel... 2024-03-21T13:04:35.884 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-03-21T13:04:35.884 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-03-21T13:04:35.884 DEBUG:teuthology.orchestra.run.smithi097:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-21T13:04:35.885 DEBUG:teuthology.orchestra.run.smithi144:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-21T13:04:35.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T13:04:35.901 DEBUG:teuthology.orchestra.run.smithi097:> uname -r 2024-03-21T13:04:35.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T13:04:35.902 DEBUG:teuthology.orchestra.run.smithi144:> uname -r 2024-03-21T13:04:35.957 INFO:teuthology.orchestra.run.smithi097.stdout:5.14.0-361.el9.x86_64 2024-03-21T13:04:35.957 INFO:teuthology.task.kernel:Running kernel on smithi097: 5.14.0-361.el9.x86_64 2024-03-21T13:04:35.958 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum install -y kernel 2024-03-21T13:04:35.959 INFO:teuthology.orchestra.run.smithi144.stdout:5.14.0-361.el9.x86_64 2024-03-21T13:04:35.959 INFO:teuthology.task.kernel:Running kernel on smithi144: 5.14.0-361.el9.x86_64 2024-03-21T13:04:35.959 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum install -y kernel 2024-03-21T13:04:38.071 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-03-21T13:04:38.555 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS-9 - AppStream 7.3 MB/s | 6.7 MB 00:00 2024-03-21T13:04:39.459 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS-9 - BaseOS 7.1 MB/s | 2.0 MB 00:00 2024-03-21T13:04:40.205 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS-9 - BaseOS 3.6 MB/s | 2.0 MB 00:00 2024-03-21T13:04:40.375 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-03-21T13:04:41.729 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - BaseOS 6.8 MB/s | 8.0 MB 00:01 2024-03-21T13:04:42.634 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - AppStream 15 MB/s | 19 MB 00:01 2024-03-21T13:04:45.238 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - AppStream 7.4 MB/s | 19 MB 00:02 2024-03-21T13:04:46.510 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.1 MB 00:00 2024-03-21T13:04:48.977 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - Extras packages 12 kB/s | 16 kB 00:01 2024-03-21T13:04:49.664 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - CRB 6.0 MB/s | 6.1 MB 00:01 2024-03-21T13:04:51.356 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - Extras packages 28 kB/s | 16 kB 00:00 2024-03-21T13:04:53.099 INFO:teuthology.orchestra.run.smithi097.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-03-21T13:04:57.924 INFO:teuthology.orchestra.run.smithi097.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-03-21T13:05:00.131 INFO:teuthology.orchestra.run.smithi097.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-21T13:05:00.131 INFO:teuthology.orchestra.run.smithi097.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-21T13:05:00.225 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-21T13:05:00.236 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-21T13:05:00.236 INFO:teuthology.orchestra.run.smithi097.stdout: Package Architecture Version Repository Size 2024-03-21T13:05:00.236 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-21T13:05:00.236 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-03-21T13:05:00.236 INFO:teuthology.orchestra.run.smithi097.stdout: kernel x86_64 5.14.0-430.el9 baseos 44 k 2024-03-21T13:05:00.236 INFO:teuthology.orchestra.run.smithi097.stdout:Installing dependencies: 2024-03-21T13:05:00.236 INFO:teuthology.orchestra.run.smithi097.stdout: kernel-core x86_64 5.14.0-430.el9 baseos 15 M 2024-03-21T13:05:00.236 INFO:teuthology.orchestra.run.smithi097.stdout: kernel-modules x86_64 5.14.0-430.el9 baseos 33 M 2024-03-21T13:05:00.237 INFO:teuthology.orchestra.run.smithi097.stdout: kernel-modules-core x86_64 5.14.0-430.el9 baseos 27 M 2024-03-21T13:05:00.237 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:05:00.237 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-03-21T13:05:00.237 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-21T13:05:00.237 INFO:teuthology.orchestra.run.smithi097.stdout:Install 4 Packages 2024-03-21T13:05:00.237 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:05:00.237 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 76 M 2024-03-21T13:05:00.238 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 123 M 2024-03-21T13:05:00.238 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-03-21T13:05:00.584 INFO:teuthology.orchestra.run.smithi097.stdout:(1/4): kernel-5.14.0-430.el9.x86_64.rpm 160 kB/s | 44 kB 00:00 2024-03-21T13:05:02.601 INFO:teuthology.orchestra.run.smithi097.stdout:(2/4): kernel-core-5.14.0-430.el9.x86_64.rpm 6.5 MB/s | 15 MB 00:02 2024-03-21T13:05:03.184 INFO:teuthology.orchestra.run.smithi097.stdout:(3/4): kernel-modules-core-5.14.0-430.el9.x86_6 11 MB/s | 27 MB 00:02 2024-03-21T13:05:03.585 INFO:teuthology.orchestra.run.smithi097.stdout:(4/4): kernel-modules-5.14.0-430.el9.x86_64.rpm 10 MB/s | 33 MB 00:03 2024-03-21T13:05:03.585 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-03-21T13:05:03.585 INFO:teuthology.orchestra.run.smithi097.stdout:Total 23 MB/s | 76 MB 00:03 2024-03-21T13:05:04.249 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-03-21T13:05:04.608 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-03-21T13:05:04.608 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-03-21T13:05:05.629 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-03-21T13:05:05.629 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-03-21T13:05:08.121 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-03-21T13:05:08.732 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : kernel-modules-core-5.14.0-430.el9.x86_64 1/4 2024-03-21T13:05:08.817 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-21T13:05:11.580 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-21T13:05:11.772 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-21T13:05:16.477 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-21T13:05:16.595 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : kernel-5.14.0-430.el9.x86_64 4/4 2024-03-21T13:05:21.338 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-21T13:05:50.640 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 4/4 2024-03-21T13:05:50.641 INFO:teuthology.orchestra.run.smithi097.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-21T13:05:50.641 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:05:50.685 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 4/4 2024-03-21T13:05:51.054 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: kernel-5.14.0-430.el9.x86_64 4/4 2024-03-21T13:05:51.054 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : kernel-5.14.0-430.el9.x86_64 1/4 2024-03-21T13:05:51.054 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-21T13:05:51.054 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-21T13:05:51.438 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-21T13:05:51.438 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:05:51.438 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-03-21T13:05:51.438 INFO:teuthology.orchestra.run.smithi097.stdout: kernel-5.14.0-430.el9.x86_64 2024-03-21T13:05:51.438 INFO:teuthology.orchestra.run.smithi097.stdout: kernel-core-5.14.0-430.el9.x86_64 2024-03-21T13:05:51.438 INFO:teuthology.orchestra.run.smithi097.stdout: kernel-modules-5.14.0-430.el9.x86_64 2024-03-21T13:05:51.438 INFO:teuthology.orchestra.run.smithi097.stdout: kernel-modules-core-5.14.0-430.el9.x86_64 2024-03-21T13:05:51.438 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:05:51.438 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-21T13:05:51.755 DEBUG:teuthology.orchestra.run.smithi097:> rpm -q kernel | sort -rV | head -n 1 2024-03-21T13:05:51.869 INFO:teuthology.orchestra.run.smithi097.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-21T13:05:51.869 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-21T13:05:51.870 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-430.el9.x86_64 2024-03-21T13:05:51.870 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-21T13:05:51.870 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-21T13:05:51.870 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-03-21T13:05:51.870 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi097.front.sepia.ceph.com, path=None, version=distro) 2024-03-21T13:05:51.870 DEBUG:teuthology.orchestra.run.smithi097:> rpm -q kernel | sort -rV | head -n 1 2024-03-21T13:05:51.949 INFO:teuthology.orchestra.run.smithi097.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-21T13:05:51.949 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-21T13:05:51.950 DEBUG:teuthology.orchestra.run.smithi097:> sudo rpm -qi grub2-tools 2024-03-21T13:05:51.992 INFO:teuthology.orchestra.run.smithi097.stdout:Name : grub2-tools 2024-03-21T13:05:51.992 INFO:teuthology.orchestra.run.smithi097.stdout:Epoch : 1 2024-03-21T13:05:51.992 INFO:teuthology.orchestra.run.smithi097.stdout:Version : 2.06 2024-03-21T13:05:51.992 INFO:teuthology.orchestra.run.smithi097.stdout:Release : 68.el9 2024-03-21T13:05:51.993 INFO:teuthology.orchestra.run.smithi097.stdout:Architecture: x86_64 2024-03-21T13:05:51.993 INFO:teuthology.orchestra.run.smithi097.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-21T13:05:51.993 INFO:teuthology.orchestra.run.smithi097.stdout:Group : Unspecified 2024-03-21T13:05:51.993 INFO:teuthology.orchestra.run.smithi097.stdout:Size : 8264254 2024-03-21T13:05:51.993 INFO:teuthology.orchestra.run.smithi097.stdout:License : GPLv3+ 2024-03-21T13:05:51.993 INFO:teuthology.orchestra.run.smithi097.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-21T13:05:51.993 INFO:teuthology.orchestra.run.smithi097.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-21T13:05:51.993 INFO:teuthology.orchestra.run.smithi097.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-21T13:05:51.993 INFO:teuthology.orchestra.run.smithi097.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-21T13:05:51.993 INFO:teuthology.orchestra.run.smithi097.stdout:Packager : builder@centos.org 2024-03-21T13:05:51.993 INFO:teuthology.orchestra.run.smithi097.stdout:Vendor : CentOS 2024-03-21T13:05:51.993 INFO:teuthology.orchestra.run.smithi097.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-21T13:05:51.993 INFO:teuthology.orchestra.run.smithi097.stdout:Summary : Support tools for GRUB. 2024-03-21T13:05:51.993 INFO:teuthology.orchestra.run.smithi097.stdout:Description : 2024-03-21T13:05:51.993 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:05:51.994 INFO:teuthology.orchestra.run.smithi097.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-21T13:05:51.994 INFO:teuthology.orchestra.run.smithi097.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-21T13:05:51.994 INFO:teuthology.orchestra.run.smithi097.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-21T13:05:51.994 INFO:teuthology.orchestra.run.smithi097.stdout:hardware devices. 2024-03-21T13:05:51.994 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:05:51.994 INFO:teuthology.orchestra.run.smithi097.stdout:This subpackage provides tools for support of all platforms. 2024-03-21T13:05:51.995 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-21T13:05:51.995 INFO:teuthology.task.kernel:Updating grub on smithi097 to boot 5.14.0-430.el9.x86_64 2024-03-21T13:05:51.995 DEBUG:teuthology.orchestra.run.smithi097:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-21T13:05:52.569 INFO:teuthology.orchestra.run.smithi097.stderr:Generating grub configuration file ... 2024-03-21T13:05:54.017 INFO:teuthology.orchestra.run.smithi097.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-21T13:05:54.037 INFO:teuthology.orchestra.run.smithi097.stderr:done 2024-03-21T13:05:54.040 DEBUG:teuthology.orchestra.run.smithi097:> mktemp 2024-03-21T13:05:54.055 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/tmp.t9hf7Q5Tix 2024-03-21T13:05:54.055 DEBUG:teuthology.orchestra.run.smithi097:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.t9hf7Q5Tix 2024-03-21T13:05:54.124 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0666 /tmp/tmp.t9hf7Q5Tix 2024-03-21T13:05:54.261 DEBUG:teuthology.orchestra.remote:smithi097:/tmp/tmp.t9hf7Q5Tix is 6KB 2024-03-21T13:05:54.281 DEBUG:teuthology.orchestra.run.smithi097:> rm -fr /tmp/tmp.t9hf7Q5Tix 2024-03-21T13:05:54.295 DEBUG:teuthology.orchestra.run.smithi097:> sudo /bin/ls /boot/loader/entries || true 2024-03-21T13:05:54.369 INFO:teuthology.orchestra.run.smithi097.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-21T13:05:54.369 INFO:teuthology.orchestra.run.smithi097.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-21T13:05:54.370 INFO:teuthology.orchestra.run.smithi097.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-21T13:05:54.370 INFO:teuthology.orchestra.run.smithi097.stdout:15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64.conf 2024-03-21T13:05:54.371 DEBUG:teuthology.orchestra.run.smithi097:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64 2024-03-21T13:05:54.527 DEBUG:teuthology.orchestra.run.smithi097:> sudo shutdown -r now 2024-03-21T13:06:24.529 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-03-21T13:06:24.529 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-03-21T13:06:24.531 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T13:06:42.846 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.97 2024-03-21T13:06:51.848 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-03-21T13:06:51.849 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T13:06:54.910 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.97 2024-03-21T13:07:06.911 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-03-21T13:07:06.912 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T13:07:25.342 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.97 2024-03-21T13:07:40.343 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-03-21T13:07:40.344 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T13:07:40.688 DEBUG:teuthology.orchestra.run.smithi097:> true 2024-03-21T13:07:41.144 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-03-21T13:07:41.145 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-430.el9.x86_64"... 2024-03-21T13:07:41.145 DEBUG:teuthology.orchestra.run.smithi097:> uname -r 2024-03-21T13:07:41.159 INFO:teuthology.orchestra.run.smithi097.stdout:5.14.0-430.el9.x86_64 2024-03-21T13:07:41.159 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-430.el9.x86_64 vs 5.14.0-430.el9.x86_64 2024-03-21T13:07:41.159 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-21T13:07:41.159 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-21T13:07:42.160 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-03-21T13:07:42.160 DEBUG:teuthology.orchestra.run.smithi097:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-21T13:07:42.219 INFO:teuthology.orchestra.run.smithi097.stdout:ttyS1 2024-03-21T13:07:42.258 DEBUG:teuthology.parallel:result is None 2024-03-21T13:09:34.627 INFO:teuthology.orchestra.run.smithi144.stdout:Extra Packages for Enterprise Linux 74 kB/s | 21 MB 04:45 2024-03-21T13:09:39.557 INFO:teuthology.orchestra.run.smithi144.stdout:lab-extras 18 kB/s | 1.7 kB 00:00 2024-03-21T13:09:41.623 INFO:teuthology.orchestra.run.smithi144.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-21T13:09:41.623 INFO:teuthology.orchestra.run.smithi144.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-21T13:09:41.714 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-03-21T13:09:41.723 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-03-21T13:09:41.724 INFO:teuthology.orchestra.run.smithi144.stdout: Package Architecture Version Repository Size 2024-03-21T13:09:41.724 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-03-21T13:09:41.724 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-03-21T13:09:41.724 INFO:teuthology.orchestra.run.smithi144.stdout: kernel x86_64 5.14.0-430.el9 baseos 44 k 2024-03-21T13:09:41.724 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2024-03-21T13:09:41.724 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-core x86_64 5.14.0-430.el9 baseos 15 M 2024-03-21T13:09:41.724 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-modules x86_64 5.14.0-430.el9 baseos 33 M 2024-03-21T13:09:41.724 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-modules-core x86_64 5.14.0-430.el9 baseos 27 M 2024-03-21T13:09:41.724 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:09:41.725 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-03-21T13:09:41.725 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-03-21T13:09:41.725 INFO:teuthology.orchestra.run.smithi144.stdout:Install 4 Packages 2024-03-21T13:09:41.725 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:09:41.725 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 76 M 2024-03-21T13:09:41.725 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 123 M 2024-03-21T13:09:41.725 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-03-21T13:09:41.901 INFO:teuthology.orchestra.run.smithi144.stdout:(1/4): kernel-5.14.0-430.el9.x86_64.rpm 463 kB/s | 44 kB 00:00 2024-03-21T13:09:42.284 INFO:teuthology.orchestra.run.smithi144.stdout:(2/4): kernel-core-5.14.0-430.el9.x86_64.rpm 31 MB/s | 15 MB 00:00 2024-03-21T13:09:42.659 INFO:teuthology.orchestra.run.smithi144.stdout:(3/4): kernel-modules-core-5.14.0-430.el9.x86_6 36 MB/s | 27 MB 00:00 2024-03-21T13:09:43.001 INFO:teuthology.orchestra.run.smithi144.stdout:(4/4): kernel-modules-5.14.0-430.el9.x86_64.rpm 28 MB/s | 33 MB 00:01 2024-03-21T13:09:43.002 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-03-21T13:09:43.002 INFO:teuthology.orchestra.run.smithi144.stdout:Total 59 MB/s | 76 MB 00:01 2024-03-21T13:09:43.662 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-03-21T13:09:44.051 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-03-21T13:09:44.051 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-03-21T13:09:45.028 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-03-21T13:09:45.029 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-03-21T13:09:47.497 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-03-21T13:09:48.248 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : kernel-modules-core-5.14.0-430.el9.x86_64 1/4 2024-03-21T13:09:48.337 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-21T13:09:51.068 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-21T13:09:51.206 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-21T13:09:55.915 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-21T13:09:56.014 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : kernel-5.14.0-430.el9.x86_64 4/4 2024-03-21T13:10:00.764 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-21T13:10:30.550 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 4/4 2024-03-21T13:10:30.550 INFO:teuthology.orchestra.run.smithi144.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-21T13:10:30.551 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:10:30.594 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 4/4 2024-03-21T13:10:30.993 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-5.14.0-430.el9.x86_64 4/4 2024-03-21T13:10:30.993 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : kernel-5.14.0-430.el9.x86_64 1/4 2024-03-21T13:10:30.993 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-21T13:10:30.993 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-21T13:10:31.397 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-21T13:10:31.398 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:10:31.398 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-03-21T13:10:31.398 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-5.14.0-430.el9.x86_64 2024-03-21T13:10:31.398 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-core-5.14.0-430.el9.x86_64 2024-03-21T13:10:31.398 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-modules-5.14.0-430.el9.x86_64 2024-03-21T13:10:31.398 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-modules-core-5.14.0-430.el9.x86_64 2024-03-21T13:10:31.398 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:10:31.398 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-03-21T13:10:31.748 DEBUG:teuthology.orchestra.run.smithi144:> rpm -q kernel | sort -rV | head -n 1 2024-03-21T13:10:31.846 INFO:teuthology.orchestra.run.smithi144.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-21T13:10:31.847 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-21T13:10:31.847 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-430.el9.x86_64 2024-03-21T13:10:31.847 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-21T13:10:31.847 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-21T13:10:31.847 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-03-21T13:10:31.847 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi144.front.sepia.ceph.com, path=None, version=distro) 2024-03-21T13:10:31.847 DEBUG:teuthology.orchestra.run.smithi144:> rpm -q kernel | sort -rV | head -n 1 2024-03-21T13:10:31.927 INFO:teuthology.orchestra.run.smithi144.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-21T13:10:31.927 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-21T13:10:31.927 DEBUG:teuthology.orchestra.run.smithi144:> sudo rpm -qi grub2-tools 2024-03-21T13:10:31.961 INFO:teuthology.orchestra.run.smithi144.stdout:Name : grub2-tools 2024-03-21T13:10:31.961 INFO:teuthology.orchestra.run.smithi144.stdout:Epoch : 1 2024-03-21T13:10:31.961 INFO:teuthology.orchestra.run.smithi144.stdout:Version : 2.06 2024-03-21T13:10:31.961 INFO:teuthology.orchestra.run.smithi144.stdout:Release : 68.el9 2024-03-21T13:10:31.962 INFO:teuthology.orchestra.run.smithi144.stdout:Architecture: x86_64 2024-03-21T13:10:31.962 INFO:teuthology.orchestra.run.smithi144.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-21T13:10:31.962 INFO:teuthology.orchestra.run.smithi144.stdout:Group : Unspecified 2024-03-21T13:10:31.962 INFO:teuthology.orchestra.run.smithi144.stdout:Size : 8264254 2024-03-21T13:10:31.962 INFO:teuthology.orchestra.run.smithi144.stdout:License : GPLv3+ 2024-03-21T13:10:31.962 INFO:teuthology.orchestra.run.smithi144.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-21T13:10:31.962 INFO:teuthology.orchestra.run.smithi144.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-21T13:10:31.962 INFO:teuthology.orchestra.run.smithi144.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-21T13:10:31.962 INFO:teuthology.orchestra.run.smithi144.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-21T13:10:31.962 INFO:teuthology.orchestra.run.smithi144.stdout:Packager : builder@centos.org 2024-03-21T13:10:31.962 INFO:teuthology.orchestra.run.smithi144.stdout:Vendor : CentOS 2024-03-21T13:10:31.962 INFO:teuthology.orchestra.run.smithi144.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-21T13:10:31.962 INFO:teuthology.orchestra.run.smithi144.stdout:Summary : Support tools for GRUB. 2024-03-21T13:10:31.963 INFO:teuthology.orchestra.run.smithi144.stdout:Description : 2024-03-21T13:10:31.963 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:10:31.963 INFO:teuthology.orchestra.run.smithi144.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-21T13:10:31.963 INFO:teuthology.orchestra.run.smithi144.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-21T13:10:31.963 INFO:teuthology.orchestra.run.smithi144.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-21T13:10:31.963 INFO:teuthology.orchestra.run.smithi144.stdout:hardware devices. 2024-03-21T13:10:31.963 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:10:31.963 INFO:teuthology.orchestra.run.smithi144.stdout:This subpackage provides tools for support of all platforms. 2024-03-21T13:10:31.964 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-21T13:10:31.964 INFO:teuthology.task.kernel:Updating grub on smithi144 to boot 5.14.0-430.el9.x86_64 2024-03-21T13:10:31.964 DEBUG:teuthology.orchestra.run.smithi144:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-21T13:10:32.582 INFO:teuthology.orchestra.run.smithi144.stderr:Generating grub configuration file ... 2024-03-21T13:10:33.927 INFO:teuthology.orchestra.run.smithi144.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-21T13:10:33.978 INFO:teuthology.orchestra.run.smithi144.stderr:done 2024-03-21T13:10:33.980 DEBUG:teuthology.orchestra.run.smithi144:> mktemp 2024-03-21T13:10:33.995 INFO:teuthology.orchestra.run.smithi144.stdout:/tmp/tmp.tjecx9w2ad 2024-03-21T13:10:33.995 DEBUG:teuthology.orchestra.run.smithi144:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.tjecx9w2ad 2024-03-21T13:10:34.071 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 0666 /tmp/tmp.tjecx9w2ad 2024-03-21T13:10:34.207 DEBUG:teuthology.orchestra.remote:smithi144:/tmp/tmp.tjecx9w2ad is 6KB 2024-03-21T13:10:34.258 DEBUG:teuthology.orchestra.run.smithi144:> rm -fr /tmp/tmp.tjecx9w2ad 2024-03-21T13:10:34.275 DEBUG:teuthology.orchestra.run.smithi144:> sudo /bin/ls /boot/loader/entries || true 2024-03-21T13:10:34.338 INFO:teuthology.orchestra.run.smithi144.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-21T13:10:34.338 INFO:teuthology.orchestra.run.smithi144.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-21T13:10:34.338 INFO:teuthology.orchestra.run.smithi144.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-21T13:10:34.338 INFO:teuthology.orchestra.run.smithi144.stdout:15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64.conf 2024-03-21T13:10:34.340 DEBUG:teuthology.orchestra.run.smithi144:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64 2024-03-21T13:10:34.495 DEBUG:teuthology.orchestra.run.smithi144:> sudo shutdown -r now 2024-03-21T13:11:04.498 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-03-21T13:11:04.499 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-03-21T13:11:04.500 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T13:11:22.910 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.144 2024-03-21T13:11:31.911 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-03-21T13:11:31.912 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T13:11:34.974 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.144 2024-03-21T13:11:46.976 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-03-21T13:11:46.977 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T13:12:05.406 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.144 2024-03-21T13:12:20.407 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-03-21T13:12:20.408 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T13:12:20.721 DEBUG:teuthology.orchestra.run.smithi144:> true 2024-03-21T13:12:21.164 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-03-21T13:12:21.164 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-430.el9.x86_64"... 2024-03-21T13:12:21.164 DEBUG:teuthology.orchestra.run.smithi144:> uname -r 2024-03-21T13:12:21.179 INFO:teuthology.orchestra.run.smithi144.stdout:5.14.0-430.el9.x86_64 2024-03-21T13:12:21.179 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-430.el9.x86_64 vs 5.14.0-430.el9.x86_64 2024-03-21T13:12:21.179 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-21T13:12:21.179 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-21T13:12:22.180 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-03-21T13:12:22.181 DEBUG:teuthology.orchestra.run.smithi144:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-21T13:12:22.282 INFO:teuthology.orchestra.run.smithi144.stdout:ttyS1 2024-03-21T13:12:22.324 DEBUG:teuthology.parallel:result is None 2024-03-21T13:12:22.324 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-21T13:12:22.334 INFO:teuthology.task.internal:Creating test directory... 2024-03-21T13:12:22.335 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-21T13:12:22.339 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-21T13:12:22.364 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-21T13:12:22.374 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-21T13:12:22.381 INFO:teuthology.task.internal:Creating archive directory... 2024-03-21T13:12:22.382 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-21T13:12:22.399 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-21T13:12:22.446 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-21T13:12:22.454 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-21T13:12:22.454 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-21T13:12:22.482 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-21T13:12:22.539 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-21T13:12:22.542 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-21T13:12:22.556 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-21T13:12:22.559 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-21T13:12:22.561 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-21T13:12:22.570 INFO:teuthology.task.internal:Configuring sudo... 2024-03-21T13:12:22.571 DEBUG:teuthology.orchestra.run.smithi097:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-21T13:12:22.574 DEBUG:teuthology.orchestra.run.smithi144:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-21T13:12:22.619 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-21T13:12:22.629 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-21T13:12:22.629 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-21T13:12:22.646 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-21T13:12:22.689 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-21T13:12:22.743 DEBUG:teuthology.orchestra.run.smithi097:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-21T13:12:22.789 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-21T13:12:22.872 DEBUG:teuthology.orchestra.run.smithi097:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-21T13:12:22.914 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-21T13:12:22.914 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-21T13:12:22.981 DEBUG:teuthology.orchestra.run.smithi144:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-21T13:12:23.027 DEBUG:teuthology.orchestra.run.smithi144:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-21T13:12:23.082 DEBUG:teuthology.orchestra.run.smithi144:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-21T13:12:23.168 DEBUG:teuthology.orchestra.run.smithi144:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-21T13:12:23.212 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-21T13:12:23.213 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-21T13:12:23.282 DEBUG:teuthology.orchestra.run.smithi097:> sudo service rsyslog restart 2024-03-21T13:12:23.285 DEBUG:teuthology.orchestra.run.smithi144:> sudo service rsyslog restart 2024-03-21T13:12:23.337 INFO:teuthology.orchestra.run.smithi097.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-21T13:12:23.446 INFO:teuthology.orchestra.run.smithi144.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-21T13:12:23.839 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-21T13:12:23.847 INFO:teuthology.task.internal:Starting timer... 2024-03-21T13:12:23.848 INFO:teuthology.run_tasks:Running task pcp... 2024-03-21T13:12:23.861 INFO:teuthology.run_tasks:Running task selinux... 2024-03-21T13:12:23.870 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-03-21T13:12:23.871 DEBUG:teuthology.orchestra.run.smithi097:> sudo service auditd rotate 2024-03-21T13:12:23.937 INFO:teuthology.orchestra.run.smithi097.stdout:Rotating logs: 2024-03-21T13:12:23.939 DEBUG:teuthology.orchestra.run.smithi144:> sudo service auditd rotate 2024-03-21T13:12:23.997 INFO:teuthology.orchestra.run.smithi144.stdout:Rotating logs: 2024-03-21T13:12:23.999 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-21T13:12:23.999 DEBUG:teuthology.orchestra.run.smithi097:> /usr/sbin/getenforce 2024-03-21T13:12:24.029 INFO:teuthology.orchestra.run.smithi097.stdout:Permissive 2024-03-21T13:12:24.029 DEBUG:teuthology.orchestra.run.smithi144:> /usr/sbin/getenforce 2024-03-21T13:12:24.049 INFO:teuthology.orchestra.run.smithi144.stdout:Permissive 2024-03-21T13:12:24.049 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi097.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi144.front.sepia.ceph.com': 'permissive'} 2024-03-21T13:12:24.050 DEBUG:teuthology.orchestra.run.smithi097:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-03-21T13:12:24.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T13:12:24.098 DEBUG:teuthology.orchestra.run.smithi144:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-03-21T13:12:24.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T13:12:24.126 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-21T13:12:24.126 DEBUG:teuthology.orchestra.run.smithi097:> sudo /usr/sbin/setenforce permissive 2024-03-21T13:12:24.173 DEBUG:teuthology.orchestra.run.smithi144:> sudo /usr/sbin/setenforce permissive 2024-03-21T13:12:24.210 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-21T13:12:24.224 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-21T13:12:24.249 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-21T13:12:24.251 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi097.front.sepia.ceph.com,smithi144.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-21T13:16:42.306 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi097.front.sepia.ceph.com'), Remote(name='ubuntu@smithi144.front.sepia.ceph.com')] 2024-03-21T13:16:42.307 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-03-21T13:16:42.308 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T13:16:42.398 DEBUG:teuthology.orchestra.run.smithi097:> true 2024-03-21T13:16:42.477 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-03-21T13:16:42.477 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-03-21T13:16:42.477 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T13:16:42.566 DEBUG:teuthology.orchestra.run.smithi144:> true 2024-03-21T13:16:42.645 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-03-21T13:16:42.646 INFO:teuthology.run_tasks:Running task clock... 2024-03-21T13:16:42.656 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-21T13:16:42.656 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-21T13:16:42.656 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-21T13:16:42.659 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-21T13:16:42.659 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-21T13:16:42.688 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-21T13:16:42.705 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-21T13:16:42.718 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-21T13:16:42.734 INFO:teuthology.orchestra.run.smithi097.stderr:sudo: ntpd: command not found 2024-03-21T13:16:42.735 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-21T13:16:42.747 INFO:teuthology.orchestra.run.smithi097.stdout:506 Cannot talk to daemon 2024-03-21T13:16:42.764 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-21T13:16:42.765 INFO:teuthology.orchestra.run.smithi144.stderr:sudo: ntpd: command not found 2024-03-21T13:16:42.779 INFO:teuthology.orchestra.run.smithi144.stdout:506 Cannot talk to daemon 2024-03-21T13:16:42.780 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-21T13:16:42.795 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-21T13:16:42.811 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-21T13:16:42.830 INFO:teuthology.orchestra.run.smithi097.stderr:bash: line 1: ntpq: command not found 2024-03-21T13:16:42.837 INFO:teuthology.orchestra.run.smithi097.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-21T13:16:42.837 INFO:teuthology.orchestra.run.smithi097.stdout:=============================================================================== 2024-03-21T13:16:42.837 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T13:16:42.837 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T13:16:42.838 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T13:16:42.838 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T13:16:42.865 INFO:teuthology.orchestra.run.smithi144.stderr:bash: line 1: ntpq: command not found 2024-03-21T13:16:42.870 INFO:teuthology.orchestra.run.smithi144.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-21T13:16:42.870 INFO:teuthology.orchestra.run.smithi144.stdout:=============================================================================== 2024-03-21T13:16:42.870 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T13:16:42.870 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T13:16:42.871 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T13:16:42.871 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T13:16:42.871 INFO:teuthology.run_tasks:Running task pexec... 2024-03-21T13:16:42.880 INFO:teuthology.task.pexec:Executing custom commands... 2024-03-21T13:16:42.880 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi097.front.sepia.ceph.com 2024-03-21T13:16:42.881 DEBUG:teuthology.orchestra.run.smithi097:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-03-21T13:16:42.881 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi144.front.sepia.ceph.com 2024-03-21T13:16:42.881 DEBUG:teuthology.orchestra.run.smithi144:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-03-21T13:16:43.462 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:00:47 ago on Thu 21 Mar 2024 01:15:56 PM UTC. 2024-03-21T13:16:43.492 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:00:47 ago on Thu 21 Mar 2024 01:15:56 PM UTC. 2024-03-21T13:16:43.608 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-21T13:16:43.625 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-21T13:16:43.625 INFO:teuthology.orchestra.run.smithi097.stdout: Package Architecture Version Repository Size 2024-03-21T13:16:43.625 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-21T13:16:43.625 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-03-21T13:16:43.625 INFO:teuthology.orchestra.run.smithi097.stdout: runc x86_64 4:1.1.12-2.el9 appstream 3.1 M 2024-03-21T13:16:43.625 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:16:43.625 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-03-21T13:16:43.625 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-21T13:16:43.625 INFO:teuthology.orchestra.run.smithi097.stdout:Install 1 Package 2024-03-21T13:16:43.625 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:16:43.627 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 3.1 M 2024-03-21T13:16:43.627 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 9.8 M 2024-03-21T13:16:43.627 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-03-21T13:16:43.637 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-03-21T13:16:43.639 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-03-21T13:16:43.639 INFO:teuthology.orchestra.run.smithi144.stdout: Package Architecture Version Repository Size 2024-03-21T13:16:43.639 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-03-21T13:16:43.639 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-03-21T13:16:43.639 INFO:teuthology.orchestra.run.smithi144.stdout: runc x86_64 4:1.1.12-2.el9 appstream 3.1 M 2024-03-21T13:16:43.639 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:16:43.639 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-03-21T13:16:43.639 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-03-21T13:16:43.639 INFO:teuthology.orchestra.run.smithi144.stdout:Install 1 Package 2024-03-21T13:16:43.639 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:16:43.657 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 3.1 M 2024-03-21T13:16:43.657 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 9.8 M 2024-03-21T13:16:43.657 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-03-21T13:16:45.342 INFO:teuthology.orchestra.run.smithi144.stdout:runc-1.1.12-2.el9.x86_64.rpm 2.0 MB/s | 3.1 MB 00:01 2024-03-21T13:16:45.342 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-03-21T13:16:45.342 INFO:teuthology.orchestra.run.smithi144.stdout:Total 1.9 MB/s | 3.1 MB 00:01 2024-03-21T13:16:45.379 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-03-21T13:16:45.385 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-03-21T13:16:45.386 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-03-21T13:16:45.466 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-03-21T13:16:45.466 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-03-21T13:16:45.675 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-03-21T13:16:45.777 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : runc-4:1.1.12-2.el9.x86_64 1/1 2024-03-21T13:16:46.366 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: runc-4:1.1.12-2.el9.x86_64 1/1 2024-03-21T13:16:46.664 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : runc-4:1.1.12-2.el9.x86_64 1/1 2024-03-21T13:16:46.664 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:16:46.664 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-03-21T13:16:46.664 INFO:teuthology.orchestra.run.smithi144.stdout: runc-4:1.1.12-2.el9.x86_64 2024-03-21T13:16:46.664 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:16:46.664 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-03-21T13:16:46.937 DEBUG:teuthology.parallel:result is None 2024-03-21T13:16:49.960 INFO:teuthology.orchestra.run.smithi097.stdout:runc-1.1.12-2.el9.x86_64.rpm 529 kB/s | 3.1 MB 00:06 2024-03-21T13:16:49.960 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-03-21T13:16:49.960 INFO:teuthology.orchestra.run.smithi097.stdout:Total 506 kB/s | 3.1 MB 00:06 2024-03-21T13:16:49.999 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-03-21T13:16:50.005 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-03-21T13:16:50.005 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-03-21T13:16:50.088 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-03-21T13:16:50.088 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-03-21T13:16:50.313 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-03-21T13:16:50.413 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : runc-4:1.1.12-2.el9.x86_64 1/1 2024-03-21T13:16:50.931 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: runc-4:1.1.12-2.el9.x86_64 1/1 2024-03-21T13:16:51.273 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : runc-4:1.1.12-2.el9.x86_64 1/1 2024-03-21T13:16:51.273 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:16:51.273 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-03-21T13:16:51.274 INFO:teuthology.orchestra.run.smithi097.stdout: runc-4:1.1.12-2.el9.x86_64 2024-03-21T13:16:51.274 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:16:51.274 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-21T13:16:51.524 DEBUG:teuthology.parallel:result is None 2024-03-21T13:16:51.524 INFO:teuthology.run_tasks:Running task install... 2024-03-21T13:16:51.536 DEBUG:teuthology.task.install:project ceph 2024-03-21T13:16:51.536 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '960fbbe4645c2becc8084669be194c95183c7753'}, 'extra_packages': ['cephadm']} 2024-03-21T13:16:51.536 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '960fbbe4645c2becc8084669be194c95183c7753'} 2024-03-21T13:16:51.536 INFO:teuthology.task.install:Using flavor: default 2024-03-21T13:16:51.542 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-21T13:16:51.542 INFO:teuthology.task.install:extra packages: [] 2024-03-21T13:16:51.542 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '960fbbe4645c2becc8084669be194c95183c7753', 'tag': None, 'wait_for_package': False} 2024-03-21T13:16:51.542 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=960fbbe4645c2becc8084669be194c95183c7753 2024-03-21T13:16:51.544 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '960fbbe4645c2becc8084669be194c95183c7753', 'tag': None, 'wait_for_package': False} 2024-03-21T13:16:51.544 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=960fbbe4645c2becc8084669be194c95183c7753 2024-03-21T13:16:51.775 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-guits-testing-2024-03-21-0818/960fbbe4645c2becc8084669be194c95183c7753/centos/9/flavors/default/ 2024-03-21T13:16:51.775 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2267.g960fbbe4 2024-03-21T13:16:51.780 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-guits-testing-2024-03-21-0818/960fbbe4645c2becc8084669be194c95183c7753/centos/9/flavors/default/ 2024-03-21T13:16:51.780 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2267.g960fbbe4 2024-03-21T13:16:51.891 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-guits-testing-2024-03-21-0818/960fbbe4645c2becc8084669be194c95183c7753/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-guits-testing-2024-03-21-0818/960fbbe4645c2becc8084669be194c95183c7753/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-guits-testing-2024-03-21-0818/960fbbe4645c2becc8084669be194c95183c7753/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-21T13:16:51.891 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-21T13:16:51.891 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-21T13:16:51.923 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-03-21T13:16:51.924 DEBUG:teuthology.orchestra.run.smithi144:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/960fbbe4645c2becc8084669be194c95183c7753/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-21T13:16:51.945 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-guits-testing-2024-03-21-0818/960fbbe4645c2becc8084669be194c95183c7753/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-guits-testing-2024-03-21-0818/960fbbe4645c2becc8084669be194c95183c7753/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-guits-testing-2024-03-21-0818/960fbbe4645c2becc8084669be194c95183c7753/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-21T13:16:51.946 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-21T13:16:51.946 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-21T13:16:51.972 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-03-21T13:16:51.973 DEBUG:teuthology.orchestra.run.smithi097:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/960fbbe4645c2becc8084669be194c95183c7753/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-21T13:16:51.996 DEBUG:teuthology.orchestra.run.smithi144:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-21T13:16:52.040 DEBUG:teuthology.orchestra.run.smithi097:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-21T13:16:52.079 DEBUG:teuthology.orchestra.run.smithi144:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-21T13:16:52.119 DEBUG:teuthology.orchestra.run.smithi097:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-21T13:16:52.146 INFO:teuthology.orchestra.run.smithi144.stdout:check_obsoletes = 1 2024-03-21T13:16:52.148 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum clean all 2024-03-21T13:16:52.185 INFO:teuthology.orchestra.run.smithi097.stdout:check_obsoletes = 1 2024-03-21T13:16:52.187 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum clean all 2024-03-21T13:16:52.383 INFO:teuthology.orchestra.run.smithi144.stdout:74 files removed 2024-03-21T13:16:52.410 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-radosgw 2024-03-21T13:16:52.434 INFO:teuthology.orchestra.run.smithi097.stdout:74 files removed 2024-03-21T13:16:52.461 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph-radosgw 2024-03-21T13:16:53.146 INFO:teuthology.orchestra.run.smithi144.stdout:ceph packages for x86_64 190 kB/s | 83 kB 00:00 2024-03-21T13:16:53.282 INFO:teuthology.orchestra.run.smithi097.stdout:ceph packages for x86_64 159 kB/s | 83 kB 00:00 2024-03-21T13:16:53.514 INFO:teuthology.orchestra.run.smithi144.stdout:ceph noarch packages 37 kB/s | 12 kB 00:00 2024-03-21T13:16:53.749 INFO:teuthology.orchestra.run.smithi097.stdout:ceph noarch packages 28 kB/s | 12 kB 00:00 2024-03-21T13:16:53.906 INFO:teuthology.orchestra.run.smithi144.stdout:ceph source packages 5.4 kB/s | 1.9 kB 00:00 2024-03-21T13:16:54.208 INFO:teuthology.orchestra.run.smithi097.stdout:ceph source packages 4.5 kB/s | 1.9 kB 00:00 2024-03-21T13:16:54.625 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS-9 - AppStream 9.8 MB/s | 6.7 MB 00:00 2024-03-21T13:16:54.652 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS-9 - AppStream 16 MB/s | 6.7 MB 00:00 2024-03-21T13:16:56.083 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS-9 - BaseOS 5.8 MB/s | 2.0 MB 00:00 2024-03-21T13:16:56.210 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS-9 - BaseOS 4.7 MB/s | 2.0 MB 00:00 2024-03-21T13:16:57.260 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-03-21T13:16:59.363 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - AppStream 18 MB/s | 19 MB 00:01 2024-03-21T13:17:04.387 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - CRB 3.8 MB/s | 6.1 MB 00:01 2024-03-21T13:17:05.956 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - BaseOS 863 kB/s | 8.0 MB 00:09 2024-03-21T13:17:06.179 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 16 kB 00:00 2024-03-21T13:17:07.625 INFO:teuthology.orchestra.run.smithi097.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-03-21T13:17:12.556 INFO:teuthology.orchestra.run.smithi097.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-03-21T13:17:15.008 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-21T13:17:15.011 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-21T13:17:15.011 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2024-03-21T13:17:15.011 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-21T13:17:15.011 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-03-21T13:17:15.011 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-radosgw x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 9.9 M 2024-03-21T13:17:15.011 INFO:teuthology.orchestra.run.smithi097.stdout:Upgrading: 2024-03-21T13:17:15.011 INFO:teuthology.orchestra.run.smithi097.stdout: librados2 x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 3.4 M 2024-03-21T13:17:15.011 INFO:teuthology.orchestra.run.smithi097.stdout: librbd1 x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 3.1 M 2024-03-21T13:17:15.011 INFO:teuthology.orchestra.run.smithi097.stdout:Installing dependencies: 2024-03-21T13:17:15.011 INFO:teuthology.orchestra.run.smithi097.stdout: boost-program-options 2024-03-21T13:17:15.011 INFO:teuthology.orchestra.run.smithi097.stdout: x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-21T13:17:15.011 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-base x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 5.4 M 2024-03-21T13:17:15.011 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-common x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 20 M 2024-03-21T13:17:15.012 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-selinux x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 25 k 2024-03-21T13:17:15.012 INFO:teuthology.orchestra.run.smithi097.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-21T13:17:15.012 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-21T13:17:15.012 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-21T13:17:15.012 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs2 x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 746 k 2024-03-21T13:17:15.012 INFO:teuthology.orchestra.run.smithi097.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-21T13:17:15.012 INFO:teuthology.orchestra.run.smithi097.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-21T13:17:15.012 INFO:teuthology.orchestra.run.smithi097.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-21T13:17:15.012 INFO:teuthology.orchestra.run.smithi097.stdout: libradosstriper1 x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 501 k 2024-03-21T13:17:15.012 INFO:teuthology.orchestra.run.smithi097.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-21T13:17:15.012 INFO:teuthology.orchestra.run.smithi097.stdout: librgw2 x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 5.1 M 2024-03-21T13:17:15.012 INFO:teuthology.orchestra.run.smithi097.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-21T13:17:15.012 INFO:teuthology.orchestra.run.smithi097.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-21T13:17:15.012 INFO:teuthology.orchestra.run.smithi097.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-21T13:17:15.012 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-argparse 2024-03-21T13:17:15.012 INFO:teuthology.orchestra.run.smithi097.stdout: x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 45 k 2024-03-21T13:17:15.013 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-common x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 131 k 2024-03-21T13:17:15.013 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cephfs x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 162 k 2024-03-21T13:17:15.013 INFO:teuthology.orchestra.run.smithi097.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-21T13:17:15.013 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rados x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 320 k 2024-03-21T13:17:15.013 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rbd x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 298 k 2024-03-21T13:17:15.013 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rgw x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 99 k 2024-03-21T13:17:15.013 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-21T13:17:15.013 INFO:teuthology.orchestra.run.smithi097.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-21T13:17:15.013 INFO:teuthology.orchestra.run.smithi097.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-21T13:17:15.013 INFO:teuthology.orchestra.run.smithi097.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-21T13:17:15.013 INFO:teuthology.orchestra.run.smithi097.stdout:Installing weak dependencies: 2024-03-21T13:17:15.013 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-21T13:17:15.013 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:17:15.013 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-03-21T13:17:15.013 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-21T13:17:15.013 INFO:teuthology.orchestra.run.smithi097.stdout:Install 30 Packages 2024-03-21T13:17:15.014 INFO:teuthology.orchestra.run.smithi097.stdout:Upgrade 2 Packages 2024-03-21T13:17:15.014 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:17:15.014 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 59 M 2024-03-21T13:17:15.014 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-03-21T13:17:16.582 INFO:teuthology.orchestra.run.smithi097.stdout:(1/32): ceph-common-19.0.0-2267.g960fbbe4.el9.x 15 MB/s | 20 MB 00:01 2024-03-21T13:17:16.632 INFO:teuthology.orchestra.run.smithi097.stdout:(2/32): ceph-selinux-19.0.0-2267.g960fbbe4.el9. 506 kB/s | 25 kB 00:00 2024-03-21T13:17:16.757 INFO:teuthology.orchestra.run.smithi097.stdout:(3/32): libcephfs2-19.0.0-2267.g960fbbe4.el9.x8 5.8 MB/s | 746 kB 00:00 2024-03-21T13:17:16.824 INFO:teuthology.orchestra.run.smithi097.stdout:(4/32): libradosstriper1-19.0.0-2267.g960fbbe4. 7.3 MB/s | 501 kB 00:00 2024-03-21T13:17:17.072 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - AppStream 1.9 MB/s | 19 MB 00:10 2024-03-21T13:17:17.124 INFO:teuthology.orchestra.run.smithi097.stdout:(5/32): ceph-base-19.0.0-2267.g960fbbe4.el9.x86 2.8 MB/s | 5.4 MB 00:01 2024-03-21T13:17:17.191 INFO:teuthology.orchestra.run.smithi097.stdout:(6/32): librgw2-19.0.0-2267.g960fbbe4.el9.x86_6 14 MB/s | 5.1 MB 00:00 2024-03-21T13:17:17.216 INFO:teuthology.orchestra.run.smithi097.stdout:(7/32): python3-ceph-argparse-19.0.0-2267.g960f 492 kB/s | 45 kB 00:00 2024-03-21T13:17:17.233 INFO:teuthology.orchestra.run.smithi097.stdout:(8/32): python3-ceph-common-19.0.0-2267.g960fbb 3.1 MB/s | 131 kB 00:00 2024-03-21T13:17:17.275 INFO:teuthology.orchestra.run.smithi097.stdout:(9/32): python3-cephfs-19.0.0-2267.g960fbbe4.el 2.7 MB/s | 162 kB 00:00 2024-03-21T13:17:17.301 INFO:teuthology.orchestra.run.smithi097.stdout:(10/32): python3-rados-19.0.0-2267.g960fbbe4.el 4.7 MB/s | 320 kB 00:00 2024-03-21T13:17:17.342 INFO:teuthology.orchestra.run.smithi097.stdout:(11/32): python3-rbd-19.0.0-2267.g960fbbe4.el9. 4.4 MB/s | 298 kB 00:00 2024-03-21T13:17:17.368 INFO:teuthology.orchestra.run.smithi097.stdout:(12/32): python3-rgw-19.0.0-2267.g960fbbe4.el9. 1.5 MB/s | 99 kB 00:00 2024-03-21T13:17:17.485 INFO:teuthology.orchestra.run.smithi097.stdout:(13/32): boost-program-options-1.75.0-8.el9.x86 747 kB/s | 106 kB 00:00 2024-03-21T13:17:17.510 INFO:teuthology.orchestra.run.smithi097.stdout:(14/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.1 MB/s | 159 kB 00:00 2024-03-21T13:17:17.543 INFO:teuthology.orchestra.run.smithi097.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 756 kB/s | 44 kB 00:00 2024-03-21T13:17:17.594 INFO:teuthology.orchestra.run.smithi097.stdout:(16/32): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 661 kB 00:00 2024-03-21T13:17:17.619 INFO:teuthology.orchestra.run.smithi097.stdout:(17/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.7 MB/s | 282 kB 00:00 2024-03-21T13:17:17.645 INFO:teuthology.orchestra.run.smithi097.stdout:(18/32): python3-prettytable-0.7.2-27.el9.noarc 827 kB/s | 41 kB 00:00 2024-03-21T13:17:17.753 INFO:teuthology.orchestra.run.smithi097.stdout:(19/32): qatlib-service-23.11.0-1.el9.x86_64.rp 322 kB/s | 35 kB 00:00 2024-03-21T13:17:17.803 INFO:teuthology.orchestra.run.smithi097.stdout:(20/32): qatzip-libs-1.1.2-2.el9.x86_64.rpm 889 kB/s | 44 kB 00:00 2024-03-21T13:17:17.829 INFO:teuthology.orchestra.run.smithi097.stdout:(21/32): qatlib-23.11.0-1.el9.x86_64.rpm 1.0 MB/s | 216 kB 00:00 2024-03-21T13:17:18.254 INFO:teuthology.orchestra.run.smithi097.stdout:(22/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 686 kB/s | 309 kB 00:00 2024-03-21T13:17:18.337 INFO:teuthology.orchestra.run.smithi097.stdout:(23/32): libarrow-doc-9.0.0-10.el9.noarch.rpm 312 kB/s | 26 kB 00:00 2024-03-21T13:17:18.438 INFO:teuthology.orchestra.run.smithi097.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 484 kB/s | 49 kB 00:00 2024-03-21T13:17:18.522 INFO:teuthology.orchestra.run.smithi097.stdout:(25/32): libarrow-9.0.0-10.el9.x86_64.rpm 6.4 MB/s | 4.4 MB 00:00 2024-03-21T13:17:18.547 INFO:teuthology.orchestra.run.smithi097.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 620 kB/s | 67 kB 00:00 2024-03-21T13:17:18.680 INFO:teuthology.orchestra.run.smithi097.stdout:(27/32): parquet-libs-9.0.0-10.el9.x86_64.rpm 5.2 MB/s | 838 kB 00:00 2024-03-21T13:17:18.706 INFO:teuthology.orchestra.run.smithi097.stdout:(28/32): re2-20211101-3.el9.x86_64.rpm 1.2 MB/s | 192 kB 00:00 2024-03-21T13:17:18.806 INFO:teuthology.orchestra.run.smithi097.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-03-21T13:17:19.131 INFO:teuthology.orchestra.run.smithi097.stdout:(30/32): librbd1-19.0.0-2267.g960fbbe4.el9.x86_ 9.7 MB/s | 3.1 MB 00:00 2024-03-21T13:17:19.515 INFO:teuthology.orchestra.run.smithi097.stdout:(31/32): librados2-19.0.0-2267.g960fbbe4.el9.x8 4.2 MB/s | 3.4 MB 00:00 2024-03-21T13:17:19.623 INFO:teuthology.orchestra.run.smithi097.stdout:(32/32): ceph-radosgw-19.0.0-2267.g960fbbe4.el9 2.2 MB/s | 9.9 MB 00:04 2024-03-21T13:17:19.629 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-03-21T13:17:19.629 INFO:teuthology.orchestra.run.smithi097.stdout:Total 13 MB/s | 59 MB 00:04 2024-03-21T13:17:19.661 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-03-21T13:17:19.699 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-03-21T13:17:19.699 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-03-21T13:17:20.195 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-03-21T13:17:20.195 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-03-21T13:17:21.056 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.1 MB 00:00 2024-03-21T13:17:21.637 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-03-21T13:17:21.725 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-03-21T13:17:21.761 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-03-21T13:17:21.786 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-03-21T13:17:21.809 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 4/34 2024-03-21T13:17:21.824 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 4/34 2024-03-21T13:17:21.970 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 5/34 2024-03-21T13:17:22.022 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 5/34 2024-03-21T13:17:22.410 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-03-21T13:17:22.447 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : librados2-2:19.0.0-2267.g960fbbe4.el9.x86_64 7/34 2024-03-21T13:17:22.546 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librados2-2:19.0.0-2267.g960fbbe4.el9.x86_64 7/34 2024-03-21T13:17:22.557 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - Extras packages 49 kB/s | 16 kB 00:00 2024-03-21T13:17:22.641 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rados-2:19.0.0-2267.g960fbbe4.el9.x86_64 8/34 2024-03-21T13:17:22.665 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephfs2-2:19.0.0-2267.g960fbbe4.el9.x86_64 9/34 2024-03-21T13:17:22.719 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libcephfs2-2:19.0.0-2267.g960fbbe4.el9.x86_64 9/34 2024-03-21T13:17:22.748 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-03-21T13:17:22.784 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ceph-argparse-2:19.0.0-2267.g960fbbe4.el9. 11/34 2024-03-21T13:17:22.852 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cephfs-2:19.0.0-2267.g960fbbe4.el9.x86_64 12/34 2024-03-21T13:17:22.876 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libradosstriper1-2:19.0.0-2267.g960fbbe4.el9.x86_6 13/34 2024-03-21T13:17:22.943 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2267.g960fbbe4.el9.x86_6 13/34 2024-03-21T13:17:22.974 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 14/34 2024-03-21T13:17:23.082 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-03-21T13:17:23.126 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 16/34 2024-03-21T13:17:23.166 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-03-21T13:17:23.198 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-03-21T13:17:23.227 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 19/34 2024-03-21T13:17:23.260 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-03-21T13:17:23.434 INFO:teuthology.orchestra.run.smithi144.stdout:Extra Packages for Enterprise Linux 27 MB/s | 21 MB 00:00 2024-03-21T13:17:23.578 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-03-21T13:17:23.615 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : librbd1-2:19.0.0-2267.g960fbbe4.el9.x86_64 22/34 2024-03-21T13:17:23.689 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librbd1-2:19.0.0-2267.g960fbbe4.el9.x86_64 22/34 2024-03-21T13:17:23.719 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rbd-2:19.0.0-2267.g960fbbe4.el9.x86_64 23/34 2024-03-21T13:17:23.772 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-03-21T13:17:24.227 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 25/34 2024-03-21T13:17:24.259 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librgw2-2:19.0.0-2267.g960fbbe4.el9.x86_64 26/34 2024-03-21T13:17:24.313 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librgw2-2:19.0.0-2267.g960fbbe4.el9.x86_64 26/34 2024-03-21T13:17:24.382 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rgw-2:19.0.0-2267.g960fbbe4.el9.x86_64 27/34 2024-03-21T13:17:24.411 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ceph-common-2:19.0.0-2267.g960fbbe4.el9.x8 28/34 2024-03-21T13:17:26.669 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-common-2:19.0.0-2267.g960fbbe4.el9.x86_64 29/34 2024-03-21T13:17:26.708 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-common-2:19.0.0-2267.g960fbbe4.el9.x86_64 29/34 2024-03-21T13:17:27.267 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-common-2:19.0.0-2267.g960fbbe4.el9.x86_64 29/34 2024-03-21T13:17:27.303 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-base-2:19.0.0-2267.g960fbbe4.el9.x86_64 30/34 2024-03-21T13:17:27.419 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-base-2:19.0.0-2267.g960fbbe4.el9.x86_64 30/34 2024-03-21T13:17:27.419 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-21T13:17:27.419 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-21T13:17:27.419 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:17:27.454 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-selinux-2:19.0.0-2267.g960fbbe4.el9.x86_64 31/34 2024-03-21T13:17:28.583 INFO:teuthology.orchestra.run.smithi144.stdout:lab-extras 6.4 kB/s | 1.7 kB 00:00 2024-03-21T13:17:30.984 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-03-21T13:17:30.987 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-03-21T13:17:30.987 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repository Size 2024-03-21T13:17:30.987 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-03-21T13:17:30.987 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-03-21T13:17:30.987 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-radosgw x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 9.9 M 2024-03-21T13:17:30.987 INFO:teuthology.orchestra.run.smithi144.stdout:Upgrading: 2024-03-21T13:17:30.987 INFO:teuthology.orchestra.run.smithi144.stdout: librados2 x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 3.4 M 2024-03-21T13:17:30.987 INFO:teuthology.orchestra.run.smithi144.stdout: librbd1 x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 3.1 M 2024-03-21T13:17:30.987 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2024-03-21T13:17:30.987 INFO:teuthology.orchestra.run.smithi144.stdout: boost-program-options 2024-03-21T13:17:30.987 INFO:teuthology.orchestra.run.smithi144.stdout: x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-21T13:17:30.988 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-base x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 5.4 M 2024-03-21T13:17:30.988 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-common x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 20 M 2024-03-21T13:17:30.988 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-selinux x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 25 k 2024-03-21T13:17:30.988 INFO:teuthology.orchestra.run.smithi144.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-21T13:17:30.988 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-21T13:17:30.988 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-21T13:17:30.988 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs2 x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 746 k 2024-03-21T13:17:30.988 INFO:teuthology.orchestra.run.smithi144.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-21T13:17:30.988 INFO:teuthology.orchestra.run.smithi144.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-21T13:17:30.988 INFO:teuthology.orchestra.run.smithi144.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-21T13:17:30.988 INFO:teuthology.orchestra.run.smithi144.stdout: libradosstriper1 x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 501 k 2024-03-21T13:17:30.988 INFO:teuthology.orchestra.run.smithi144.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-21T13:17:30.988 INFO:teuthology.orchestra.run.smithi144.stdout: librgw2 x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 5.1 M 2024-03-21T13:17:30.988 INFO:teuthology.orchestra.run.smithi144.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-21T13:17:30.989 INFO:teuthology.orchestra.run.smithi144.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-21T13:17:30.989 INFO:teuthology.orchestra.run.smithi144.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-21T13:17:30.989 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-argparse 2024-03-21T13:17:30.989 INFO:teuthology.orchestra.run.smithi144.stdout: x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 45 k 2024-03-21T13:17:30.989 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-common x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 131 k 2024-03-21T13:17:30.989 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cephfs x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 162 k 2024-03-21T13:17:30.989 INFO:teuthology.orchestra.run.smithi144.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-21T13:17:30.989 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rados x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 320 k 2024-03-21T13:17:30.989 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rbd x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 298 k 2024-03-21T13:17:30.989 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rgw x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 99 k 2024-03-21T13:17:30.989 INFO:teuthology.orchestra.run.smithi144.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-21T13:17:30.989 INFO:teuthology.orchestra.run.smithi144.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-21T13:17:30.989 INFO:teuthology.orchestra.run.smithi144.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-21T13:17:30.989 INFO:teuthology.orchestra.run.smithi144.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-21T13:17:30.989 INFO:teuthology.orchestra.run.smithi144.stdout:Installing weak dependencies: 2024-03-21T13:17:30.989 INFO:teuthology.orchestra.run.smithi144.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-21T13:17:30.990 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:17:30.990 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-03-21T13:17:30.990 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-03-21T13:17:30.990 INFO:teuthology.orchestra.run.smithi144.stdout:Install 30 Packages 2024-03-21T13:17:30.990 INFO:teuthology.orchestra.run.smithi144.stdout:Upgrade 2 Packages 2024-03-21T13:17:30.990 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:17:30.990 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 59 M 2024-03-21T13:17:30.990 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-03-21T13:17:32.525 INFO:teuthology.orchestra.run.smithi144.stdout:(1/32): ceph-radosgw-19.0.0-2267.g960fbbe4.el9. 9.5 MB/s | 9.9 MB 00:01 2024-03-21T13:17:32.576 INFO:teuthology.orchestra.run.smithi144.stdout:(2/32): ceph-selinux-19.0.0-2267.g960fbbe4.el9. 499 kB/s | 25 kB 00:00 2024-03-21T13:17:32.676 INFO:teuthology.orchestra.run.smithi144.stdout:(3/32): libcephfs2-19.0.0-2267.g960fbbe4.el9.x8 7.3 MB/s | 746 kB 00:00 2024-03-21T13:17:32.760 INFO:teuthology.orchestra.run.smithi144.stdout:(4/32): libradosstriper1-19.0.0-2267.g960fbbe4. 5.9 MB/s | 501 kB 00:00 2024-03-21T13:17:33.236 INFO:teuthology.orchestra.run.smithi144.stdout:(5/32): librgw2-19.0.0-2267.g960fbbe4.el9.x86_6 11 MB/s | 5.1 MB 00:00 2024-03-21T13:17:33.302 INFO:teuthology.orchestra.run.smithi144.stdout:(6/32): ceph-base-19.0.0-2267.g960fbbe4.el9.x86 2.9 MB/s | 5.4 MB 00:01 2024-03-21T13:17:33.319 INFO:teuthology.orchestra.run.smithi144.stdout:(7/32): python3-ceph-argparse-19.0.0-2267.g960f 539 kB/s | 45 kB 00:00 2024-03-21T13:17:33.378 INFO:teuthology.orchestra.run.smithi144.stdout:(8/32): python3-ceph-common-19.0.0-2267.g960fbb 1.7 MB/s | 131 kB 00:00 2024-03-21T13:17:33.404 INFO:teuthology.orchestra.run.smithi144.stdout:(9/32): python3-cephfs-19.0.0-2267.g960fbbe4.el 1.9 MB/s | 162 kB 00:00 2024-03-21T13:17:33.454 INFO:teuthology.orchestra.run.smithi144.stdout:(10/32): python3-rbd-19.0.0-2267.g960fbbe4.el9. 5.8 MB/s | 298 kB 00:00 2024-03-21T13:17:33.479 INFO:teuthology.orchestra.run.smithi144.stdout:(11/32): python3-rados-19.0.0-2267.g960fbbe4.el 3.1 MB/s | 320 kB 00:00 2024-03-21T13:17:33.504 INFO:teuthology.orchestra.run.smithi144.stdout:(12/32): python3-rgw-19.0.0-2267.g960fbbe4.el9. 1.9 MB/s | 99 kB 00:00 2024-03-21T13:17:33.638 INFO:teuthology.orchestra.run.smithi144.stdout:(13/32): boost-program-options-1.75.0-8.el9.x86 668 kB/s | 106 kB 00:00 2024-03-21T13:17:33.663 INFO:teuthology.orchestra.run.smithi144.stdout:(14/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 159 kB 00:00 2024-03-21T13:17:33.688 INFO:teuthology.orchestra.run.smithi144.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 885 kB/s | 44 kB 00:00 2024-03-21T13:17:33.739 INFO:teuthology.orchestra.run.smithi144.stdout:(16/32): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 661 kB 00:00 2024-03-21T13:17:33.764 INFO:teuthology.orchestra.run.smithi144.stdout:(17/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.7 MB/s | 282 kB 00:00 2024-03-21T13:17:33.789 INFO:teuthology.orchestra.run.smithi144.stdout:(18/32): python3-prettytable-0.7.2-27.el9.noarc 825 kB/s | 41 kB 00:00 2024-03-21T13:17:33.873 INFO:teuthology.orchestra.run.smithi144.stdout:(19/32): qatlib-service-23.11.0-1.el9.x86_64.rp 419 kB/s | 35 kB 00:00 2024-03-21T13:17:33.932 INFO:teuthology.orchestra.run.smithi144.stdout:(20/32): qatzip-libs-1.1.2-2.el9.x86_64.rpm 764 kB/s | 44 kB 00:00 2024-03-21T13:17:33.957 INFO:teuthology.orchestra.run.smithi144.stdout:(21/32): qatlib-23.11.0-1.el9.x86_64.rpm 1.1 MB/s | 216 kB 00:00 2024-03-21T13:17:34.765 INFO:teuthology.orchestra.run.smithi144.stdout:(22/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 371 kB/s | 309 kB 00:00 2024-03-21T13:17:34.974 INFO:teuthology.orchestra.run.smithi144.stdout:(23/32): ceph-common-19.0.0-2267.g960fbbe4.el9. 5.9 MB/s | 20 MB 00:03 2024-03-21T13:17:34.999 INFO:teuthology.orchestra.run.smithi144.stdout:(24/32): libarrow-doc-9.0.0-10.el9.noarch.rpm 111 kB/s | 26 kB 00:00 2024-03-21T13:17:35.116 INFO:teuthology.orchestra.run.smithi144.stdout:(25/32): libunwind-1.6.2-1.el9.x86_64.rpm 577 kB/s | 67 kB 00:00 2024-03-21T13:17:35.583 INFO:teuthology.orchestra.run.smithi144.stdout:(26/32): liboath-2.6.7-2.el9.x86_64.rpm 80 kB/s | 49 kB 00:00 2024-03-21T13:17:35.858 INFO:teuthology.orchestra.run.smithi144.stdout:(27/32): re2-20211101-3.el9.x86_64.rpm 697 kB/s | 192 kB 00:00 2024-03-21T13:17:36.183 INFO:teuthology.orchestra.run.smithi144.stdout:(28/32): parquet-libs-9.0.0-10.el9.x86_64.rpm 785 kB/s | 838 kB 00:01 2024-03-21T13:17:36.834 INFO:teuthology.orchestra.run.smithi144.stdout:(29/32): librados2-19.0.0-2267.g960fbbe4.el9.x8 5.2 MB/s | 3.4 MB 00:00 2024-03-21T13:17:37.254 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2267.g960fbbe4.el9.x86_64 31/34 2024-03-21T13:17:37.255 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /sys 2024-03-21T13:17:37.255 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /proc 2024-03-21T13:17:37.255 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /mnt 2024-03-21T13:17:37.255 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /var/tmp 2024-03-21T13:17:37.255 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /home 2024-03-21T13:17:37.255 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /root 2024-03-21T13:17:37.255 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /tmp 2024-03-21T13:17:37.255 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:17:37.392 INFO:teuthology.orchestra.run.smithi144.stdout:(30/32): librbd1-19.0.0-2267.g960fbbe4.el9.x86_ 5.6 MB/s | 3.1 MB 00:00 2024-03-21T13:17:38.192 INFO:teuthology.orchestra.run.smithi144.stdout:(31/32): thrift-0.15.0-2.el9.x86_64.rpm 694 kB/s | 1.6 MB 00:02 2024-03-21T13:17:38.219 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-radosgw-2:19.0.0-2267.g960fbbe4.el9.x86_64 32/34 2024-03-21T13:17:38.243 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2267.g960fbbe4.el9.x86_64 32/34 2024-03-21T13:17:38.243 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T13:17:38.243 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-21T13:17:38.243 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-21T13:17:38.243 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-21T13:17:38.243 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:17:38.245 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-03-21T13:17:38.330 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-03-21T13:17:38.330 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-03-21T13:17:39.495 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-03-21T13:17:39.496 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-base-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/34 2024-03-21T13:17:39.496 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-common-2:19.0.0-2267.g960fbbe4.el9.x86_64 2/34 2024-03-21T13:17:39.496 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-radosgw-2:19.0.0-2267.g960fbbe4.el9.x86_64 3/34 2024-03-21T13:17:39.496 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-selinux-2:19.0.0-2267.g960fbbe4.el9.x86_64 4/34 2024-03-21T13:17:39.496 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephfs2-2:19.0.0-2267.g960fbbe4.el9.x86_64 5/34 2024-03-21T13:17:39.496 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libradosstriper1-2:19.0.0-2267.g960fbbe4.el9.x86_6 6/34 2024-03-21T13:17:39.496 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librgw2-2:19.0.0-2267.g960fbbe4.el9.x86_64 7/34 2024-03-21T13:17:39.496 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2267.g960fbbe4.el9. 8/34 2024-03-21T13:17:39.496 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ceph-common-2:19.0.0-2267.g960fbbe4.el9.x8 9/34 2024-03-21T13:17:39.496 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cephfs-2:19.0.0-2267.g960fbbe4.el9.x86_64 10/34 2024-03-21T13:17:39.497 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rados-2:19.0.0-2267.g960fbbe4.el9.x86_64 11/34 2024-03-21T13:17:39.497 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rbd-2:19.0.0-2267.g960fbbe4.el9.x86_64 12/34 2024-03-21T13:17:39.497 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rgw-2:19.0.0-2267.g960fbbe4.el9.x86_64 13/34 2024-03-21T13:17:39.497 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-03-21T13:17:39.497 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-03-21T13:17:39.497 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-03-21T13:17:39.497 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-03-21T13:17:39.497 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-03-21T13:17:39.497 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-03-21T13:17:39.497 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 20/34 2024-03-21T13:17:39.497 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 21/34 2024-03-21T13:17:39.497 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 22/34 2024-03-21T13:17:39.497 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-03-21T13:17:39.497 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 24/34 2024-03-21T13:17:39.498 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 25/34 2024-03-21T13:17:39.498 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-03-21T13:17:39.498 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-03-21T13:17:39.498 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 28/34 2024-03-21T13:17:39.498 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-03-21T13:17:39.498 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-03-21T13:17:39.498 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados2-2:19.0.0-2267.g960fbbe4.el9.x86_64 31/34 2024-03-21T13:17:39.498 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-03-21T13:17:39.498 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librbd1-2:19.0.0-2267.g960fbbe4.el9.x86_64 33/34 2024-03-21T13:17:40.449 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-03-21T13:17:40.450 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:17:40.450 INFO:teuthology.orchestra.run.smithi097.stdout:Upgraded: 2024-03-21T13:17:40.450 INFO:teuthology.orchestra.run.smithi097.stdout: librados2-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:17:40.450 INFO:teuthology.orchestra.run.smithi097.stdout: librbd1-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:17:40.450 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-03-21T13:17:40.450 INFO:teuthology.orchestra.run.smithi097.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-21T13:17:40.450 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-base-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:17:40.450 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-common-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:17:40.450 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-radosgw-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:17:40.450 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-selinux-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:17:40.450 INFO:teuthology.orchestra.run.smithi097.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-21T13:17:40.451 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-21T13:17:40.451 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-21T13:17:40.451 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs2-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:17:40.451 INFO:teuthology.orchestra.run.smithi097.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-21T13:17:40.451 INFO:teuthology.orchestra.run.smithi097.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-21T13:17:40.451 INFO:teuthology.orchestra.run.smithi097.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-21T13:17:40.451 INFO:teuthology.orchestra.run.smithi097.stdout: libradosstriper1-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:17:40.451 INFO:teuthology.orchestra.run.smithi097.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-21T13:17:40.451 INFO:teuthology.orchestra.run.smithi097.stdout: librgw2-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:17:40.451 INFO:teuthology.orchestra.run.smithi097.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-21T13:17:40.451 INFO:teuthology.orchestra.run.smithi097.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-21T13:17:40.451 INFO:teuthology.orchestra.run.smithi097.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-21T13:17:40.451 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-argparse-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:17:40.451 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-common-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:17:40.452 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cephfs-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:17:40.452 INFO:teuthology.orchestra.run.smithi097.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-21T13:17:40.452 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rados-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:17:40.452 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rbd-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:17:40.452 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rgw-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:17:40.452 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-21T13:17:40.452 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-21T13:17:40.452 INFO:teuthology.orchestra.run.smithi097.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-21T13:17:40.452 INFO:teuthology.orchestra.run.smithi097.stdout: re2-1:20211101-3.el9.x86_64 2024-03-21T13:17:40.452 INFO:teuthology.orchestra.run.smithi097.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-21T13:17:40.452 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:17:40.452 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-21T13:17:40.787 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph-test 2024-03-21T13:17:40.975 INFO:teuthology.orchestra.run.smithi144.stdout:(32/32): libarrow-9.0.0-10.el9.x86_64.rpm 644 kB/s | 4.4 MB 00:07 2024-03-21T13:17:40.982 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-03-21T13:17:40.982 INFO:teuthology.orchestra.run.smithi144.stdout:Total 5.9 MB/s | 59 MB 00:09 2024-03-21T13:17:41.014 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-03-21T13:17:41.052 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-03-21T13:17:41.052 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-03-21T13:17:41.344 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:00:29 ago on Thu 21 Mar 2024 01:17:12 PM UTC. 2024-03-21T13:17:41.493 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-21T13:17:41.494 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-21T13:17:41.494 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2024-03-21T13:17:41.494 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-21T13:17:41.494 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-03-21T13:17:41.494 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-test x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 48 M 2024-03-21T13:17:41.494 INFO:teuthology.orchestra.run.smithi097.stdout:Installing dependencies: 2024-03-21T13:17:41.494 INFO:teuthology.orchestra.run.smithi097.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-21T13:17:41.494 INFO:teuthology.orchestra.run.smithi097.stdout: libcephsqlite x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 163 k 2024-03-21T13:17:41.494 INFO:teuthology.orchestra.run.smithi097.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-21T13:17:41.495 INFO:teuthology.orchestra.run.smithi097.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-21T13:17:41.495 INFO:teuthology.orchestra.run.smithi097.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-21T13:17:41.495 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:17:41.495 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-03-21T13:17:41.495 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-21T13:17:41.495 INFO:teuthology.orchestra.run.smithi097.stdout:Install 6 Packages 2024-03-21T13:17:41.495 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:17:41.495 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 49 M 2024-03-21T13:17:41.496 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 206 M 2024-03-21T13:17:41.496 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-03-21T13:17:41.552 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-03-21T13:17:41.553 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-03-21T13:17:41.743 INFO:teuthology.orchestra.run.smithi097.stdout:(1/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.2 MB/s | 217 kB 00:00 2024-03-21T13:17:41.785 INFO:teuthology.orchestra.run.smithi097.stdout:(2/6): socat-1.7.4.1-5.el9.x86_64.rpm 7.0 MB/s | 300 kB 00:00 2024-03-21T13:17:41.827 INFO:teuthology.orchestra.run.smithi097.stdout:(3/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 63 kB 00:00 2024-03-21T13:17:41.961 INFO:teuthology.orchestra.run.smithi097.stdout:(4/6): libcephsqlite-19.0.0-2267.g960fbbe4.el9. 408 kB/s | 163 kB 00:00 2024-03-21T13:17:42.336 INFO:teuthology.orchestra.run.smithi097.stdout:(5/6): jq-1.6-16.el9.x86_64.rpm 368 kB/s | 187 kB 00:00 2024-03-21T13:17:43.009 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-03-21T13:17:43.101 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-03-21T13:17:43.130 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-03-21T13:17:43.155 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-03-21T13:17:43.178 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 4/34 2024-03-21T13:17:43.194 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 4/34 2024-03-21T13:17:43.335 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 5/34 2024-03-21T13:17:43.383 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 5/34 2024-03-21T13:17:43.711 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-03-21T13:17:43.765 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : librados2-2:19.0.0-2267.g960fbbe4.el9.x86_64 7/34 2024-03-21T13:17:43.856 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librados2-2:19.0.0-2267.g960fbbe4.el9.x86_64 7/34 2024-03-21T13:17:43.944 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rados-2:19.0.0-2267.g960fbbe4.el9.x86_64 8/34 2024-03-21T13:17:43.992 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libcephfs2-2:19.0.0-2267.g960fbbe4.el9.x86_64 9/34 2024-03-21T13:17:44.054 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libcephfs2-2:19.0.0-2267.g960fbbe4.el9.x86_64 9/34 2024-03-21T13:17:44.083 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-03-21T13:17:44.135 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-ceph-argparse-2:19.0.0-2267.g960fbbe4.el9. 11/34 2024-03-21T13:17:44.136 INFO:teuthology.orchestra.run.smithi097.stdout:(6/6): ceph-test-19.0.0-2267.g960fbbe4.el9.x86_ 19 MB/s | 48 MB 00:02 2024-03-21T13:17:44.139 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-03-21T13:17:44.140 INFO:teuthology.orchestra.run.smithi097.stdout:Total 18 MB/s | 49 MB 00:02 2024-03-21T13:17:44.153 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-03-21T13:17:44.174 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-03-21T13:17:44.174 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-03-21T13:17:44.211 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cephfs-2:19.0.0-2267.g960fbbe4.el9.x86_64 12/34 2024-03-21T13:17:44.236 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libradosstriper1-2:19.0.0-2267.g960fbbe4.el9.x86_6 13/34 2024-03-21T13:17:44.299 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2267.g960fbbe4.el9.x86_6 13/34 2024-03-21T13:17:44.334 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 14/34 2024-03-21T13:17:44.443 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-03-21T13:17:44.477 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 16/34 2024-03-21T13:17:44.518 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-03-21T13:17:44.550 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-03-21T13:17:44.558 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-03-21T13:17:44.559 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-03-21T13:17:44.571 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 19/34 2024-03-21T13:17:44.603 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-03-21T13:17:44.917 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-03-21T13:17:44.950 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : librbd1-2:19.0.0-2267.g960fbbe4.el9.x86_64 22/34 2024-03-21T13:17:45.040 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librbd1-2:19.0.0-2267.g960fbbe4.el9.x86_64 22/34 2024-03-21T13:17:45.071 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rbd-2:19.0.0-2267.g960fbbe4.el9.x86_64 23/34 2024-03-21T13:17:45.111 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-03-21T13:17:45.135 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-03-21T13:17:45.195 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2024-03-21T13:17:45.243 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2024-03-21T13:17:45.271 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-21T13:17:45.311 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : jq-1.6-16.el9.x86_64 4/6 2024-03-21T13:17:45.339 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephsqlite-2:19.0.0-2267.g960fbbe4.el9.x86_64 5/6 2024-03-21T13:17:45.572 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 25/34 2024-03-21T13:17:45.594 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librgw2-2:19.0.0-2267.g960fbbe4.el9.x86_64 26/34 2024-03-21T13:17:45.665 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librgw2-2:19.0.0-2267.g960fbbe4.el9.x86_64 26/34 2024-03-21T13:17:45.733 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rgw-2:19.0.0-2267.g960fbbe4.el9.x86_64 27/34 2024-03-21T13:17:45.763 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-ceph-common-2:19.0.0-2267.g960fbbe4.el9.x8 28/34 2024-03-21T13:17:47.976 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-common-2:19.0.0-2267.g960fbbe4.el9.x86_64 29/34 2024-03-21T13:17:48.018 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-common-2:19.0.0-2267.g960fbbe4.el9.x86_64 29/34 2024-03-21T13:17:48.587 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-common-2:19.0.0-2267.g960fbbe4.el9.x86_64 29/34 2024-03-21T13:17:48.630 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-base-2:19.0.0-2267.g960fbbe4.el9.x86_64 30/34 2024-03-21T13:17:48.705 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-base-2:19.0.0-2267.g960fbbe4.el9.x86_64 30/34 2024-03-21T13:17:48.705 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-21T13:17:48.705 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-21T13:17:48.705 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:17:48.739 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-selinux-2:19.0.0-2267.g960fbbe4.el9.x86_64 31/34 2024-03-21T13:17:49.936 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2267.g960fbbe4.el9.x86_64 5/6 2024-03-21T13:17:50.014 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-test-2:19.0.0-2267.g960fbbe4.el9.x86_64 6/6 2024-03-21T13:17:52.397 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-test-2:19.0.0-2267.g960fbbe4.el9.x86_64 6/6 2024-03-21T13:17:52.398 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-test-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/6 2024-03-21T13:17:52.398 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephsqlite-2:19.0.0-2267.g960fbbe4.el9.x86_64 2/6 2024-03-21T13:17:52.398 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-21T13:17:52.398 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 4/6 2024-03-21T13:17:52.398 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/6 2024-03-21T13:17:52.824 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : jq-1.6-16.el9.x86_64 6/6 2024-03-21T13:17:52.883 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:17:52.883 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-03-21T13:17:52.883 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-test-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:17:52.883 INFO:teuthology.orchestra.run.smithi097.stdout: jq-1.6-16.el9.x86_64 2024-03-21T13:17:52.883 INFO:teuthology.orchestra.run.smithi097.stdout: libcephsqlite-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:17:52.883 INFO:teuthology.orchestra.run.smithi097.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-21T13:17:52.883 INFO:teuthology.orchestra.run.smithi097.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-21T13:17:52.883 INFO:teuthology.orchestra.run.smithi097.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-21T13:17:52.883 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:17:52.883 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-21T13:17:53.045 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph 2024-03-21T13:17:53.628 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:00:41 ago on Thu 21 Mar 2024 01:17:12 PM UTC. 2024-03-21T13:17:53.786 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-21T13:17:53.788 INFO:teuthology.orchestra.run.smithi097.stdout:========================================================================================= 2024-03-21T13:17:53.788 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2024-03-21T13:17:53.789 INFO:teuthology.orchestra.run.smithi097.stdout:========================================================================================= 2024-03-21T13:17:53.789 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-03-21T13:17:53.789 INFO:teuthology.orchestra.run.smithi097.stdout: ceph x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 6.6 k 2024-03-21T13:17:53.789 INFO:teuthology.orchestra.run.smithi097.stdout:Installing dependencies: 2024-03-21T13:17:53.789 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mds x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 2.3 M 2024-03-21T13:17:53.789 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 1.5 M 2024-03-21T13:17:53.789 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2267.g960fbbe4.el9 ceph-noarch 248 k 2024-03-21T13:17:53.789 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mon x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 4.7 M 2024-03-21T13:17:53.789 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-osd x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 17 M 2024-03-21T13:17:53.789 INFO:teuthology.orchestra.run.smithi097.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-21T13:17:53.789 INFO:teuthology.orchestra.run.smithi097.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-21T13:17:53.789 INFO:teuthology.orchestra.run.smithi097.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-21T13:17:53.790 INFO:teuthology.orchestra.run.smithi097.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-21T13:17:53.790 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-21T13:17:53.790 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-21T13:17:53.790 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-21T13:17:53.790 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-21T13:17:53.790 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-21T13:17:53.790 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-21T13:17:53.790 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-21T13:17:53.790 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-21T13:17:53.791 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-21T13:17:53.791 INFO:teuthology.orchestra.run.smithi097.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-21T13:17:53.791 INFO:teuthology.orchestra.run.smithi097.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-21T13:17:53.791 INFO:teuthology.orchestra.run.smithi097.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-21T13:17:53.791 INFO:teuthology.orchestra.run.smithi097.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-21T13:17:53.791 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-21T13:17:53.791 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-21T13:17:53.791 INFO:teuthology.orchestra.run.smithi097.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-21T13:17:53.791 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-21T13:17:53.791 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-21T13:17:53.791 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-21T13:17:53.791 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-21T13:17:53.792 INFO:teuthology.orchestra.run.smithi097.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-21T13:17:53.792 INFO:teuthology.orchestra.run.smithi097.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-21T13:17:53.792 INFO:teuthology.orchestra.run.smithi097.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-21T13:17:53.792 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-21T13:17:53.792 INFO:teuthology.orchestra.run.smithi097.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-21T13:17:53.792 INFO:teuthology.orchestra.run.smithi097.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-21T13:17:53.792 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:17:53.792 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-03-21T13:17:53.792 INFO:teuthology.orchestra.run.smithi097.stdout:========================================================================================= 2024-03-21T13:17:53.792 INFO:teuthology.orchestra.run.smithi097.stdout:Install 36 Packages 2024-03-21T13:17:53.792 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:17:53.793 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 30 M 2024-03-21T13:17:53.793 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 107 M 2024-03-21T13:17:53.793 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-03-21T13:17:54.182 INFO:teuthology.orchestra.run.smithi097.stdout:(1/36): ceph-19.0.0-2267.g960fbbe4.el9.x86_64.r 32 kB/s | 6.6 kB 00:00 2024-03-21T13:17:54.582 INFO:teuthology.orchestra.run.smithi097.stdout:(2/36): ceph-mgr-19.0.0-2267.g960fbbe4.el9.x86_ 2.5 MB/s | 1.5 MB 00:00 2024-03-21T13:17:54.641 INFO:teuthology.orchestra.run.smithi097.stdout:(3/36): ceph-mds-19.0.0-2267.g960fbbe4.el9.x86_ 3.5 MB/s | 2.3 MB 00:00 2024-03-21T13:17:54.737 INFO:teuthology.orchestra.run.smithi097.stdout:(4/36): ceph-mgr-modules-core-19.0.0-2267.g960f 2.5 MB/s | 248 kB 00:00 2024-03-21T13:17:54.900 INFO:teuthology.orchestra.run.smithi097.stdout:(5/36): lua-5.4.4-4.el9.x86_64.rpm 1.1 MB/s | 187 kB 00:00 2024-03-21T13:17:54.975 INFO:teuthology.orchestra.run.smithi097.stdout:(6/36): python3-mako-1.1.4-6.el9.noarch.rpm 1.9 MB/s | 144 kB 00:00 2024-03-21T13:17:55.025 INFO:teuthology.orchestra.run.smithi097.stdout:(7/36): python3-markupsafe-1.1.1-12.el9.x86_64. 631 kB/s | 32 kB 00:00 2024-03-21T13:17:55.067 INFO:teuthology.orchestra.run.smithi097.stdout:(8/36): python3-pytz-2021.1-5.el9.noarch.rpm 1.1 MB/s | 47 kB 00:00 2024-03-21T13:17:55.101 INFO:teuthology.orchestra.run.smithi097.stdout:(9/36): python3-toml-0.10.2-6.el9.noarch.rpm 1.1 MB/s | 39 kB 00:00 2024-03-21T13:17:55.151 INFO:teuthology.orchestra.run.smithi097.stdout:(10/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-21T13:17:55.252 INFO:teuthology.orchestra.run.smithi097.stdout:(11/36): ceph-mon-19.0.0-2267.g960fbbe4.el9.x86 4.4 MB/s | 4.7 MB 00:01 2024-03-21T13:17:55.285 INFO:teuthology.orchestra.run.smithi097.stdout:(12/36): python3-cryptography-36.0.1-4.el9.x86_ 8.6 MB/s | 1.1 MB 00:00 2024-03-21T13:17:55.310 INFO:teuthology.orchestra.run.smithi097.stdout:(13/36): python3-ply-3.11-14.el9.noarch.rpm 1.7 MB/s | 103 kB 00:00 2024-03-21T13:17:55.336 INFO:teuthology.orchestra.run.smithi097.stdout:(14/36): python3-pycparser-2.20-6.el9.noarch.rp 2.4 MB/s | 124 kB 00:00 2024-03-21T13:17:55.361 INFO:teuthology.orchestra.run.smithi097.stdout:(15/36): python3-urllib3-1.26.5-5.el9.noarch.rp 3.6 MB/s | 187 kB 00:00 2024-03-21T13:17:55.603 INFO:teuthology.orchestra.run.smithi097.stdout:(16/36): ceph-osd-19.0.0-2267.g960fbbe4.el9.x86 16 MB/s | 17 MB 00:01 2024-03-21T13:17:55.629 INFO:teuthology.orchestra.run.smithi097.stdout:(17/36): lua-devel-5.4.4-4.el9.x86_64.rpm 83 kB/s | 22 kB 00:00 2024-03-21T13:17:55.662 INFO:teuthology.orchestra.run.smithi097.stdout:(18/36): python3-requests-2.25.1-8.el9.noarch.r 384 kB/s | 125 kB 00:00 2024-03-21T13:17:55.737 INFO:teuthology.orchestra.run.smithi097.stdout:(19/36): luarocks-3.9.2-1.el9.noarch.rpm 1.1 MB/s | 151 kB 00:00 2024-03-21T13:17:55.754 INFO:teuthology.orchestra.run.smithi097.stdout:(20/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 346 kB/s | 43 kB 00:00 2024-03-21T13:17:55.796 INFO:teuthology.orchestra.run.smithi097.stdout:(21/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-03-21T13:17:55.813 INFO:teuthology.orchestra.run.smithi097.stdout:(22/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 182 kB/s | 11 kB 00:00 2024-03-21T13:17:55.838 INFO:teuthology.orchestra.run.smithi097.stdout:(23/36): python3-cherrypy-18.6.1-2.el9.noarch.r 3.5 MB/s | 358 kB 00:00 2024-03-21T13:17:55.855 INFO:teuthology.orchestra.run.smithi097.stdout:(24/36): python3-jaraco-classes-3.2.1-5.el9.noa 300 kB/s | 18 kB 00:00 2024-03-21T13:17:55.872 INFO:teuthology.orchestra.run.smithi097.stdout:(25/36): python3-jaraco-collections-3.0.0-8.el9 394 kB/s | 23 kB 00:00 2024-03-21T13:17:55.897 INFO:teuthology.orchestra.run.smithi097.stdout:(26/36): python3-jaraco-functools-3.5.0-2.el9.n 331 kB/s | 19 kB 00:00 2024-03-21T13:17:55.914 INFO:teuthology.orchestra.run.smithi097.stdout:(27/36): python3-jaraco-text-3.2.0-6.el9.noarch 469 kB/s | 20 kB 00:00 2024-03-21T13:17:55.932 INFO:teuthology.orchestra.run.smithi097.stdout:(28/36): python3-logutils-0.3.5-21.el9.noarch.r 787 kB/s | 46 kB 00:00 2024-03-21T13:17:55.957 INFO:teuthology.orchestra.run.smithi097.stdout:(29/36): python3-more-itertools-8.12.0-2.el9.no 1.3 MB/s | 79 kB 00:00 2024-03-21T13:17:55.973 INFO:teuthology.orchestra.run.smithi097.stdout:(30/36): python3-portend-3.1.0-2.el9.noarch.rpm 394 kB/s | 16 kB 00:00 2024-03-21T13:17:56.007 INFO:teuthology.orchestra.run.smithi097.stdout:(31/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.8 MB/s | 90 kB 00:00 2024-03-21T13:17:56.024 INFO:teuthology.orchestra.run.smithi097.stdout:(32/36): python3-tempora-5.0.0-2.el9.noarch.rpm 711 kB/s | 36 kB 00:00 2024-03-21T13:17:56.050 INFO:teuthology.orchestra.run.smithi097.stdout:(33/36): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-03-21T13:17:56.075 INFO:teuthology.orchestra.run.smithi097.stdout:(34/36): python3-webob-1.8.7-6.el9.noarch.rpm 3.3 MB/s | 230 kB 00:00 2024-03-21T13:17:56.092 INFO:teuthology.orchestra.run.smithi097.stdout:(35/36): python3-zc-lockfile-2.0-10.el9.noarch. 479 kB/s | 20 kB 00:00 2024-03-21T13:17:56.117 INFO:teuthology.orchestra.run.smithi097.stdout:(36/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.5 MB/s | 427 kB 00:00 2024-03-21T13:17:56.119 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-03-21T13:17:56.119 INFO:teuthology.orchestra.run.smithi097.stdout:Total 13 MB/s | 30 MB 00:02 2024-03-21T13:17:56.142 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-03-21T13:17:56.184 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-03-21T13:17:56.184 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-03-21T13:17:56.591 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-03-21T13:17:56.591 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-03-21T13:17:57.906 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-03-21T13:17:57.986 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-03-21T13:17:58.028 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-03-21T13:17:58.055 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-03-21T13:17:58.118 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-03-21T13:17:58.188 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-03-21T13:17:58.220 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-03-21T13:17:58.310 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-03-21T13:17:58.382 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-03-21T13:17:58.393 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2267.g960fbbe4.el9.x86_64 31/34 2024-03-21T13:17:58.393 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /sys 2024-03-21T13:17:58.393 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /proc 2024-03-21T13:17:58.393 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /mnt 2024-03-21T13:17:58.393 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /var/tmp 2024-03-21T13:17:58.394 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /home 2024-03-21T13:17:58.394 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /root 2024-03-21T13:17:58.394 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /tmp 2024-03-21T13:17:58.394 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:17:58.423 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-03-21T13:17:58.494 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-03-21T13:17:58.557 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 11/36 2024-03-21T13:17:58.595 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 12/36 2024-03-21T13:17:58.644 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ply-3.11-14.el9.noarch 13/36 2024-03-21T13:17:58.707 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 14/36 2024-03-21T13:17:58.889 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 15/36 2024-03-21T13:17:58.933 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 16/36 2024-03-21T13:17:58.968 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 17/36 2024-03-21T13:17:58.998 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 18/36 2024-03-21T13:17:59.028 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 19/36 2024-03-21T13:17:59.105 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 20/36 2024-03-21T13:17:59.142 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-03-21T13:17:59.172 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 22/36 2024-03-21T13:17:59.212 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 23/36 2024-03-21T13:17:59.272 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/36 2024-03-21T13:17:59.307 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 25/36 2024-03-21T13:17:59.321 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-radosgw-2:19.0.0-2267.g960fbbe4.el9.x86_64 32/34 2024-03-21T13:17:59.346 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2267.g960fbbe4.el9.x86_64 32/34 2024-03-21T13:17:59.346 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T13:17:59.346 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-21T13:17:59.346 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-21T13:17:59.346 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-21T13:17:59.346 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:17:59.349 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-03-21T13:17:59.397 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 26/36 2024-03-21T13:17:59.416 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-03-21T13:17:59.417 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-03-21T13:17:59.438 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/36 2024-03-21T13:17:59.515 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/36 2024-03-21T13:17:59.655 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 29/36 2024-03-21T13:17:59.781 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-03-21T13:17:59.938 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2267.g960fbbe4.el9. 31/36 2024-03-21T13:17:59.968 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-2:19.0.0-2267.g960fbbe4.el9.x86_64 32/36 2024-03-21T13:17:59.993 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2267.g960fbbe4.el9.x86_64 32/36 2024-03-21T13:17:59.994 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T13:17:59.994 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-21T13:17:59.994 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-21T13:17:59.994 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-21T13:17:59.994 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:18:00.670 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-03-21T13:18:00.670 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-base-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/34 2024-03-21T13:18:00.671 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-common-2:19.0.0-2267.g960fbbe4.el9.x86_64 2/34 2024-03-21T13:18:00.671 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-radosgw-2:19.0.0-2267.g960fbbe4.el9.x86_64 3/34 2024-03-21T13:18:00.671 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-selinux-2:19.0.0-2267.g960fbbe4.el9.x86_64 4/34 2024-03-21T13:18:00.671 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libcephfs2-2:19.0.0-2267.g960fbbe4.el9.x86_64 5/34 2024-03-21T13:18:00.671 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libradosstriper1-2:19.0.0-2267.g960fbbe4.el9.x86_6 6/34 2024-03-21T13:18:00.671 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librgw2-2:19.0.0-2267.g960fbbe4.el9.x86_64 7/34 2024-03-21T13:18:00.671 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2267.g960fbbe4.el9. 8/34 2024-03-21T13:18:00.671 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-ceph-common-2:19.0.0-2267.g960fbbe4.el9.x8 9/34 2024-03-21T13:18:00.671 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cephfs-2:19.0.0-2267.g960fbbe4.el9.x86_64 10/34 2024-03-21T13:18:00.671 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rados-2:19.0.0-2267.g960fbbe4.el9.x86_64 11/34 2024-03-21T13:18:00.671 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rbd-2:19.0.0-2267.g960fbbe4.el9.x86_64 12/34 2024-03-21T13:18:00.671 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rgw-2:19.0.0-2267.g960fbbe4.el9.x86_64 13/34 2024-03-21T13:18:00.672 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-03-21T13:18:00.672 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-03-21T13:18:00.672 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-03-21T13:18:00.672 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-03-21T13:18:00.672 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-03-21T13:18:00.672 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-03-21T13:18:00.672 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 20/34 2024-03-21T13:18:00.672 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 21/34 2024-03-21T13:18:00.672 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 22/34 2024-03-21T13:18:00.672 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-03-21T13:18:00.672 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 24/34 2024-03-21T13:18:00.672 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 25/34 2024-03-21T13:18:00.672 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-03-21T13:18:00.673 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-03-21T13:18:00.673 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 28/34 2024-03-21T13:18:00.673 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-03-21T13:18:00.673 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-03-21T13:18:00.673 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librados2-2:19.0.0-2267.g960fbbe4.el9.x86_64 31/34 2024-03-21T13:18:00.673 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-03-21T13:18:00.673 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librbd1-2:19.0.0-2267.g960fbbe4.el9.x86_64 33/34 2024-03-21T13:18:01.436 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-osd-2:19.0.0-2267.g960fbbe4.el9.x86_64 33/36 2024-03-21T13:18:01.463 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-osd-2:19.0.0-2267.g960fbbe4.el9.x86_64 33/36 2024-03-21T13:18:01.463 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T13:18:01.463 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-21T13:18:01.463 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-21T13:18:01.463 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-21T13:18:01.463 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:18:01.606 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-03-21T13:18:01.606 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:18:01.606 INFO:teuthology.orchestra.run.smithi144.stdout:Upgraded: 2024-03-21T13:18:01.606 INFO:teuthology.orchestra.run.smithi144.stdout: librados2-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:18:01.606 INFO:teuthology.orchestra.run.smithi144.stdout: librbd1-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:18:01.606 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-03-21T13:18:01.606 INFO:teuthology.orchestra.run.smithi144.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-21T13:18:01.606 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-base-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:18:01.607 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-common-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:18:01.607 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-radosgw-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:18:01.607 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-selinux-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:18:01.607 INFO:teuthology.orchestra.run.smithi144.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-21T13:18:01.607 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-21T13:18:01.607 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-21T13:18:01.607 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs2-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:18:01.607 INFO:teuthology.orchestra.run.smithi144.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-21T13:18:01.607 INFO:teuthology.orchestra.run.smithi144.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-21T13:18:01.607 INFO:teuthology.orchestra.run.smithi144.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-21T13:18:01.607 INFO:teuthology.orchestra.run.smithi144.stdout: libradosstriper1-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:18:01.607 INFO:teuthology.orchestra.run.smithi144.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-21T13:18:01.607 INFO:teuthology.orchestra.run.smithi144.stdout: librgw2-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:18:01.607 INFO:teuthology.orchestra.run.smithi144.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-21T13:18:01.607 INFO:teuthology.orchestra.run.smithi144.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-21T13:18:01.607 INFO:teuthology.orchestra.run.smithi144.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-21T13:18:01.608 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-argparse-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:18:01.608 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-common-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:18:01.608 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cephfs-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:18:01.608 INFO:teuthology.orchestra.run.smithi144.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-21T13:18:01.608 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rados-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:18:01.608 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rbd-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:18:01.608 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rgw-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:18:01.608 INFO:teuthology.orchestra.run.smithi144.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-21T13:18:01.608 INFO:teuthology.orchestra.run.smithi144.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-21T13:18:01.608 INFO:teuthology.orchestra.run.smithi144.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-21T13:18:01.608 INFO:teuthology.orchestra.run.smithi144.stdout: re2-1:20211101-3.el9.x86_64 2024-03-21T13:18:01.608 INFO:teuthology.orchestra.run.smithi144.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-21T13:18:01.608 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:18:01.609 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-03-21T13:18:01.904 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-test 2024-03-21T13:18:01.904 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mon-2:19.0.0-2267.g960fbbe4.el9.x86_64 34/36 2024-03-21T13:18:01.926 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mon-2:19.0.0-2267.g960fbbe4.el9.x86_64 34/36 2024-03-21T13:18:01.926 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T13:18:01.927 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-21T13:18:01.927 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-21T13:18:01.927 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-21T13:18:01.927 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:18:02.146 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mds-2:19.0.0-2267.g960fbbe4.el9.x86_64 35/36 2024-03-21T13:18:02.168 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mds-2:19.0.0-2267.g960fbbe4.el9.x86_64 35/36 2024-03-21T13:18:02.169 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T13:18:02.169 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-21T13:18:02.169 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-21T13:18:02.169 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-21T13:18:02.169 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:18:02.235 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-2:19.0.0-2267.g960fbbe4.el9.x86_64 36/36 2024-03-21T13:18:02.469 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:00:34 ago on Thu 21 Mar 2024 01:17:28 PM UTC. 2024-03-21T13:18:02.626 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-03-21T13:18:02.627 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-03-21T13:18:02.627 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repository Size 2024-03-21T13:18:02.627 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-03-21T13:18:02.627 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-03-21T13:18:02.627 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-test x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 48 M 2024-03-21T13:18:02.627 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2024-03-21T13:18:02.628 INFO:teuthology.orchestra.run.smithi144.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-21T13:18:02.628 INFO:teuthology.orchestra.run.smithi144.stdout: libcephsqlite x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 163 k 2024-03-21T13:18:02.628 INFO:teuthology.orchestra.run.smithi144.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-21T13:18:02.628 INFO:teuthology.orchestra.run.smithi144.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-21T13:18:02.628 INFO:teuthology.orchestra.run.smithi144.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-21T13:18:02.628 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:18:02.628 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-03-21T13:18:02.628 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-03-21T13:18:02.628 INFO:teuthology.orchestra.run.smithi144.stdout:Install 6 Packages 2024-03-21T13:18:02.628 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:18:02.629 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 49 M 2024-03-21T13:18:02.629 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 206 M 2024-03-21T13:18:02.629 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-03-21T13:18:02.912 INFO:teuthology.orchestra.run.smithi144.stdout:(1/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.3 MB/s | 217 kB 00:00 2024-03-21T13:18:02.962 INFO:teuthology.orchestra.run.smithi144.stdout:(2/6): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 300 kB 00:00 2024-03-21T13:18:02.995 INFO:teuthology.orchestra.run.smithi144.stdout:(3/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.8 MB/s | 63 kB 00:00 2024-03-21T13:18:03.121 INFO:teuthology.orchestra.run.smithi144.stdout:(4/6): libcephsqlite-19.0.0-2267.g960fbbe4.el9. 445 kB/s | 163 kB 00:00 2024-03-21T13:18:03.305 INFO:teuthology.orchestra.run.smithi144.stdout:(5/6): jq-1.6-16.el9.x86_64.rpm 606 kB/s | 187 kB 00:00 2024-03-21T13:18:04.079 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-2:19.0.0-2267.g960fbbe4.el9.x86_64 36/36 2024-03-21T13:18:04.079 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/36 2024-03-21T13:18:04.079 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mds-2:19.0.0-2267.g960fbbe4.el9.x86_64 2/36 2024-03-21T13:18:04.079 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-2:19.0.0-2267.g960fbbe4.el9.x86_64 3/36 2024-03-21T13:18:04.079 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mon-2:19.0.0-2267.g960fbbe4.el9.x86_64 4/36 2024-03-21T13:18:04.079 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-osd-2:19.0.0-2267.g960fbbe4.el9.x86_64 5/36 2024-03-21T13:18:04.079 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2267.g960fbbe4.el9. 6/36 2024-03-21T13:18:04.079 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lua-5.4.4-4.el9.x86_64 7/36 2024-03-21T13:18:04.079 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 8/36 2024-03-21T13:18:04.079 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 9/36 2024-03-21T13:18:04.080 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 10/36 2024-03-21T13:18:04.080 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 11/36 2024-03-21T13:18:04.080 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 12/36 2024-03-21T13:18:04.080 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 13/36 2024-03-21T13:18:04.080 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ply-3.11-14.el9.noarch 14/36 2024-03-21T13:18:04.080 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 15/36 2024-03-21T13:18:04.080 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 16/36 2024-03-21T13:18:04.080 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 17/36 2024-03-21T13:18:04.080 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-03-21T13:18:04.080 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-03-21T13:18:04.080 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-03-21T13:18:04.080 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-03-21T13:18:04.080 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-03-21T13:18:04.080 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-03-21T13:18:04.080 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-03-21T13:18:04.081 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-03-21T13:18:04.081 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-03-21T13:18:04.081 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-03-21T13:18:04.081 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-03-21T13:18:04.081 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-03-21T13:18:04.081 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-03-21T13:18:04.081 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-03-21T13:18:04.081 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-03-21T13:18:04.081 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-03-21T13:18:04.081 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-03-21T13:18:04.081 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-03-21T13:18:05.035 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-03-21T13:18:05.035 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:18:05.035 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-03-21T13:18:05.035 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:18:05.036 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mds-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:18:05.036 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:18:05.036 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-modules-core-2:19.0.0-2267.g960fbbe4.el9.noarch 2024-03-21T13:18:05.036 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mon-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:18:05.036 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-osd-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:18:05.036 INFO:teuthology.orchestra.run.smithi097.stdout: lua-5.4.4-4.el9.x86_64 2024-03-21T13:18:05.036 INFO:teuthology.orchestra.run.smithi097.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-21T13:18:05.036 INFO:teuthology.orchestra.run.smithi097.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-21T13:18:05.036 INFO:teuthology.orchestra.run.smithi097.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-21T13:18:05.036 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-21T13:18:05.036 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-21T13:18:05.036 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-21T13:18:05.036 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-21T13:18:05.036 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-21T13:18:05.037 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-21T13:18:05.037 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-21T13:18:05.037 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-21T13:18:05.037 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-21T13:18:05.037 INFO:teuthology.orchestra.run.smithi097.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-21T13:18:05.037 INFO:teuthology.orchestra.run.smithi097.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-21T13:18:05.037 INFO:teuthology.orchestra.run.smithi097.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-21T13:18:05.037 INFO:teuthology.orchestra.run.smithi097.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-21T13:18:05.037 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-21T13:18:05.037 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ply-3.11-14.el9.noarch 2024-03-21T13:18:05.037 INFO:teuthology.orchestra.run.smithi097.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-21T13:18:05.037 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-21T13:18:05.037 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-21T13:18:05.037 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-21T13:18:05.038 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-21T13:18:05.038 INFO:teuthology.orchestra.run.smithi097.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-21T13:18:05.038 INFO:teuthology.orchestra.run.smithi097.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-21T13:18:05.038 INFO:teuthology.orchestra.run.smithi097.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-21T13:18:05.038 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-21T13:18:05.038 INFO:teuthology.orchestra.run.smithi097.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-21T13:18:05.038 INFO:teuthology.orchestra.run.smithi097.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-21T13:18:05.038 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:18:05.038 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-21T13:18:05.138 INFO:teuthology.orchestra.run.smithi144.stdout:(6/6): ceph-test-19.0.0-2267.g960fbbe4.el9.x86_ 20 MB/s | 48 MB 00:02 2024-03-21T13:18:05.141 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-03-21T13:18:05.141 INFO:teuthology.orchestra.run.smithi144.stdout:Total 19 MB/s | 49 MB 00:02 2024-03-21T13:18:05.155 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-03-21T13:18:05.176 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-03-21T13:18:05.176 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-03-21T13:18:05.420 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph-base 2024-03-21T13:18:05.554 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-03-21T13:18:05.555 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-03-21T13:18:05.986 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:00:53 ago on Thu 21 Mar 2024 01:17:12 PM UTC. 2024-03-21T13:18:06.082 INFO:teuthology.orchestra.run.smithi097.stdout:Package ceph-base-2:19.0.0-2267.g960fbbe4.el9.x86_64 is already installed. 2024-03-21T13:18:06.133 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-21T13:18:06.134 INFO:teuthology.orchestra.run.smithi097.stdout:Nothing to do. 2024-03-21T13:18:06.134 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-21T13:18:06.147 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-03-21T13:18:06.192 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install cephadm 2024-03-21T13:18:06.213 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2024-03-21T13:18:06.253 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2024-03-21T13:18:06.293 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-21T13:18:06.347 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : jq-1.6-16.el9.x86_64 4/6 2024-03-21T13:18:06.374 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libcephsqlite-2:19.0.0-2267.g960fbbe4.el9.x86_64 5/6 2024-03-21T13:18:06.754 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:00:54 ago on Thu 21 Mar 2024 01:17:12 PM UTC. 2024-03-21T13:18:06.902 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-21T13:18:06.903 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-21T13:18:06.903 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2024-03-21T13:18:06.903 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-21T13:18:06.903 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-03-21T13:18:06.903 INFO:teuthology.orchestra.run.smithi097.stdout: cephadm noarch 2:19.0.0-2267.g960fbbe4.el9 ceph-noarch 750 k 2024-03-21T13:18:06.903 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:18:06.903 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-03-21T13:18:06.903 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-21T13:18:06.903 INFO:teuthology.orchestra.run.smithi097.stdout:Install 1 Package 2024-03-21T13:18:06.903 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:18:06.904 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 750 k 2024-03-21T13:18:06.904 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 757 k 2024-03-21T13:18:06.904 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-03-21T13:18:07.308 INFO:teuthology.orchestra.run.smithi097.stdout:cephadm-19.0.0-2267.g960fbbe4.el9.noarch.rpm 1.8 MB/s | 750 kB 00:00 2024-03-21T13:18:07.309 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-03-21T13:18:07.309 INFO:teuthology.orchestra.run.smithi097.stdout:Total 1.8 MB/s | 750 kB 00:00 2024-03-21T13:18:07.309 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-03-21T13:18:07.314 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-03-21T13:18:07.314 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-03-21T13:18:07.320 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-03-21T13:18:07.320 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-03-21T13:18:07.393 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-03-21T13:18:07.791 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: cephadm-2:19.0.0-2267.g960fbbe4.el9.noarch 1/1 2024-03-21T13:18:07.880 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephadm-2:19.0.0-2267.g960fbbe4.el9.noarch 1/1 2024-03-21T13:18:08.256 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: cephadm-2:19.0.0-2267.g960fbbe4.el9.noarch 1/1 2024-03-21T13:18:08.522 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephadm-2:19.0.0-2267.g960fbbe4.el9.noarch 1/1 2024-03-21T13:18:08.523 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:18:08.523 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-03-21T13:18:08.523 INFO:teuthology.orchestra.run.smithi097.stdout: cephadm-2:19.0.0-2267.g960fbbe4.el9.noarch 2024-03-21T13:18:08.523 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:18:08.523 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-21T13:18:08.744 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph-immutable-object-cache 2024-03-21T13:18:09.315 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:00:57 ago on Thu 21 Mar 2024 01:17:12 PM UTC. 2024-03-21T13:18:09.469 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-21T13:18:09.469 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-21T13:18:09.469 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repo Size 2024-03-21T13:18:09.470 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-21T13:18:09.470 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-03-21T13:18:09.470 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 145 k 2024-03-21T13:18:09.470 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:18:09.470 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-03-21T13:18:09.470 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-21T13:18:09.470 INFO:teuthology.orchestra.run.smithi097.stdout:Install 1 Package 2024-03-21T13:18:09.470 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:18:09.471 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 145 k 2024-03-21T13:18:09.471 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 436 k 2024-03-21T13:18:09.471 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-03-21T13:18:09.707 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-immutable-object-cache-19.0.0-2267.g960fbb 616 kB/s | 145 kB 00:00 2024-03-21T13:18:09.707 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-03-21T13:18:09.707 INFO:teuthology.orchestra.run.smithi097.stdout:Total 612 kB/s | 145 kB 00:00 2024-03-21T13:18:09.707 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-03-21T13:18:09.715 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-03-21T13:18:09.715 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-03-21T13:18:09.768 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-03-21T13:18:09.768 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-03-21T13:18:09.906 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-03-21T13:18:09.966 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2267.g960fbbe4. 1/1 2024-03-21T13:18:09.990 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2267.g960fbbe4. 1/1 2024-03-21T13:18:09.990 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T13:18:09.990 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-21T13:18:09.990 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:18:10.891 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2267.g960fbbe4.el9.x86_64 5/6 2024-03-21T13:18:10.964 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-test-2:19.0.0-2267.g960fbbe4.el9.x86_64 6/6 2024-03-21T13:18:10.986 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2267.g960fbbe4. 1/1 2024-03-21T13:18:10.986 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:18:10.986 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-03-21T13:18:10.987 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-immutable-object-cache-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:18:10.987 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:18:10.987 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-21T13:18:11.212 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph-mgr 2024-03-21T13:18:11.783 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:00:59 ago on Thu 21 Mar 2024 01:17:12 PM UTC. 2024-03-21T13:18:11.880 INFO:teuthology.orchestra.run.smithi097.stdout:Package ceph-mgr-2:19.0.0-2267.g960fbbe4.el9.x86_64 is already installed. 2024-03-21T13:18:11.933 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-21T13:18:11.934 INFO:teuthology.orchestra.run.smithi097.stdout:Nothing to do. 2024-03-21T13:18:11.934 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-21T13:18:12.008 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph-mgr-dashboard 2024-03-21T13:18:12.566 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:01:00 ago on Thu 21 Mar 2024 01:17:12 PM UTC. 2024-03-21T13:18:12.715 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-21T13:18:12.716 INFO:teuthology.orchestra.run.smithi097.stdout:====================================================================================== 2024-03-21T13:18:12.716 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2024-03-21T13:18:12.716 INFO:teuthology.orchestra.run.smithi097.stdout:====================================================================================== 2024-03-21T13:18:12.716 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-03-21T13:18:12.716 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2267.g960fbbe4.el9 ceph-noarch 3.5 M 2024-03-21T13:18:12.716 INFO:teuthology.orchestra.run.smithi097.stdout:Installing dependencies: 2024-03-21T13:18:12.716 INFO:teuthology.orchestra.run.smithi097.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-21T13:18:12.716 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2267.g960fbbe4.el9 ceph-noarch 34 k 2024-03-21T13:18:12.716 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2267.g960fbbe4.el9 ceph-noarch 16 k 2024-03-21T13:18:12.716 INFO:teuthology.orchestra.run.smithi097.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-21T13:18:12.717 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-21T13:18:12.717 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-21T13:18:12.717 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-21T13:18:12.717 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-21T13:18:12.717 INFO:teuthology.orchestra.run.smithi097.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-21T13:18:12.717 INFO:teuthology.orchestra.run.smithi097.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-21T13:18:12.717 INFO:teuthology.orchestra.run.smithi097.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-21T13:18:12.717 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:18:12.717 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-03-21T13:18:12.717 INFO:teuthology.orchestra.run.smithi097.stdout:====================================================================================== 2024-03-21T13:18:12.717 INFO:teuthology.orchestra.run.smithi097.stdout:Install 12 Packages 2024-03-21T13:18:12.717 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:18:12.718 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 8.6 M 2024-03-21T13:18:12.718 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 103 M 2024-03-21T13:18:12.718 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-03-21T13:18:13.103 INFO:teuthology.orchestra.run.smithi097.stdout:(1/12): ceph-grafana-dashboards-19.0.0-2267.g96 180 kB/s | 34 kB 00:00 2024-03-21T13:18:13.128 INFO:teuthology.orchestra.run.smithi097.stdout:(2/12): ceph-prometheus-alerts-19.0.0-2267.g960 77 kB/s | 16 kB 00:00 2024-03-21T13:18:13.254 INFO:teuthology.orchestra.run.smithi097.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.9 MB/s | 238 kB 00:00 2024-03-21T13:18:13.295 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-test-2:19.0.0-2267.g960fbbe4.el9.x86_64 6/6 2024-03-21T13:18:13.295 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-test-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/6 2024-03-21T13:18:13.295 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libcephsqlite-2:19.0.0-2267.g960fbbe4.el9.x86_64 2/6 2024-03-21T13:18:13.296 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-21T13:18:13.296 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 4/6 2024-03-21T13:18:13.296 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/6 2024-03-21T13:18:13.312 INFO:teuthology.orchestra.run.smithi097.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 4.8 MB/s | 1.0 MB 00:00 2024-03-21T13:18:13.382 INFO:teuthology.orchestra.run.smithi097.stdout:(5/12): ceph-mgr-dashboard-19.0.0-2267.g960fbbe 7.5 MB/s | 3.5 MB 00:00 2024-03-21T13:18:13.671 INFO:teuthology.orchestra.run.smithi097.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 67 kB/s | 19 kB 00:00 2024-03-21T13:18:13.696 INFO:teuthology.orchestra.run.smithi097.stdout:(7/12): protobuf-compiler-3.14.0-13.el9.x86_64. 1.9 MB/s | 863 kB 00:00 2024-03-21T13:18:13.728 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : jq-1.6-16.el9.x86_64 6/6 2024-03-21T13:18:13.728 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:18:13.728 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-03-21T13:18:13.728 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-test-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:18:13.728 INFO:teuthology.orchestra.run.smithi144.stdout: jq-1.6-16.el9.x86_64 2024-03-21T13:18:13.728 INFO:teuthology.orchestra.run.smithi144.stdout: libcephsqlite-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:18:13.728 INFO:teuthology.orchestra.run.smithi144.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-21T13:18:13.728 INFO:teuthology.orchestra.run.smithi144.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-21T13:18:13.728 INFO:teuthology.orchestra.run.smithi144.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-21T13:18:13.728 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:18:13.729 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-03-21T13:18:13.976 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph 2024-03-21T13:18:13.996 INFO:teuthology.orchestra.run.smithi097.stdout:(8/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 801 kB/s | 548 kB 00:00 2024-03-21T13:18:14.097 INFO:teuthology.orchestra.run.smithi097.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 307 kB/s | 31 kB 00:00 2024-03-21T13:18:14.155 INFO:teuthology.orchestra.run.smithi097.stdout:(10/12): python3-grpcio-tools-1.46.7-10.el9.x86 315 kB/s | 144 kB 00:00 2024-03-21T13:18:14.206 INFO:teuthology.orchestra.run.smithi097.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 1.7 MB/s | 188 kB 00:00 2024-03-21T13:18:14.322 INFO:teuthology.orchestra.run.smithi097.stdout:(12/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 3.1 MB/s | 2.0 MB 00:00 2024-03-21T13:18:14.325 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-03-21T13:18:14.325 INFO:teuthology.orchestra.run.smithi097.stdout:Total 5.3 MB/s | 8.6 MB 00:01 2024-03-21T13:18:14.343 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-03-21T13:18:14.360 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-03-21T13:18:14.360 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-03-21T13:18:14.535 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-03-21T13:18:14.536 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-03-21T13:18:14.580 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:00:46 ago on Thu 21 Mar 2024 01:17:28 PM UTC. 2024-03-21T13:18:14.753 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-03-21T13:18:14.755 INFO:teuthology.orchestra.run.smithi144.stdout:========================================================================================= 2024-03-21T13:18:14.756 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repository Size 2024-03-21T13:18:14.756 INFO:teuthology.orchestra.run.smithi144.stdout:========================================================================================= 2024-03-21T13:18:14.756 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-03-21T13:18:14.756 INFO:teuthology.orchestra.run.smithi144.stdout: ceph x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 6.6 k 2024-03-21T13:18:14.756 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2024-03-21T13:18:14.756 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mds x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 2.3 M 2024-03-21T13:18:14.756 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 1.5 M 2024-03-21T13:18:14.756 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2267.g960fbbe4.el9 ceph-noarch 248 k 2024-03-21T13:18:14.756 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mon x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 4.7 M 2024-03-21T13:18:14.756 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-osd x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 17 M 2024-03-21T13:18:14.756 INFO:teuthology.orchestra.run.smithi144.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-21T13:18:14.757 INFO:teuthology.orchestra.run.smithi144.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-21T13:18:14.757 INFO:teuthology.orchestra.run.smithi144.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-21T13:18:14.757 INFO:teuthology.orchestra.run.smithi144.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-21T13:18:14.757 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-21T13:18:14.757 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-21T13:18:14.757 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-21T13:18:14.757 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-21T13:18:14.757 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-21T13:18:14.757 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-21T13:18:14.757 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-21T13:18:14.757 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-21T13:18:14.758 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-21T13:18:14.758 INFO:teuthology.orchestra.run.smithi144.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-21T13:18:14.758 INFO:teuthology.orchestra.run.smithi144.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-21T13:18:14.758 INFO:teuthology.orchestra.run.smithi144.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-21T13:18:14.758 INFO:teuthology.orchestra.run.smithi144.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-21T13:18:14.758 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-21T13:18:14.758 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-21T13:18:14.758 INFO:teuthology.orchestra.run.smithi144.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-21T13:18:14.758 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-21T13:18:14.758 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-21T13:18:14.758 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-21T13:18:14.759 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-21T13:18:14.759 INFO:teuthology.orchestra.run.smithi144.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-21T13:18:14.759 INFO:teuthology.orchestra.run.smithi144.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-21T13:18:14.759 INFO:teuthology.orchestra.run.smithi144.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-21T13:18:14.759 INFO:teuthology.orchestra.run.smithi144.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-21T13:18:14.759 INFO:teuthology.orchestra.run.smithi144.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-21T13:18:14.759 INFO:teuthology.orchestra.run.smithi144.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-21T13:18:14.759 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:18:14.759 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-03-21T13:18:14.759 INFO:teuthology.orchestra.run.smithi144.stdout:========================================================================================= 2024-03-21T13:18:14.759 INFO:teuthology.orchestra.run.smithi144.stdout:Install 36 Packages 2024-03-21T13:18:14.760 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:18:14.760 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 30 M 2024-03-21T13:18:14.760 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 107 M 2024-03-21T13:18:14.760 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-03-21T13:18:15.058 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-03-21T13:18:15.139 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-03-21T13:18:15.175 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-03-21T13:18:15.221 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-03-21T13:18:15.252 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-03-21T13:18:15.368 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-03-21T13:18:15.493 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-03-21T13:18:15.549 INFO:teuthology.orchestra.run.smithi144.stdout:(1/36): ceph-19.0.0-2267.g960fbbe4.el9.x86_64.r 28 kB/s | 6.6 kB 00:00 2024-03-21T13:18:15.579 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-03-21T13:18:15.632 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-03-21T13:18:15.661 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-03-21T13:18:15.743 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2267.g960fbbe4.el9 10/12 2024-03-21T13:18:15.757 INFO:teuthology.orchestra.run.smithi144.stdout:(2/36): ceph-mds-19.0.0-2267.g960fbbe4.el9.x86_ 5.2 MB/s | 2.3 MB 00:00 2024-03-21T13:18:15.799 INFO:teuthology.orchestra.run.smithi144.stdout:(3/36): ceph-mgr-19.0.0-2267.g960fbbe4.el9.x86_ 3.1 MB/s | 1.5 MB 00:00 2024-03-21T13:18:15.891 INFO:teuthology.orchestra.run.smithi144.stdout:(4/36): ceph-mgr-modules-core-19.0.0-2267.g960f 2.7 MB/s | 248 kB 00:00 2024-03-21T13:18:16.008 INFO:teuthology.orchestra.run.smithi144.stdout:(5/36): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 187 kB 00:00 2024-03-21T13:18:16.050 INFO:teuthology.orchestra.run.smithi144.stdout:(6/36): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 144 kB 00:00 2024-03-21T13:18:16.075 INFO:teuthology.orchestra.run.smithi144.stdout:(7/36): python3-markupsafe-1.1.1-12.el9.x86_64. 1.2 MB/s | 32 kB 00:00 2024-03-21T13:18:16.109 INFO:teuthology.orchestra.run.smithi144.stdout:(8/36): python3-pytz-2021.1-5.el9.noarch.rpm 1.4 MB/s | 47 kB 00:00 2024-03-21T13:18:16.134 INFO:teuthology.orchestra.run.smithi144.stdout:(9/36): python3-toml-0.10.2-6.el9.noarch.rpm 1.5 MB/s | 39 kB 00:00 2024-03-21T13:18:16.176 INFO:teuthology.orchestra.run.smithi144.stdout:(10/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.6 MB/s | 241 kB 00:00 2024-03-21T13:18:16.243 INFO:teuthology.orchestra.run.smithi144.stdout:(11/36): python3-cryptography-36.0.1-4.el9.x86_ 17 MB/s | 1.1 MB 00:00 2024-03-21T13:18:16.268 INFO:teuthology.orchestra.run.smithi144.stdout:(12/36): python3-ply-3.11-14.el9.noarch.rpm 4.0 MB/s | 103 kB 00:00 2024-03-21T13:18:16.302 INFO:teuthology.orchestra.run.smithi144.stdout:(13/36): python3-pycparser-2.20-6.el9.noarch.rp 3.6 MB/s | 124 kB 00:00 2024-03-21T13:18:16.335 INFO:teuthology.orchestra.run.smithi144.stdout:(14/36): python3-urllib3-1.26.5-5.el9.noarch.rp 5.5 MB/s | 187 kB 00:00 2024-03-21T13:18:16.553 INFO:teuthology.orchestra.run.smithi144.stdout:(15/36): ceph-osd-19.0.0-2267.g960fbbe4.el9.x86 21 MB/s | 17 MB 00:00 2024-03-21T13:18:16.595 INFO:teuthology.orchestra.run.smithi144.stdout:(16/36): python3-requests-2.25.1-8.el9.noarch.r 483 kB/s | 125 kB 00:00 2024-03-21T13:18:16.728 INFO:teuthology.orchestra.run.smithi144.stdout:(17/36): lua-devel-5.4.4-4.el9.x86_64.rpm 127 kB/s | 22 kB 00:00 2024-03-21T13:18:16.801 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2267.g960fbbe4.el 11/12 2024-03-21T13:18:16.835 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2267.g960fbbe4.el9.noa 12/12 2024-03-21T13:18:17.020 INFO:teuthology.orchestra.run.smithi144.stdout:(18/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 149 kB/s | 43 kB 00:00 2024-03-21T13:18:17.062 INFO:teuthology.orchestra.run.smithi144.stdout:(19/36): luarocks-3.9.2-1.el9.noarch.rpm 323 kB/s | 151 kB 00:00 2024-03-21T13:18:17.212 INFO:teuthology.orchestra.run.smithi144.stdout:(20/36): ceph-mon-19.0.0-2267.g960fbbe4.el9.x86 2.8 MB/s | 4.7 MB 00:01 2024-03-21T13:18:17.238 INFO:teuthology.orchestra.run.smithi144.stdout:(21/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 790 kB/s | 172 kB 00:00 2024-03-21T13:18:17.379 INFO:teuthology.orchestra.run.smithi144.stdout:(22/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 64 kB/s | 11 kB 00:00 2024-03-21T13:18:17.404 INFO:teuthology.orchestra.run.smithi144.stdout:(23/36): python3-cherrypy-18.6.1-2.el9.noarch.r 1.0 MB/s | 358 kB 00:00 2024-03-21T13:18:17.422 INFO:teuthology.orchestra.run.smithi144.stdout:(24/36): python3-jaraco-classes-3.2.1-5.el9.noa 96 kB/s | 18 kB 00:00 2024-03-21T13:18:17.464 INFO:teuthology.orchestra.run.smithi144.stdout:(25/36): python3-jaraco-collections-3.0.0-8.el9 275 kB/s | 23 kB 00:00 2024-03-21T13:18:17.514 INFO:teuthology.orchestra.run.smithi144.stdout:(26/36): python3-jaraco-functools-3.5.0-2.el9.n 212 kB/s | 19 kB 00:00 2024-03-21T13:18:17.539 INFO:teuthology.orchestra.run.smithi144.stdout:(27/36): python3-jaraco-text-3.2.0-6.el9.noarch 168 kB/s | 20 kB 00:00 2024-03-21T13:18:17.564 INFO:teuthology.orchestra.run.smithi144.stdout:(28/36): python3-logutils-0.3.5-21.el9.noarch.r 460 kB/s | 46 kB 00:00 2024-03-21T13:18:17.598 INFO:teuthology.orchestra.run.smithi144.stdout:(29/36): python3-more-itertools-8.12.0-2.el9.no 939 kB/s | 79 kB 00:00 2024-03-21T13:18:18.065 INFO:teuthology.orchestra.run.smithi144.stdout:(30/36): python3-pecan-1.4.2-3.el9.noarch.rpm 518 kB/s | 272 kB 00:00 2024-03-21T13:18:18.082 INFO:teuthology.orchestra.run.smithi144.stdout:(31/36): python3-portend-3.1.0-2.el9.noarch.rpm 32 kB/s | 16 kB 00:00 2024-03-21T13:18:18.133 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2267.g960fbbe4.el9.noa 12/12 2024-03-21T13:18:18.133 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2267.g960fbbe4.el 1/12 2024-03-21T13:18:18.133 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2267.g960fbbe4.el9.noa 2/12 2024-03-21T13:18:18.133 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2267.g960fbbe4.el9 3/12 2024-03-21T13:18:18.133 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-03-21T13:18:18.133 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-03-21T13:18:18.134 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-03-21T13:18:18.134 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-03-21T13:18:18.134 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-03-21T13:18:18.134 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-03-21T13:18:18.134 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-03-21T13:18:18.134 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-03-21T13:18:18.157 INFO:teuthology.orchestra.run.smithi144.stdout:(32/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 161 kB/s | 90 kB 00:00 2024-03-21T13:18:18.174 INFO:teuthology.orchestra.run.smithi144.stdout:(33/36): python3-tempora-5.0.0-2.el9.noarch.rpm 328 kB/s | 36 kB 00:00 2024-03-21T13:18:18.358 INFO:teuthology.orchestra.run.smithi144.stdout:(34/36): python3-webob-1.8.7-6.el9.noarch.rpm 834 kB/s | 230 kB 00:00 2024-03-21T13:18:18.475 INFO:teuthology.orchestra.run.smithi144.stdout:(35/36): python3-zc-lockfile-2.0-10.el9.noarch. 66 kB/s | 20 kB 00:00 2024-03-21T13:18:18.545 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-03-21T13:18:18.545 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:18:18.545 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-03-21T13:18:18.545 INFO:teuthology.orchestra.run.smithi097.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-21T13:18:18.545 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-grafana-dashboards-2:19.0.0-2267.g960fbbe4.el9.noarch 2024-03-21T13:18:18.545 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-dashboard-2:19.0.0-2267.g960fbbe4.el9.noarch 2024-03-21T13:18:18.545 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-prometheus-alerts-2:19.0.0-2267.g960fbbe4.el9.noarch 2024-03-21T13:18:18.545 INFO:teuthology.orchestra.run.smithi097.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-21T13:18:18.545 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-21T13:18:18.545 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-21T13:18:18.545 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-21T13:18:18.545 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-21T13:18:18.545 INFO:teuthology.orchestra.run.smithi097.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-21T13:18:18.545 INFO:teuthology.orchestra.run.smithi097.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-21T13:18:18.546 INFO:teuthology.orchestra.run.smithi097.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-21T13:18:18.546 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:18:18.546 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-21T13:18:18.667 INFO:teuthology.orchestra.run.smithi144.stdout:(36/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 839 kB/s | 427 kB 00:00 2024-03-21T13:18:18.671 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-03-21T13:18:18.671 INFO:teuthology.orchestra.run.smithi144.stdout:Total 7.6 MB/s | 30 MB 00:03 2024-03-21T13:18:18.695 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-03-21T13:18:18.738 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-03-21T13:18:18.738 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-03-21T13:18:18.799 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph-mgr-diskprediction-local 2024-03-21T13:18:19.141 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-03-21T13:18:19.142 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-03-21T13:18:19.365 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:01:07 ago on Thu 21 Mar 2024 01:17:12 PM UTC. 2024-03-21T13:18:19.518 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-21T13:18:19.519 INFO:teuthology.orchestra.run.smithi097.stdout:============================================================================================ 2024-03-21T13:18:19.520 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2024-03-21T13:18:19.520 INFO:teuthology.orchestra.run.smithi097.stdout:============================================================================================ 2024-03-21T13:18:19.520 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-03-21T13:18:19.520 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2267.g960fbbe4.el9 ceph-noarch 7.4 M 2024-03-21T13:18:19.520 INFO:teuthology.orchestra.run.smithi097.stdout:Installing dependencies: 2024-03-21T13:18:19.520 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-21T13:18:19.520 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-21T13:18:19.520 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-21T13:18:19.520 INFO:teuthology.orchestra.run.smithi097.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-21T13:18:19.520 INFO:teuthology.orchestra.run.smithi097.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-21T13:18:19.520 INFO:teuthology.orchestra.run.smithi097.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-21T13:18:19.520 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-21T13:18:19.520 INFO:teuthology.orchestra.run.smithi097.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-21T13:18:19.520 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-21T13:18:19.520 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-21T13:18:19.521 INFO:teuthology.orchestra.run.smithi097.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-21T13:18:19.521 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:18:19.521 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-03-21T13:18:19.521 INFO:teuthology.orchestra.run.smithi097.stdout:============================================================================================ 2024-03-21T13:18:19.521 INFO:teuthology.orchestra.run.smithi097.stdout:Install 12 Packages 2024-03-21T13:18:19.521 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:18:19.521 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 37 M 2024-03-21T13:18:19.521 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 209 M 2024-03-21T13:18:19.521 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-03-21T13:18:20.023 INFO:teuthology.orchestra.run.smithi097.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 197 kB/s | 32 kB 00:00 2024-03-21T13:18:20.065 INFO:teuthology.orchestra.run.smithi097.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 420 kB/s | 17 kB 00:00 2024-03-21T13:18:20.107 INFO:teuthology.orchestra.run.smithi097.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 833 kB/s | 34 kB 00:00 2024-03-21T13:18:20.257 INFO:teuthology.orchestra.run.smithi097.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 7.6 MB/s | 3.0 MB 00:00 2024-03-21T13:18:20.472 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-03-21T13:18:20.544 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-03-21T13:18:20.577 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-03-21T13:18:20.605 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-03-21T13:18:20.674 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-03-21T13:18:20.712 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-03-21T13:18:20.744 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-03-21T13:18:20.832 INFO:teuthology.orchestra.run.smithi097.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 6.5 MB/s | 4.7 MB 00:00 2024-03-21T13:18:20.834 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-03-21T13:18:20.899 INFO:teuthology.orchestra.run.smithi097.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.4 MB/s | 4.8 MB 00:00 2024-03-21T13:18:20.914 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-03-21T13:18:20.924 INFO:teuthology.orchestra.run.smithi097.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.3 MB/s | 217 kB 00:00 2024-03-21T13:18:20.964 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-03-21T13:18:21.035 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-03-21T13:18:21.089 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 11/36 2024-03-21T13:18:21.128 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 12/36 2024-03-21T13:18:21.176 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-ply-3.11-14.el9.noarch 13/36 2024-03-21T13:18:21.232 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 14/36 2024-03-21T13:18:21.400 INFO:teuthology.orchestra.run.smithi097.stdout:(8/12): ceph-mgr-diskprediction-local-19.0.0-22 4.8 MB/s | 7.4 MB 00:01 2024-03-21T13:18:21.414 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 15/36 2024-03-21T13:18:21.466 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 16/36 2024-03-21T13:18:21.504 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 17/36 2024-03-21T13:18:21.539 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 18/36 2024-03-21T13:18:21.550 INFO:teuthology.orchestra.run.smithi097.stdout:(9/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 24 MB/s | 16 MB 00:00 2024-03-21T13:18:21.569 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 19/36 2024-03-21T13:18:21.644 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 20/36 2024-03-21T13:18:21.675 INFO:teuthology.orchestra.run.smithi097.stdout:(10/12): python3-devel-3.9.17-2.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-03-21T13:18:21.680 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-03-21T13:18:21.713 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 22/36 2024-03-21T13:18:21.745 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 23/36 2024-03-21T13:18:21.780 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/36 2024-03-21T13:18:21.814 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 25/36 2024-03-21T13:18:21.903 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 26/36 2024-03-21T13:18:21.953 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/36 2024-03-21T13:18:22.026 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/36 2024-03-21T13:18:22.162 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 29/36 2024-03-21T13:18:22.279 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-03-21T13:18:22.367 INFO:teuthology.orchestra.run.smithi097.stdout:(11/12): libquadmath-11.4.1-3.el9.x86_64.rpm 202 kB/s | 195 kB 00:00 2024-03-21T13:18:22.446 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2267.g960fbbe4.el9. 31/36 2024-03-21T13:18:22.476 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-2:19.0.0-2267.g960fbbe4.el9.x86_64 32/36 2024-03-21T13:18:22.500 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2267.g960fbbe4.el9.x86_64 32/36 2024-03-21T13:18:22.500 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T13:18:22.501 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-21T13:18:22.501 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-21T13:18:22.501 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-21T13:18:22.501 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:18:22.934 INFO:teuthology.orchestra.run.smithi097.stdout:(12/12): libgfortran-11.4.1-3.el9.x86_64.rpm 400 kB/s | 803 kB 00:02 2024-03-21T13:18:22.937 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-03-21T13:18:22.938 INFO:teuthology.orchestra.run.smithi097.stdout:Total 11 MB/s | 37 MB 00:03 2024-03-21T13:18:22.982 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-03-21T13:18:23.007 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-03-21T13:18:23.007 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-03-21T13:18:23.362 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-03-21T13:18:23.363 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-03-21T13:18:23.935 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-osd-2:19.0.0-2267.g960fbbe4.el9.x86_64 33/36 2024-03-21T13:18:23.961 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-osd-2:19.0.0-2267.g960fbbe4.el9.x86_64 33/36 2024-03-21T13:18:23.961 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T13:18:23.961 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-21T13:18:23.961 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-21T13:18:23.961 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-21T13:18:23.961 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:18:24.091 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-03-21T13:18:24.165 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-03-21T13:18:24.196 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-03-21T13:18:24.297 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-03-21T13:18:24.342 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-03-21T13:18:24.402 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mon-2:19.0.0-2267.g960fbbe4.el9.x86_64 34/36 2024-03-21T13:18:24.425 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mon-2:19.0.0-2267.g960fbbe4.el9.x86_64 34/36 2024-03-21T13:18:24.425 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T13:18:24.426 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-21T13:18:24.426 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-21T13:18:24.426 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-21T13:18:24.426 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:18:24.522 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-21T13:18:24.549 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-21T13:18:24.626 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-03-21T13:18:24.645 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mds-2:19.0.0-2267.g960fbbe4.el9.x86_64 35/36 2024-03-21T13:18:24.667 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mds-2:19.0.0-2267.g960fbbe4.el9.x86_64 35/36 2024-03-21T13:18:24.667 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T13:18:24.667 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-21T13:18:24.668 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-21T13:18:24.668 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-21T13:18:24.668 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:18:24.734 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-2:19.0.0-2267.g960fbbe4.el9.x86_64 36/36 2024-03-21T13:18:25.299 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-03-21T13:18:25.412 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-03-21T13:18:26.539 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-2:19.0.0-2267.g960fbbe4.el9.x86_64 36/36 2024-03-21T13:18:26.539 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/36 2024-03-21T13:18:26.539 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mds-2:19.0.0-2267.g960fbbe4.el9.x86_64 2/36 2024-03-21T13:18:26.539 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-2:19.0.0-2267.g960fbbe4.el9.x86_64 3/36 2024-03-21T13:18:26.539 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mon-2:19.0.0-2267.g960fbbe4.el9.x86_64 4/36 2024-03-21T13:18:26.539 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-osd-2:19.0.0-2267.g960fbbe4.el9.x86_64 5/36 2024-03-21T13:18:26.539 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2267.g960fbbe4.el9. 6/36 2024-03-21T13:18:26.539 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : lua-5.4.4-4.el9.x86_64 7/36 2024-03-21T13:18:26.539 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 8/36 2024-03-21T13:18:26.540 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 9/36 2024-03-21T13:18:26.540 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 10/36 2024-03-21T13:18:26.540 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 11/36 2024-03-21T13:18:26.540 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 12/36 2024-03-21T13:18:26.540 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 13/36 2024-03-21T13:18:26.540 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-ply-3.11-14.el9.noarch 14/36 2024-03-21T13:18:26.540 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 15/36 2024-03-21T13:18:26.540 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 16/36 2024-03-21T13:18:26.540 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 17/36 2024-03-21T13:18:26.540 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-03-21T13:18:26.540 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-03-21T13:18:26.540 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-03-21T13:18:26.540 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-03-21T13:18:26.540 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-03-21T13:18:26.541 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-03-21T13:18:26.541 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-03-21T13:18:26.541 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-03-21T13:18:26.542 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-03-21T13:18:26.542 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-03-21T13:18:26.542 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-03-21T13:18:26.542 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-03-21T13:18:26.542 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-03-21T13:18:26.542 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-03-21T13:18:26.542 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-03-21T13:18:26.543 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-03-21T13:18:26.543 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-03-21T13:18:26.543 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-03-21T13:18:26.633 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-03-21T13:18:27.331 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-03-21T13:18:27.331 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:18:27.332 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-03-21T13:18:27.332 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:18:27.332 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mds-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:18:27.332 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:18:27.332 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-modules-core-2:19.0.0-2267.g960fbbe4.el9.noarch 2024-03-21T13:18:27.332 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mon-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:18:27.332 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-osd-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:18:27.332 INFO:teuthology.orchestra.run.smithi144.stdout: lua-5.4.4-4.el9.x86_64 2024-03-21T13:18:27.332 INFO:teuthology.orchestra.run.smithi144.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-21T13:18:27.332 INFO:teuthology.orchestra.run.smithi144.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-21T13:18:27.332 INFO:teuthology.orchestra.run.smithi144.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-21T13:18:27.332 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-21T13:18:27.332 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-21T13:18:27.333 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-21T13:18:27.333 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-21T13:18:27.333 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-21T13:18:27.333 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-21T13:18:27.333 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-21T13:18:27.333 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-21T13:18:27.333 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-21T13:18:27.333 INFO:teuthology.orchestra.run.smithi144.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-21T13:18:27.333 INFO:teuthology.orchestra.run.smithi144.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-21T13:18:27.333 INFO:teuthology.orchestra.run.smithi144.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-21T13:18:27.333 INFO:teuthology.orchestra.run.smithi144.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-21T13:18:27.333 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-21T13:18:27.333 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ply-3.11-14.el9.noarch 2024-03-21T13:18:27.333 INFO:teuthology.orchestra.run.smithi144.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-21T13:18:27.333 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-21T13:18:27.334 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-21T13:18:27.334 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-21T13:18:27.334 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-21T13:18:27.334 INFO:teuthology.orchestra.run.smithi144.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-21T13:18:27.334 INFO:teuthology.orchestra.run.smithi144.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-21T13:18:27.334 INFO:teuthology.orchestra.run.smithi144.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-21T13:18:27.334 INFO:teuthology.orchestra.run.smithi144.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-21T13:18:27.334 INFO:teuthology.orchestra.run.smithi144.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-21T13:18:27.334 INFO:teuthology.orchestra.run.smithi144.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-21T13:18:27.334 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:18:27.334 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-03-21T13:18:27.660 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-base 2024-03-21T13:18:27.674 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-03-21T13:18:27.699 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2267.g960fb 12/12 2024-03-21T13:18:28.241 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:00 ago on Thu 21 Mar 2024 01:17:28 PM UTC. 2024-03-21T13:18:28.338 INFO:teuthology.orchestra.run.smithi144.stdout:Package ceph-base-2:19.0.0-2267.g960fbbe4.el9.x86_64 is already installed. 2024-03-21T13:18:28.390 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-03-21T13:18:28.391 INFO:teuthology.orchestra.run.smithi144.stdout:Nothing to do. 2024-03-21T13:18:28.391 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-03-21T13:18:28.459 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install cephadm 2024-03-21T13:18:29.024 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:01 ago on Thu 21 Mar 2024 01:17:28 PM UTC. 2024-03-21T13:18:29.172 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-03-21T13:18:29.173 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-03-21T13:18:29.173 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repository Size 2024-03-21T13:18:29.173 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-03-21T13:18:29.173 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-03-21T13:18:29.173 INFO:teuthology.orchestra.run.smithi144.stdout: cephadm noarch 2:19.0.0-2267.g960fbbe4.el9 ceph-noarch 750 k 2024-03-21T13:18:29.173 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:18:29.173 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-03-21T13:18:29.173 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-03-21T13:18:29.173 INFO:teuthology.orchestra.run.smithi144.stdout:Install 1 Package 2024-03-21T13:18:29.173 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:18:29.174 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 750 k 2024-03-21T13:18:29.174 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 757 k 2024-03-21T13:18:29.174 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-03-21T13:18:29.638 INFO:teuthology.orchestra.run.smithi144.stdout:cephadm-19.0.0-2267.g960fbbe4.el9.noarch.rpm 1.6 MB/s | 750 kB 00:00 2024-03-21T13:18:29.638 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-03-21T13:18:29.639 INFO:teuthology.orchestra.run.smithi144.stdout:Total 1.6 MB/s | 750 kB 00:00 2024-03-21T13:18:29.639 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-03-21T13:18:29.644 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-03-21T13:18:29.644 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-03-21T13:18:29.650 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-03-21T13:18:29.650 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-03-21T13:18:29.725 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-03-21T13:18:30.196 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: cephadm-2:19.0.0-2267.g960fbbe4.el9.noarch 1/1 2024-03-21T13:18:30.285 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : cephadm-2:19.0.0-2267.g960fbbe4.el9.noarch 1/1 2024-03-21T13:18:30.299 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2267.g960fb 12/12 2024-03-21T13:18:30.299 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2267.g960fb 1/12 2024-03-21T13:18:30.299 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-03-21T13:18:30.299 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-03-21T13:18:30.299 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-03-21T13:18:30.299 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-21T13:18:30.299 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-21T13:18:30.299 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-03-21T13:18:30.300 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-03-21T13:18:30.300 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-03-21T13:18:30.300 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-03-21T13:18:30.300 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-03-21T13:18:30.656 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: cephadm-2:19.0.0-2267.g960fbbe4.el9.noarch 1/1 2024-03-21T13:18:30.682 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-03-21T13:18:30.682 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:18:30.683 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-03-21T13:18:30.683 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2267.g960fbbe4.el9.noarch 2024-03-21T13:18:30.683 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-21T13:18:30.683 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-21T13:18:30.683 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-21T13:18:30.683 INFO:teuthology.orchestra.run.smithi097.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-21T13:18:30.683 INFO:teuthology.orchestra.run.smithi097.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-21T13:18:30.683 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-21T13:18:30.683 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-21T13:18:30.683 INFO:teuthology.orchestra.run.smithi097.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-21T13:18:30.683 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-21T13:18:30.683 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-21T13:18:30.683 INFO:teuthology.orchestra.run.smithi097.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-21T13:18:30.684 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:18:30.684 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-21T13:18:30.962 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : cephadm-2:19.0.0-2267.g960fbbe4.el9.noarch 1/1 2024-03-21T13:18:30.962 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:18:30.962 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-03-21T13:18:30.963 INFO:teuthology.orchestra.run.smithi144.stdout: cephadm-2:19.0.0-2267.g960fbbe4.el9.noarch 2024-03-21T13:18:30.963 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:18:30.963 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-03-21T13:18:31.090 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph-mgr-rook 2024-03-21T13:18:31.184 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-immutable-object-cache 2024-03-21T13:18:31.652 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:01:19 ago on Thu 21 Mar 2024 01:17:12 PM UTC. 2024-03-21T13:18:31.764 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:03 ago on Thu 21 Mar 2024 01:17:28 PM UTC. 2024-03-21T13:18:31.801 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-21T13:18:31.802 INFO:teuthology.orchestra.run.smithi097.stdout:======================================================================================== 2024-03-21T13:18:31.803 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2024-03-21T13:18:31.803 INFO:teuthology.orchestra.run.smithi097.stdout:======================================================================================== 2024-03-21T13:18:31.803 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-03-21T13:18:31.803 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-rook noarch 2:19.0.0-2267.g960fbbe4.el9 ceph-noarch 49 k 2024-03-21T13:18:31.803 INFO:teuthology.orchestra.run.smithi097.stdout:Installing dependencies: 2024-03-21T13:18:31.803 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-21T13:18:31.803 INFO:teuthology.orchestra.run.smithi097.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-21T13:18:31.803 INFO:teuthology.orchestra.run.smithi097.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-21T13:18:31.803 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-21T13:18:31.803 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-21T13:18:31.803 INFO:teuthology.orchestra.run.smithi097.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-21T13:18:31.803 INFO:teuthology.orchestra.run.smithi097.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-21T13:18:31.803 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-21T13:18:31.803 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-21T13:18:31.804 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-21T13:18:31.804 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-21T13:18:31.804 INFO:teuthology.orchestra.run.smithi097.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-21T13:18:31.804 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:18:31.804 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-03-21T13:18:31.804 INFO:teuthology.orchestra.run.smithi097.stdout:======================================================================================== 2024-03-21T13:18:31.804 INFO:teuthology.orchestra.run.smithi097.stdout:Install 13 Packages 2024-03-21T13:18:31.804 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:18:31.804 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 2.0 M 2024-03-21T13:18:31.805 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 26 M 2024-03-21T13:18:31.805 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-03-21T13:18:31.920 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-03-21T13:18:31.920 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-03-21T13:18:31.920 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repo Size 2024-03-21T13:18:31.920 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-03-21T13:18:31.920 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-03-21T13:18:31.920 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 145 k 2024-03-21T13:18:31.920 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:18:31.920 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-03-21T13:18:31.920 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-03-21T13:18:31.921 INFO:teuthology.orchestra.run.smithi144.stdout:Install 1 Package 2024-03-21T13:18:31.921 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:18:31.921 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 145 k 2024-03-21T13:18:31.921 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 436 k 2024-03-21T13:18:31.921 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-03-21T13:18:32.153 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-immutable-object-cache-19.0.0-2267.g960fbb 628 kB/s | 145 kB 00:00 2024-03-21T13:18:32.153 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-03-21T13:18:32.154 INFO:teuthology.orchestra.run.smithi144.stdout:Total 625 kB/s | 145 kB 00:00 2024-03-21T13:18:32.154 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-03-21T13:18:32.161 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-03-21T13:18:32.161 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-03-21T13:18:32.213 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-03-21T13:18:32.213 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-03-21T13:18:32.335 INFO:teuthology.orchestra.run.smithi097.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 138 kB/s | 24 kB 00:00 2024-03-21T13:18:32.354 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-03-21T13:18:32.360 INFO:teuthology.orchestra.run.smithi097.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 89 kB/s | 18 kB 00:00 2024-03-21T13:18:32.420 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2267.g960fbbe4. 1/1 2024-03-21T13:18:32.445 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2267.g960fbbe4. 1/1 2024-03-21T13:18:32.445 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T13:18:32.445 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-21T13:18:32.445 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:18:32.527 INFO:teuthology.orchestra.run.smithi097.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 914 kB/s | 174 kB 00:00 2024-03-21T13:18:32.552 INFO:teuthology.orchestra.run.smithi097.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 688 kB/s | 132 kB 00:00 2024-03-21T13:18:32.603 INFO:teuthology.orchestra.run.smithi097.stdout:(5/13): ceph-mgr-rook-19.0.0-2267.g960fbbe4.el9 111 kB/s | 49 kB 00:00 2024-03-21T13:18:32.628 INFO:teuthology.orchestra.run.smithi097.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.0 MB/s | 211 kB 00:00 2024-03-21T13:18:32.653 INFO:teuthology.orchestra.run.smithi097.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 424 kB/s | 43 kB 00:00 2024-03-21T13:18:32.878 INFO:teuthology.orchestra.run.smithi097.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 117 kB/s | 32 kB 00:00 2024-03-21T13:18:32.904 INFO:teuthology.orchestra.run.smithi097.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 60 kB/s | 15 kB 00:00 2024-03-21T13:18:32.987 INFO:teuthology.orchestra.run.smithi097.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 709 kB/s | 59 kB 00:00 2024-03-21T13:18:33.021 INFO:teuthology.orchestra.run.smithi097.stdout:(11/13): python3-google-auth-2.28.2-1.el9.noarc 552 kB/s | 203 kB 00:00 2024-03-21T13:18:33.096 INFO:teuthology.orchestra.run.smithi097.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 827 kB/s | 90 kB 00:00 2024-03-21T13:18:33.396 INFO:teuthology.orchestra.run.smithi097.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 2.0 MB/s | 1.0 MB 00:00 2024-03-21T13:18:33.400 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-03-21T13:18:33.400 INFO:teuthology.orchestra.run.smithi097.stdout:Total 1.3 MB/s | 2.0 MB 00:01 2024-03-21T13:18:33.401 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-03-21T13:18:33.412 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2267.g960fbbe4. 1/1 2024-03-21T13:18:33.412 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:18:33.412 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-03-21T13:18:33.412 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-immutable-object-cache-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:18:33.412 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:18:33.412 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-03-21T13:18:33.413 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-03-21T13:18:33.413 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-03-21T13:18:33.507 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-03-21T13:18:33.508 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-03-21T13:18:33.658 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-mgr 2024-03-21T13:18:34.058 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-03-21T13:18:34.233 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:06 ago on Thu 21 Mar 2024 01:17:28 PM UTC. 2024-03-21T13:18:34.243 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-03-21T13:18:34.306 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-03-21T13:18:34.332 INFO:teuthology.orchestra.run.smithi144.stdout:Package ceph-mgr-2:19.0.0-2267.g960fbbe4.el9.x86_64 is already installed. 2024-03-21T13:18:34.361 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-03-21T13:18:34.385 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-03-21T13:18:34.386 INFO:teuthology.orchestra.run.smithi144.stdout:Nothing to do. 2024-03-21T13:18:34.386 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-03-21T13:18:34.398 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-03-21T13:18:34.440 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-03-21T13:18:34.461 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-mgr-dashboard 2024-03-21T13:18:34.534 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-03-21T13:18:34.632 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 7/13 2024-03-21T13:18:34.684 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-03-21T13:18:35.026 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:07 ago on Thu 21 Mar 2024 01:17:28 PM UTC. 2024-03-21T13:18:35.175 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-03-21T13:18:35.176 INFO:teuthology.orchestra.run.smithi144.stdout:====================================================================================== 2024-03-21T13:18:35.176 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repository Size 2024-03-21T13:18:35.177 INFO:teuthology.orchestra.run.smithi144.stdout:====================================================================================== 2024-03-21T13:18:35.177 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-03-21T13:18:35.177 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2267.g960fbbe4.el9 ceph-noarch 3.5 M 2024-03-21T13:18:35.177 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2024-03-21T13:18:35.177 INFO:teuthology.orchestra.run.smithi144.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-21T13:18:35.177 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2267.g960fbbe4.el9 ceph-noarch 34 k 2024-03-21T13:18:35.177 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2267.g960fbbe4.el9 ceph-noarch 16 k 2024-03-21T13:18:35.177 INFO:teuthology.orchestra.run.smithi144.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-21T13:18:35.177 INFO:teuthology.orchestra.run.smithi144.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-21T13:18:35.177 INFO:teuthology.orchestra.run.smithi144.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-21T13:18:35.177 INFO:teuthology.orchestra.run.smithi144.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-21T13:18:35.177 INFO:teuthology.orchestra.run.smithi144.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-21T13:18:35.178 INFO:teuthology.orchestra.run.smithi144.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-21T13:18:35.178 INFO:teuthology.orchestra.run.smithi144.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-21T13:18:35.178 INFO:teuthology.orchestra.run.smithi144.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-21T13:18:35.178 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:18:35.178 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-03-21T13:18:35.178 INFO:teuthology.orchestra.run.smithi144.stdout:====================================================================================== 2024-03-21T13:18:35.178 INFO:teuthology.orchestra.run.smithi144.stdout:Install 12 Packages 2024-03-21T13:18:35.178 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:18:35.179 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 8.6 M 2024-03-21T13:18:35.179 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 103 M 2024-03-21T13:18:35.179 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-03-21T13:18:35.393 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-03-21T13:18:35.440 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-03-21T13:18:35.473 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-03-21T13:18:35.522 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-03-21T13:18:35.544 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-rook-2:19.0.0-2267.g960fbbe4.el9.noarch 13/13 2024-03-21T13:18:35.574 INFO:teuthology.orchestra.run.smithi144.stdout:(1/12): ceph-prometheus-alerts-19.0.0-2267.g960 83 kB/s | 16 kB 00:00 2024-03-21T13:18:35.674 INFO:teuthology.orchestra.run.smithi144.stdout:(2/12): ceph-grafana-dashboards-19.0.0-2267.g96 114 kB/s | 34 kB 00:00 2024-03-21T13:18:35.733 INFO:teuthology.orchestra.run.smithi144.stdout:(3/12): protobuf-3.14.0-13.el9.x86_64.rpm 6.4 MB/s | 1.0 MB 00:00 2024-03-21T13:18:35.758 INFO:teuthology.orchestra.run.smithi144.stdout:(4/12): python3-protobuf-3.14.0-13.el9.noarch.r 2.8 MB/s | 238 kB 00:00 2024-03-21T13:18:35.875 INFO:teuthology.orchestra.run.smithi144.stdout:(5/12): ceph-mgr-dashboard-19.0.0-2267.g960fbbe 7.1 MB/s | 3.5 MB 00:00 2024-03-21T13:18:36.000 INFO:teuthology.orchestra.run.smithi144.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 155 kB/s | 19 kB 00:00 2024-03-21T13:18:36.025 INFO:teuthology.orchestra.run.smithi144.stdout:(7/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 2.0 MB/s | 548 kB 00:00 2024-03-21T13:18:36.109 INFO:teuthology.orchestra.run.smithi144.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 1.7 MB/s | 144 kB 00:00 2024-03-21T13:18:36.184 INFO:teuthology.orchestra.run.smithi144.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 410 kB/s | 31 kB 00:00 2024-03-21T13:18:36.259 INFO:teuthology.orchestra.run.smithi144.stdout:(10/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 7.9 MB/s | 2.0 MB 00:00 2024-03-21T13:18:36.284 INFO:teuthology.orchestra.run.smithi144.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 1.8 MB/s | 188 kB 00:00 2024-03-21T13:18:36.351 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2267.g960fbbe4.el9.noarch 13/13 2024-03-21T13:18:36.351 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2267.g960fbbe4.el9.noarch 1/13 2024-03-21T13:18:36.351 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-03-21T13:18:36.351 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-03-21T13:18:36.351 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-03-21T13:18:36.351 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-03-21T13:18:36.351 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-03-21T13:18:36.351 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-03-21T13:18:36.351 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-03-21T13:18:36.351 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-03-21T13:18:36.352 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 10/13 2024-03-21T13:18:36.352 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-03-21T13:18:36.352 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-03-21T13:18:36.712 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-03-21T13:18:36.712 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:18:36.712 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-03-21T13:18:36.712 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-rook-2:19.0.0-2267.g960fbbe4.el9.noarch 2024-03-21T13:18:36.712 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-21T13:18:36.712 INFO:teuthology.orchestra.run.smithi097.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-21T13:18:36.713 INFO:teuthology.orchestra.run.smithi097.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-21T13:18:36.713 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-21T13:18:36.713 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-21T13:18:36.713 INFO:teuthology.orchestra.run.smithi097.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-21T13:18:36.713 INFO:teuthology.orchestra.run.smithi097.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-21T13:18:36.713 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-21T13:18:36.713 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-21T13:18:36.713 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-21T13:18:36.713 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-21T13:18:36.713 INFO:teuthology.orchestra.run.smithi097.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-21T13:18:36.713 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:18:36.713 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-21T13:18:36.966 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph-mgr-cephadm 2024-03-21T13:18:37.468 INFO:teuthology.orchestra.run.smithi144.stdout:(12/12): protobuf-compiler-3.14.0-13.el9.x86_64 498 kB/s | 863 kB 00:01 2024-03-21T13:18:37.472 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-03-21T13:18:37.473 INFO:teuthology.orchestra.run.smithi144.stdout:Total 3.7 MB/s | 8.6 MB 00:02 2024-03-21T13:18:37.493 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-03-21T13:18:37.511 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-03-21T13:18:37.512 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-03-21T13:18:37.530 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:01:25 ago on Thu 21 Mar 2024 01:17:12 PM UTC. 2024-03-21T13:18:37.680 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-21T13:18:37.681 INFO:teuthology.orchestra.run.smithi097.stdout:======================================================================================== 2024-03-21T13:18:37.681 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2024-03-21T13:18:37.681 INFO:teuthology.orchestra.run.smithi097.stdout:======================================================================================== 2024-03-21T13:18:37.681 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-03-21T13:18:37.681 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2267.g960fbbe4.el9 ceph-noarch 140 k 2024-03-21T13:18:37.681 INFO:teuthology.orchestra.run.smithi097.stdout:Installing dependencies: 2024-03-21T13:18:37.681 INFO:teuthology.orchestra.run.smithi097.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-21T13:18:37.681 INFO:teuthology.orchestra.run.smithi097.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-21T13:18:37.682 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-21T13:18:37.682 INFO:teuthology.orchestra.run.smithi097.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-21T13:18:37.682 INFO:teuthology.orchestra.run.smithi097.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-21T13:18:37.682 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:18:37.682 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-03-21T13:18:37.682 INFO:teuthology.orchestra.run.smithi097.stdout:======================================================================================== 2024-03-21T13:18:37.682 INFO:teuthology.orchestra.run.smithi097.stdout:Install 6 Packages 2024-03-21T13:18:37.682 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:18:37.683 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 6.8 M 2024-03-21T13:18:37.683 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 33 M 2024-03-21T13:18:37.683 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-03-21T13:18:37.702 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-03-21T13:18:37.703 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-03-21T13:18:37.968 INFO:teuthology.orchestra.run.smithi097.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.6 MB/s | 248 kB 00:00 2024-03-21T13:18:38.143 INFO:teuthology.orchestra.run.smithi097.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 18 MB/s | 5.8 MB 00:00 2024-03-21T13:18:38.234 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-03-21T13:18:38.243 INFO:teuthology.orchestra.run.smithi097.stdout:(3/6): ceph-mgr-cephadm-19.0.0-2267.g960fbbe4.e 331 kB/s | 140 kB 00:00 2024-03-21T13:18:38.310 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-03-21T13:18:38.353 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-03-21T13:18:38.399 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-03-21T13:18:38.431 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-03-21T13:18:38.544 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-03-21T13:18:38.627 INFO:teuthology.orchestra.run.smithi097.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 119 kB/s | 58 kB 00:00 2024-03-21T13:18:38.652 INFO:teuthology.orchestra.run.smithi097.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 802 kB/s | 548 kB 00:00 2024-03-21T13:18:38.673 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-03-21T13:18:38.685 INFO:teuthology.orchestra.run.smithi097.stdout:(6/6): python3-typing-extensions-4.4.0-2.el9.no 122 kB/s | 51 kB 00:00 2024-03-21T13:18:38.689 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-03-21T13:18:38.689 INFO:teuthology.orchestra.run.smithi097.stdout:Total 6.8 MB/s | 6.8 MB 00:01 2024-03-21T13:18:38.703 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-03-21T13:18:38.714 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-03-21T13:18:38.714 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-03-21T13:18:38.776 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-03-21T13:18:38.805 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-03-21T13:18:38.805 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-03-21T13:18:38.819 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-03-21T13:18:38.848 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-03-21T13:18:38.879 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2267.g960fbbe4.el9 10/12 2024-03-21T13:18:39.107 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-03-21T13:18:39.268 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-03-21T13:18:39.312 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-03-21T13:18:39.829 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-03-21T13:18:39.906 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-03-21T13:18:39.926 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2267.g960fbbe4.el 11/12 2024-03-21T13:18:39.963 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2267.g960fbbe4.el9.noa 12/12 2024-03-21T13:18:39.969 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-03-21T13:18:39.999 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2267.g960fbbe4.el9.noarch 6/6 2024-03-21T13:18:40.829 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2267.g960fbbe4.el9.noarch 6/6 2024-03-21T13:18:40.829 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2267.g960fbbe4.el9.noarch 1/6 2024-03-21T13:18:40.829 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-03-21T13:18:40.829 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-03-21T13:18:40.830 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-03-21T13:18:40.830 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-03-21T13:18:41.126 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-03-21T13:18:41.126 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:18:41.126 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-03-21T13:18:41.126 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-cephadm-2:19.0.0-2267.g960fbbe4.el9.noarch 2024-03-21T13:18:41.127 INFO:teuthology.orchestra.run.smithi097.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-21T13:18:41.127 INFO:teuthology.orchestra.run.smithi097.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-21T13:18:41.127 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-21T13:18:41.127 INFO:teuthology.orchestra.run.smithi097.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-21T13:18:41.127 INFO:teuthology.orchestra.run.smithi097.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-21T13:18:41.127 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:18:41.127 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-21T13:18:41.331 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2267.g960fbbe4.el9.noa 12/12 2024-03-21T13:18:41.331 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2267.g960fbbe4.el 1/12 2024-03-21T13:18:41.331 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2267.g960fbbe4.el9.noa 2/12 2024-03-21T13:18:41.331 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2267.g960fbbe4.el9 3/12 2024-03-21T13:18:41.331 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-03-21T13:18:41.331 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-03-21T13:18:41.331 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-03-21T13:18:41.331 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-03-21T13:18:41.331 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-03-21T13:18:41.331 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-03-21T13:18:41.332 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-03-21T13:18:41.332 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-03-21T13:18:41.376 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph-fuse 2024-03-21T13:18:41.669 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-03-21T13:18:41.669 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:18:41.669 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-03-21T13:18:41.669 INFO:teuthology.orchestra.run.smithi144.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-21T13:18:41.669 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-grafana-dashboards-2:19.0.0-2267.g960fbbe4.el9.noarch 2024-03-21T13:18:41.669 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-dashboard-2:19.0.0-2267.g960fbbe4.el9.noarch 2024-03-21T13:18:41.669 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-prometheus-alerts-2:19.0.0-2267.g960fbbe4.el9.noarch 2024-03-21T13:18:41.669 INFO:teuthology.orchestra.run.smithi144.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-21T13:18:41.669 INFO:teuthology.orchestra.run.smithi144.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-21T13:18:41.670 INFO:teuthology.orchestra.run.smithi144.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-21T13:18:41.670 INFO:teuthology.orchestra.run.smithi144.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-21T13:18:41.670 INFO:teuthology.orchestra.run.smithi144.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-21T13:18:41.670 INFO:teuthology.orchestra.run.smithi144.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-21T13:18:41.670 INFO:teuthology.orchestra.run.smithi144.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-21T13:18:41.670 INFO:teuthology.orchestra.run.smithi144.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-21T13:18:41.670 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:18:41.670 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-03-21T13:18:41.935 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:01:29 ago on Thu 21 Mar 2024 01:17:12 PM UTC. 2024-03-21T13:18:41.949 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-mgr-diskprediction-local 2024-03-21T13:18:42.087 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-21T13:18:42.087 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-21T13:18:42.088 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2024-03-21T13:18:42.088 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-21T13:18:42.088 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-03-21T13:18:42.088 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-fuse x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 890 k 2024-03-21T13:18:42.088 INFO:teuthology.orchestra.run.smithi097.stdout:Installing dependencies: 2024-03-21T13:18:42.088 INFO:teuthology.orchestra.run.smithi097.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-21T13:18:42.088 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:18:42.088 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-03-21T13:18:42.088 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-21T13:18:42.088 INFO:teuthology.orchestra.run.smithi097.stdout:Install 2 Packages 2024-03-21T13:18:42.088 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:18:42.089 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 969 k 2024-03-21T13:18:42.089 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 2.8 M 2024-03-21T13:18:42.089 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-03-21T13:18:42.238 INFO:teuthology.orchestra.run.smithi097.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 529 kB/s | 79 kB 00:00 2024-03-21T13:18:42.438 INFO:teuthology.orchestra.run.smithi097.stdout:(2/2): ceph-fuse-19.0.0-2267.g960fbbe4.el9.x86_ 2.5 MB/s | 890 kB 00:00 2024-03-21T13:18:42.439 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-03-21T13:18:42.439 INFO:teuthology.orchestra.run.smithi097.stdout:Total 2.7 MB/s | 969 kB 00:00 2024-03-21T13:18:42.440 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-03-21T13:18:42.460 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-03-21T13:18:42.460 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-03-21T13:18:42.517 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:14 ago on Thu 21 Mar 2024 01:17:28 PM UTC. 2024-03-21T13:18:42.534 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-03-21T13:18:42.535 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-03-21T13:18:42.671 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-03-21T13:18:42.673 INFO:teuthology.orchestra.run.smithi144.stdout:============================================================================================ 2024-03-21T13:18:42.673 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repository Size 2024-03-21T13:18:42.673 INFO:teuthology.orchestra.run.smithi144.stdout:============================================================================================ 2024-03-21T13:18:42.673 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-03-21T13:18:42.673 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2267.g960fbbe4.el9 ceph-noarch 7.4 M 2024-03-21T13:18:42.673 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2024-03-21T13:18:42.673 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-21T13:18:42.673 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-21T13:18:42.673 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-21T13:18:42.673 INFO:teuthology.orchestra.run.smithi144.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-21T13:18:42.673 INFO:teuthology.orchestra.run.smithi144.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-21T13:18:42.673 INFO:teuthology.orchestra.run.smithi144.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-21T13:18:42.674 INFO:teuthology.orchestra.run.smithi144.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-21T13:18:42.674 INFO:teuthology.orchestra.run.smithi144.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-21T13:18:42.674 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-21T13:18:42.674 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-21T13:18:42.674 INFO:teuthology.orchestra.run.smithi144.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-21T13:18:42.674 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:18:42.674 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-03-21T13:18:42.674 INFO:teuthology.orchestra.run.smithi144.stdout:============================================================================================ 2024-03-21T13:18:42.674 INFO:teuthology.orchestra.run.smithi144.stdout:Install 12 Packages 2024-03-21T13:18:42.674 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:18:42.675 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 37 M 2024-03-21T13:18:42.675 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 209 M 2024-03-21T13:18:42.675 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-03-21T13:18:42.712 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-03-21T13:18:42.852 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-03-21T13:18:42.922 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-fuse-2:19.0.0-2267.g960fbbe4.el9.x86_64 2/2 2024-03-21T13:18:43.068 INFO:teuthology.orchestra.run.smithi144.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 223 kB/s | 32 kB 00:00 2024-03-21T13:18:43.118 INFO:teuthology.orchestra.run.smithi144.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 349 kB/s | 17 kB 00:00 2024-03-21T13:18:43.177 INFO:teuthology.orchestra.run.smithi144.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 588 kB/s | 34 kB 00:00 2024-03-21T13:18:43.344 INFO:teuthology.orchestra.run.smithi144.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 7.2 MB/s | 3.0 MB 00:00 2024-03-21T13:18:43.596 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-fuse-2:19.0.0-2267.g960fbbe4.el9.x86_64 2/2 2024-03-21T13:18:43.596 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-fuse-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/2 2024-03-21T13:18:43.915 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-03-21T13:18:43.915 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:18:43.915 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-03-21T13:18:43.915 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-fuse-2:19.0.0-2267.g960fbbe4.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-03-21T13:18:43.915 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:18:43.916 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-21T13:18:44.069 INFO:teuthology.orchestra.run.smithi144.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 5.3 MB/s | 4.7 MB 00:00 2024-03-21T13:18:44.190 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph-volume 2024-03-21T13:18:44.211 INFO:teuthology.orchestra.run.smithi144.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 5.5 MB/s | 4.8 MB 00:00 2024-03-21T13:18:44.236 INFO:teuthology.orchestra.run.smithi144.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 1.3 MB/s | 217 kB 00:00 2024-03-21T13:18:44.678 INFO:teuthology.orchestra.run.smithi144.stdout:(8/12): libgfortran-11.4.1-3.el9.x86_64.rpm 1.8 MB/s | 803 kB 00:00 2024-03-21T13:18:44.766 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:01:32 ago on Thu 21 Mar 2024 01:17:12 PM UTC. 2024-03-21T13:18:44.820 INFO:teuthology.orchestra.run.smithi144.stdout:(9/12): libquadmath-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 195 kB 00:00 2024-03-21T13:18:44.915 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-21T13:18:44.916 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-21T13:18:44.916 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2024-03-21T13:18:44.916 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-21T13:18:44.916 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-03-21T13:18:44.916 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-volume noarch 2:19.0.0-2267.g960fbbe4.el9 ceph-noarch 273 k 2024-03-21T13:18:44.916 INFO:teuthology.orchestra.run.smithi097.stdout:Installing dependencies: 2024-03-21T13:18:44.916 INFO:teuthology.orchestra.run.smithi097.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-21T13:18:44.916 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:18:44.916 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-03-21T13:18:44.916 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-21T13:18:44.917 INFO:teuthology.orchestra.run.smithi097.stdout:Install 2 Packages 2024-03-21T13:18:44.917 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:18:44.917 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 342 k 2024-03-21T13:18:44.917 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 1.5 M 2024-03-21T13:18:44.917 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-03-21T13:18:45.243 INFO:teuthology.orchestra.run.smithi097.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 211 kB/s | 69 kB 00:00 2024-03-21T13:18:45.379 INFO:teuthology.orchestra.run.smithi144.stdout:(10/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 13 MB/s | 16 MB 00:01 2024-03-21T13:18:45.504 INFO:teuthology.orchestra.run.smithi144.stdout:(11/12): python3-devel-3.9.17-2.el9.x86_64.rpm 360 kB/s | 245 kB 00:00 2024-03-21T13:18:45.518 INFO:teuthology.orchestra.run.smithi097.stdout:(2/2): ceph-volume-19.0.0-2267.g960fbbe4.el9.no 455 kB/s | 273 kB 00:00 2024-03-21T13:18:45.519 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-03-21T13:18:45.519 INFO:teuthology.orchestra.run.smithi097.stdout:Total 567 kB/s | 342 kB 00:00 2024-03-21T13:18:45.520 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-03-21T13:18:45.528 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-03-21T13:18:45.528 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-03-21T13:18:45.561 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-03-21T13:18:45.562 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-03-21T13:18:45.706 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-03-21T13:18:45.821 INFO:teuthology.orchestra.run.smithi144.stdout:(12/12): ceph-mgr-diskprediction-local-19.0.0-2 2.6 MB/s | 7.4 MB 00:02 2024-03-21T13:18:45.823 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-03-21T13:18:45.823 INFO:teuthology.orchestra.run.smithi144.stdout:Total 12 MB/s | 37 MB 00:03 2024-03-21T13:18:45.866 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-03-21T13:18:45.891 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-03-21T13:18:45.891 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-03-21T13:18:45.897 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-03-21T13:18:45.937 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-volume-2:19.0.0-2267.g960fbbe4.el9.noarch 2/2 2024-03-21T13:18:45.953 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-volume-2:19.0.0-2267.g960fbbe4.el9.noarch 2/2 2024-03-21T13:18:45.953 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T13:18:45.953 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-21T13:18:45.953 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:18:46.243 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-03-21T13:18:46.243 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-03-21T13:18:46.701 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-volume-2:19.0.0-2267.g960fbbe4.el9.noarch 1/2 2024-03-21T13:18:46.937 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-03-21T13:18:47.002 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-03-21T13:18:47.019 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-03-21T13:18:47.019 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:18:47.020 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-03-21T13:18:47.020 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-volume-2:19.0.0-2267.g960fbbe4.el9.noarch 2024-03-21T13:18:47.020 INFO:teuthology.orchestra.run.smithi097.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-21T13:18:47.020 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:18:47.020 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-21T13:18:47.033 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-03-21T13:18:47.135 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-03-21T13:18:47.170 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-03-21T13:18:47.339 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install librados-devel 2024-03-21T13:18:47.347 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-21T13:18:47.377 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-21T13:18:47.454 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-03-21T13:18:47.921 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:01:35 ago on Thu 21 Mar 2024 01:17:12 PM UTC. 2024-03-21T13:18:48.073 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-21T13:18:48.073 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-21T13:18:48.073 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repo Size 2024-03-21T13:18:48.073 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-21T13:18:48.073 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-03-21T13:18:48.073 INFO:teuthology.orchestra.run.smithi097.stdout: librados-devel x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 127 k 2024-03-21T13:18:48.073 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:18:48.073 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-03-21T13:18:48.074 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-21T13:18:48.074 INFO:teuthology.orchestra.run.smithi097.stdout:Install 1 Package 2024-03-21T13:18:48.074 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:18:48.074 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 127 k 2024-03-21T13:18:48.074 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 456 k 2024-03-21T13:18:48.074 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-03-21T13:18:48.100 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-03-21T13:18:48.214 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-03-21T13:18:48.457 INFO:teuthology.orchestra.run.smithi097.stdout:librados-devel-19.0.0-2267.g960fbbe4.el9.x86_64 331 kB/s | 127 kB 00:00 2024-03-21T13:18:48.458 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-03-21T13:18:48.458 INFO:teuthology.orchestra.run.smithi097.stdout:Total 329 kB/s | 127 kB 00:00 2024-03-21T13:18:48.458 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-03-21T13:18:48.464 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-03-21T13:18:48.464 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-03-21T13:18:48.528 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-03-21T13:18:48.528 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-03-21T13:18:48.747 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-03-21T13:18:48.865 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librados-devel-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T13:18:49.296 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librados-devel-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T13:18:49.444 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-03-21T13:18:49.593 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados-devel-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T13:18:49.593 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:18:49.594 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-03-21T13:18:49.594 INFO:teuthology.orchestra.run.smithi097.stdout: librados-devel-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:18:49.594 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:18:49.594 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-21T13:18:49.826 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install libcephfs2 2024-03-21T13:18:50.403 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:01:38 ago on Thu 21 Mar 2024 01:17:12 PM UTC. 2024-03-21T13:18:50.458 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-03-21T13:18:50.485 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2267.g960fb 12/12 2024-03-21T13:18:50.503 INFO:teuthology.orchestra.run.smithi097.stdout:Package libcephfs2-2:19.0.0-2267.g960fbbe4.el9.x86_64 is already installed. 2024-03-21T13:18:50.556 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-21T13:18:50.558 INFO:teuthology.orchestra.run.smithi097.stdout:Nothing to do. 2024-03-21T13:18:50.558 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-21T13:18:50.615 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install libcephfs-devel 2024-03-21T13:18:51.181 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:01:39 ago on Thu 21 Mar 2024 01:17:12 PM UTC. 2024-03-21T13:18:51.331 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-21T13:18:51.332 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-21T13:18:51.332 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repo Size 2024-03-21T13:18:51.332 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-21T13:18:51.332 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-03-21T13:18:51.332 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs-devel x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 31 k 2024-03-21T13:18:51.332 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:18:51.332 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-03-21T13:18:51.333 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-21T13:18:51.333 INFO:teuthology.orchestra.run.smithi097.stdout:Install 1 Package 2024-03-21T13:18:51.333 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:18:51.333 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 31 k 2024-03-21T13:18:51.333 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 140 k 2024-03-21T13:18:51.333 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-03-21T13:18:51.495 INFO:teuthology.orchestra.run.smithi097.stdout:libcephfs-devel-19.0.0-2267.g960fbbe4.el9.x86_6 193 kB/s | 31 kB 00:00 2024-03-21T13:18:51.495 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-03-21T13:18:51.495 INFO:teuthology.orchestra.run.smithi097.stdout:Total 192 kB/s | 31 kB 00:00 2024-03-21T13:18:51.496 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-03-21T13:18:51.499 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-03-21T13:18:51.499 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-03-21T13:18:51.528 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-03-21T13:18:51.528 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-03-21T13:18:51.616 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-03-21T13:18:51.726 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephfs-devel-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T13:18:52.081 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T13:18:52.359 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephfs-devel-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T13:18:52.359 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:18:52.359 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-03-21T13:18:52.359 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs-devel-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:18:52.359 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:18:52.359 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-21T13:18:52.535 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install librados2 2024-03-21T13:18:53.007 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2267.g960fb 12/12 2024-03-21T13:18:53.007 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2267.g960fb 1/12 2024-03-21T13:18:53.007 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-03-21T13:18:53.007 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-03-21T13:18:53.007 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-03-21T13:18:53.007 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-21T13:18:53.007 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-21T13:18:53.007 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-03-21T13:18:53.007 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-03-21T13:18:53.007 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-03-21T13:18:53.007 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-03-21T13:18:53.007 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-03-21T13:18:53.099 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:01:41 ago on Thu 21 Mar 2024 01:17:12 PM UTC. 2024-03-21T13:18:53.197 INFO:teuthology.orchestra.run.smithi097.stdout:Package librados2-2:19.0.0-2267.g960fbbe4.el9.x86_64 is already installed. 2024-03-21T13:18:53.249 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-21T13:18:53.249 INFO:teuthology.orchestra.run.smithi097.stdout:Nothing to do. 2024-03-21T13:18:53.250 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-21T13:18:53.319 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install librbd1 2024-03-21T13:18:53.496 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-03-21T13:18:53.496 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:18:53.496 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-03-21T13:18:53.496 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2267.g960fbbe4.el9.noarch 2024-03-21T13:18:53.496 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-21T13:18:53.496 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-21T13:18:53.496 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-21T13:18:53.496 INFO:teuthology.orchestra.run.smithi144.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-21T13:18:53.497 INFO:teuthology.orchestra.run.smithi144.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-21T13:18:53.497 INFO:teuthology.orchestra.run.smithi144.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-21T13:18:53.497 INFO:teuthology.orchestra.run.smithi144.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-21T13:18:53.497 INFO:teuthology.orchestra.run.smithi144.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-21T13:18:53.497 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-21T13:18:53.497 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-21T13:18:53.497 INFO:teuthology.orchestra.run.smithi144.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-21T13:18:53.497 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:18:53.497 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-03-21T13:18:53.879 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:01:41 ago on Thu 21 Mar 2024 01:17:12 PM UTC. 2024-03-21T13:18:53.933 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-mgr-rook 2024-03-21T13:18:53.979 INFO:teuthology.orchestra.run.smithi097.stdout:Package librbd1-2:19.0.0-2267.g960fbbe4.el9.x86_64 is already installed. 2024-03-21T13:18:54.030 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-21T13:18:54.031 INFO:teuthology.orchestra.run.smithi097.stdout:Nothing to do. 2024-03-21T13:18:54.031 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-21T13:18:54.104 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install python3-rados 2024-03-21T13:18:54.507 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:26 ago on Thu 21 Mar 2024 01:17:28 PM UTC. 2024-03-21T13:18:54.658 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-03-21T13:18:54.660 INFO:teuthology.orchestra.run.smithi144.stdout:======================================================================================== 2024-03-21T13:18:54.660 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repository Size 2024-03-21T13:18:54.660 INFO:teuthology.orchestra.run.smithi144.stdout:======================================================================================== 2024-03-21T13:18:54.660 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-03-21T13:18:54.660 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-rook noarch 2:19.0.0-2267.g960fbbe4.el9 ceph-noarch 49 k 2024-03-21T13:18:54.660 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2024-03-21T13:18:54.660 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-21T13:18:54.660 INFO:teuthology.orchestra.run.smithi144.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-21T13:18:54.660 INFO:teuthology.orchestra.run.smithi144.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-21T13:18:54.660 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-21T13:18:54.660 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-21T13:18:54.660 INFO:teuthology.orchestra.run.smithi144.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-21T13:18:54.660 INFO:teuthology.orchestra.run.smithi144.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-21T13:18:54.660 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-21T13:18:54.660 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-21T13:18:54.661 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-21T13:18:54.661 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-21T13:18:54.661 INFO:teuthology.orchestra.run.smithi144.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-21T13:18:54.661 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:18:54.661 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-03-21T13:18:54.661 INFO:teuthology.orchestra.run.smithi144.stdout:======================================================================================== 2024-03-21T13:18:54.661 INFO:teuthology.orchestra.run.smithi144.stdout:Install 13 Packages 2024-03-21T13:18:54.661 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:18:54.661 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 2.0 M 2024-03-21T13:18:54.662 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 26 M 2024-03-21T13:18:54.662 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-03-21T13:18:54.668 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:01:42 ago on Thu 21 Mar 2024 01:17:12 PM UTC. 2024-03-21T13:18:54.766 INFO:teuthology.orchestra.run.smithi097.stdout:Package python3-rados-2:19.0.0-2267.g960fbbe4.el9.x86_64 is already installed. 2024-03-21T13:18:54.820 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-21T13:18:54.821 INFO:teuthology.orchestra.run.smithi097.stdout:Nothing to do. 2024-03-21T13:18:54.821 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-21T13:18:54.870 INFO:teuthology.orchestra.run.smithi144.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 156 kB/s | 24 kB 00:00 2024-03-21T13:18:54.887 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install python3-rgw 2024-03-21T13:18:54.888 INFO:teuthology.orchestra.run.smithi144.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 104 kB/s | 18 kB 00:00 2024-03-21T13:18:54.912 INFO:teuthology.orchestra.run.smithi144.stdout:(3/13): ceph-mgr-rook-19.0.0-2267.g960fbbe4.el9 248 kB/s | 49 kB 00:00 2024-03-21T13:18:55.038 INFO:teuthology.orchestra.run.smithi144.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.0 MB/s | 174 kB 00:00 2024-03-21T13:18:55.088 INFO:teuthology.orchestra.run.smithi144.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 658 kB/s | 132 kB 00:00 2024-03-21T13:18:55.113 INFO:teuthology.orchestra.run.smithi144.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.0 MB/s | 211 kB 00:00 2024-03-21T13:18:55.138 INFO:teuthology.orchestra.run.smithi144.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 425 kB/s | 43 kB 00:00 2024-03-21T13:18:55.339 INFO:teuthology.orchestra.run.smithi144.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 75 kB/s | 15 kB 00:00 2024-03-21T13:18:55.364 INFO:teuthology.orchestra.run.smithi144.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 117 kB/s | 32 kB 00:00 2024-03-21T13:18:55.431 INFO:teuthology.orchestra.run.smithi144.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 888 kB/s | 59 kB 00:00 2024-03-21T13:18:55.453 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:01:43 ago on Thu 21 Mar 2024 01:17:12 PM UTC. 2024-03-21T13:18:55.465 INFO:teuthology.orchestra.run.smithi144.stdout:(11/13): python3-google-auth-2.28.2-1.el9.noarc 622 kB/s | 203 kB 00:00 2024-03-21T13:18:55.514 INFO:teuthology.orchestra.run.smithi144.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.0 MB/s | 90 kB 00:00 2024-03-21T13:18:55.552 INFO:teuthology.orchestra.run.smithi097.stdout:Package python3-rgw-2:19.0.0-2267.g960fbbe4.el9.x86_64 is already installed. 2024-03-21T13:18:55.605 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-21T13:18:55.606 INFO:teuthology.orchestra.run.smithi097.stdout:Nothing to do. 2024-03-21T13:18:55.606 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-21T13:18:55.682 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install python3-cephfs 2024-03-21T13:18:55.831 INFO:teuthology.orchestra.run.smithi144.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 2.1 MB/s | 1.0 MB 00:00 2024-03-21T13:18:55.835 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-03-21T13:18:55.836 INFO:teuthology.orchestra.run.smithi144.stdout:Total 1.7 MB/s | 2.0 MB 00:01 2024-03-21T13:18:55.836 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-03-21T13:18:55.849 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-03-21T13:18:55.849 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-03-21T13:18:55.938 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-03-21T13:18:55.940 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-03-21T13:18:56.242 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:01:44 ago on Thu 21 Mar 2024 01:17:12 PM UTC. 2024-03-21T13:18:56.338 INFO:teuthology.orchestra.run.smithi097.stdout:Package python3-cephfs-2:19.0.0-2267.g960fbbe4.el9.x86_64 is already installed. 2024-03-21T13:18:56.390 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-21T13:18:56.391 INFO:teuthology.orchestra.run.smithi097.stdout:Nothing to do. 2024-03-21T13:18:56.392 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-21T13:18:56.447 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-03-21T13:18:56.452 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install python3-rbd 2024-03-21T13:18:56.619 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-03-21T13:18:56.665 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-03-21T13:18:56.720 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-03-21T13:18:56.750 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-03-21T13:18:56.791 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-03-21T13:18:56.884 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-03-21T13:18:57.003 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 7/13 2024-03-21T13:18:57.019 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:01:45 ago on Thu 21 Mar 2024 01:17:12 PM UTC. 2024-03-21T13:18:57.061 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-03-21T13:18:57.116 INFO:teuthology.orchestra.run.smithi097.stdout:Package python3-rbd-2:19.0.0-2267.g960fbbe4.el9.x86_64 is already installed. 2024-03-21T13:18:57.167 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-21T13:18:57.168 INFO:teuthology.orchestra.run.smithi097.stdout:Nothing to do. 2024-03-21T13:18:57.168 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-21T13:18:57.239 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install rbd-fuse 2024-03-21T13:18:57.787 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-03-21T13:18:57.804 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:01:45 ago on Thu 21 Mar 2024 01:17:12 PM UTC. 2024-03-21T13:18:57.833 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-03-21T13:18:57.867 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-03-21T13:18:57.915 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-03-21T13:18:57.946 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-rook-2:19.0.0-2267.g960fbbe4.el9.noarch 13/13 2024-03-21T13:18:57.953 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-21T13:18:57.954 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-21T13:18:57.954 INFO:teuthology.orchestra.run.smithi097.stdout: Package Architecture Version Repository Size 2024-03-21T13:18:57.954 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-21T13:18:57.954 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-03-21T13:18:57.954 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-fuse x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 85 k 2024-03-21T13:18:57.954 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:18:57.954 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-03-21T13:18:57.954 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-21T13:18:57.954 INFO:teuthology.orchestra.run.smithi097.stdout:Install 1 Package 2024-03-21T13:18:57.954 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:18:57.955 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 85 k 2024-03-21T13:18:57.955 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 231 k 2024-03-21T13:18:57.955 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-03-21T13:18:58.298 INFO:teuthology.orchestra.run.smithi097.stdout:rbd-fuse-19.0.0-2267.g960fbbe4.el9.x86_64.rpm 248 kB/s | 85 kB 00:00 2024-03-21T13:18:58.299 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-03-21T13:18:58.299 INFO:teuthology.orchestra.run.smithi097.stdout:Total 247 kB/s | 85 kB 00:00 2024-03-21T13:18:58.299 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-03-21T13:18:58.306 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-03-21T13:18:58.306 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-03-21T13:18:58.362 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-03-21T13:18:58.362 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-03-21T13:18:58.506 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-03-21T13:18:58.622 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-fuse-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T13:18:58.687 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2267.g960fbbe4.el9.noarch 13/13 2024-03-21T13:18:58.687 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2267.g960fbbe4.el9.noarch 1/13 2024-03-21T13:18:58.687 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-03-21T13:18:58.687 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-03-21T13:18:58.687 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-03-21T13:18:58.687 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-03-21T13:18:58.687 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-03-21T13:18:58.688 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-03-21T13:18:58.688 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-03-21T13:18:58.688 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-03-21T13:18:58.688 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 10/13 2024-03-21T13:18:58.688 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-03-21T13:18:58.688 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-03-21T13:18:59.005 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: rbd-fuse-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T13:18:59.041 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-03-21T13:18:59.041 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:18:59.041 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-03-21T13:18:59.041 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-rook-2:19.0.0-2267.g960fbbe4.el9.noarch 2024-03-21T13:18:59.041 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-21T13:18:59.041 INFO:teuthology.orchestra.run.smithi144.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-21T13:18:59.041 INFO:teuthology.orchestra.run.smithi144.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-21T13:18:59.041 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-21T13:18:59.042 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-21T13:18:59.042 INFO:teuthology.orchestra.run.smithi144.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-21T13:18:59.042 INFO:teuthology.orchestra.run.smithi144.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-21T13:18:59.042 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-21T13:18:59.042 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-21T13:18:59.042 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-21T13:18:59.042 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-21T13:18:59.042 INFO:teuthology.orchestra.run.smithi144.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-21T13:18:59.042 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:18:59.042 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-03-21T13:18:59.289 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-mgr-cephadm 2024-03-21T13:18:59.292 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-fuse-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T13:18:59.292 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:18:59.292 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-03-21T13:18:59.292 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-fuse-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:18:59.292 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:18:59.293 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-21T13:18:59.562 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install rbd-mirror 2024-03-21T13:18:59.854 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:31 ago on Thu 21 Mar 2024 01:17:28 PM UTC. 2024-03-21T13:19:00.003 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-03-21T13:19:00.004 INFO:teuthology.orchestra.run.smithi144.stdout:======================================================================================== 2024-03-21T13:19:00.005 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repository Size 2024-03-21T13:19:00.005 INFO:teuthology.orchestra.run.smithi144.stdout:======================================================================================== 2024-03-21T13:19:00.005 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-03-21T13:19:00.005 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2267.g960fbbe4.el9 ceph-noarch 140 k 2024-03-21T13:19:00.005 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2024-03-21T13:19:00.005 INFO:teuthology.orchestra.run.smithi144.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-21T13:19:00.005 INFO:teuthology.orchestra.run.smithi144.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-21T13:19:00.005 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-21T13:19:00.005 INFO:teuthology.orchestra.run.smithi144.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-21T13:19:00.005 INFO:teuthology.orchestra.run.smithi144.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-21T13:19:00.005 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:19:00.005 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-03-21T13:19:00.005 INFO:teuthology.orchestra.run.smithi144.stdout:======================================================================================== 2024-03-21T13:19:00.005 INFO:teuthology.orchestra.run.smithi144.stdout:Install 6 Packages 2024-03-21T13:19:00.006 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:19:00.006 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 6.8 M 2024-03-21T13:19:00.006 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 33 M 2024-03-21T13:19:00.006 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-03-21T13:19:00.152 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:01:48 ago on Thu 21 Mar 2024 01:17:12 PM UTC. 2024-03-21T13:19:00.310 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-21T13:19:00.311 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-21T13:19:00.311 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repo Size 2024-03-21T13:19:00.311 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-21T13:19:00.311 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-03-21T13:19:00.312 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-mirror x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 3.1 M 2024-03-21T13:19:00.312 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:19:00.312 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-03-21T13:19:00.312 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-21T13:19:00.312 INFO:teuthology.orchestra.run.smithi097.stdout:Install 1 Package 2024-03-21T13:19:00.312 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:19:00.312 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 3.1 M 2024-03-21T13:19:00.312 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 13 M 2024-03-21T13:19:00.312 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-03-21T13:19:00.518 INFO:teuthology.orchestra.run.smithi144.stdout:(1/6): python3-babel-2.9.1-2.el9.noarch.rpm 19 MB/s | 5.8 MB 00:00 2024-03-21T13:19:00.618 INFO:teuthology.orchestra.run.smithi144.stdout:(2/6): ceph-mgr-cephadm-19.0.0-2267.g960fbbe4.e 347 kB/s | 140 kB 00:00 2024-03-21T13:19:00.827 INFO:teuthology.orchestra.run.smithi144.stdout:(3/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 405 kB/s | 248 kB 00:00 2024-03-21T13:19:00.927 INFO:teuthology.orchestra.run.smithi144.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 187 kB/s | 58 kB 00:00 2024-03-21T13:19:00.961 INFO:teuthology.orchestra.run.smithi097.stdout:rbd-mirror-19.0.0-2267.g960fbbe4.el9.x86_64.rpm 4.8 MB/s | 3.1 MB 00:00 2024-03-21T13:19:00.962 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-03-21T13:19:00.962 INFO:teuthology.orchestra.run.smithi097.stdout:Total 4.7 MB/s | 3.1 MB 00:00 2024-03-21T13:19:00.962 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-03-21T13:19:00.971 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-03-21T13:19:00.972 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-03-21T13:19:00.977 INFO:teuthology.orchestra.run.smithi144.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.2 MB/s | 548 kB 00:00 2024-03-21T13:19:01.044 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-03-21T13:19:01.044 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-03-21T13:19:01.169 INFO:teuthology.orchestra.run.smithi144.stdout:(6/6): python3-typing-extensions-4.4.0-2.el9.no 149 kB/s | 51 kB 00:00 2024-03-21T13:19:01.172 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-03-21T13:19:01.172 INFO:teuthology.orchestra.run.smithi144.stdout:Total 5.8 MB/s | 6.8 MB 00:01 2024-03-21T13:19:01.185 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-03-21T13:19:01.195 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-03-21T13:19:01.195 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-03-21T13:19:01.284 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-03-21T13:19:01.284 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-03-21T13:19:01.461 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-03-21T13:19:01.529 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-mirror-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T13:19:01.554 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T13:19:01.554 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T13:19:01.554 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-21T13:19:01.554 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-21T13:19:01.554 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-21T13:19:01.554 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:19:01.582 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-03-21T13:19:01.740 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-03-21T13:19:01.787 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-03-21T13:19:02.301 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-03-21T13:19:02.380 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-03-21T13:19:02.444 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-03-21T13:19:02.467 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2267.g960fbbe4.el9.noarch 6/6 2024-03-21T13:19:02.676 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-mirror-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T13:19:02.676 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:19:02.676 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-03-21T13:19:02.676 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-mirror-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:19:02.676 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:19:02.676 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-21T13:19:02.923 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install rbd-nbd 2024-03-21T13:19:03.270 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2267.g960fbbe4.el9.noarch 6/6 2024-03-21T13:19:03.270 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2267.g960fbbe4.el9.noarch 1/6 2024-03-21T13:19:03.270 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-03-21T13:19:03.271 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-03-21T13:19:03.271 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-03-21T13:19:03.271 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-03-21T13:19:03.490 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:01:51 ago on Thu 21 Mar 2024 01:17:12 PM UTC. 2024-03-21T13:19:03.565 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-03-21T13:19:03.565 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:19:03.565 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-03-21T13:19:03.565 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-cephadm-2:19.0.0-2267.g960fbbe4.el9.noarch 2024-03-21T13:19:03.565 INFO:teuthology.orchestra.run.smithi144.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-21T13:19:03.565 INFO:teuthology.orchestra.run.smithi144.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-21T13:19:03.565 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-21T13:19:03.565 INFO:teuthology.orchestra.run.smithi144.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-21T13:19:03.565 INFO:teuthology.orchestra.run.smithi144.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-21T13:19:03.565 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:19:03.566 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-03-21T13:19:03.642 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-21T13:19:03.643 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-21T13:19:03.643 INFO:teuthology.orchestra.run.smithi097.stdout: Package Architecture Version Repository Size 2024-03-21T13:19:03.643 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-21T13:19:03.643 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-03-21T13:19:03.643 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-nbd x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 170 k 2024-03-21T13:19:03.643 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:19:03.643 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-03-21T13:19:03.643 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-21T13:19:03.643 INFO:teuthology.orchestra.run.smithi097.stdout:Install 1 Package 2024-03-21T13:19:03.644 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:19:03.644 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 170 k 2024-03-21T13:19:03.644 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 490 k 2024-03-21T13:19:03.644 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-03-21T13:19:03.834 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-fuse 2024-03-21T13:19:03.999 INFO:teuthology.orchestra.run.smithi097.stdout:rbd-nbd-19.0.0-2267.g960fbbe4.el9.x86_64.rpm 480 kB/s | 170 kB 00:00 2024-03-21T13:19:04.000 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-03-21T13:19:04.000 INFO:teuthology.orchestra.run.smithi097.stdout:Total 478 kB/s | 170 kB 00:00 2024-03-21T13:19:04.000 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-03-21T13:19:04.008 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-03-21T13:19:04.008 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-03-21T13:19:04.063 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-03-21T13:19:04.063 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-03-21T13:19:04.206 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-03-21T13:19:04.315 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-nbd-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T13:19:04.397 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:36 ago on Thu 21 Mar 2024 01:17:28 PM UTC. 2024-03-21T13:19:04.550 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-03-21T13:19:04.551 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-03-21T13:19:04.551 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repository Size 2024-03-21T13:19:04.551 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-03-21T13:19:04.551 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-03-21T13:19:04.551 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-fuse x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 890 k 2024-03-21T13:19:04.552 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2024-03-21T13:19:04.552 INFO:teuthology.orchestra.run.smithi144.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-21T13:19:04.552 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:19:04.552 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-03-21T13:19:04.552 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-03-21T13:19:04.552 INFO:teuthology.orchestra.run.smithi144.stdout:Install 2 Packages 2024-03-21T13:19:04.552 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:19:04.552 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 969 k 2024-03-21T13:19:04.552 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 2.8 M 2024-03-21T13:19:04.552 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-03-21T13:19:04.679 INFO:teuthology.orchestra.run.smithi144.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 620 kB/s | 79 kB 00:00 2024-03-21T13:19:04.761 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: rbd-nbd-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T13:19:05.040 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-nbd-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T13:19:05.040 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:19:05.040 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-03-21T13:19:05.041 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-nbd-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:19:05.041 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:19:05.041 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-21T13:19:05.297 DEBUG:teuthology.parallel:result is None 2024-03-21T13:19:05.371 INFO:teuthology.orchestra.run.smithi144.stdout:(2/2): ceph-fuse-19.0.0-2267.g960fbbe4.el9.x86_ 1.1 MB/s | 890 kB 00:00 2024-03-21T13:19:05.372 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-03-21T13:19:05.373 INFO:teuthology.orchestra.run.smithi144.stdout:Total 1.2 MB/s | 969 kB 00:00 2024-03-21T13:19:05.373 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-03-21T13:19:05.395 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-03-21T13:19:05.395 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-03-21T13:19:05.470 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-03-21T13:19:05.471 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-03-21T13:19:05.650 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-03-21T13:19:05.795 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-03-21T13:19:05.863 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-fuse-2:19.0.0-2267.g960fbbe4.el9.x86_64 2/2 2024-03-21T13:19:06.521 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-fuse-2:19.0.0-2267.g960fbbe4.el9.x86_64 2/2 2024-03-21T13:19:06.522 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-fuse-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/2 2024-03-21T13:19:06.816 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-03-21T13:19:06.817 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:19:06.817 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-03-21T13:19:06.817 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-fuse-2:19.0.0-2267.g960fbbe4.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-03-21T13:19:06.817 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:19:06.817 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-03-21T13:19:07.123 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-volume 2024-03-21T13:19:07.704 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:39 ago on Thu 21 Mar 2024 01:17:28 PM UTC. 2024-03-21T13:19:07.859 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-03-21T13:19:07.859 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-03-21T13:19:07.859 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repository Size 2024-03-21T13:19:07.860 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-03-21T13:19:07.860 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-03-21T13:19:07.860 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-volume noarch 2:19.0.0-2267.g960fbbe4.el9 ceph-noarch 273 k 2024-03-21T13:19:07.860 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2024-03-21T13:19:07.860 INFO:teuthology.orchestra.run.smithi144.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-21T13:19:07.860 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:19:07.860 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-03-21T13:19:07.860 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-03-21T13:19:07.860 INFO:teuthology.orchestra.run.smithi144.stdout:Install 2 Packages 2024-03-21T13:19:07.860 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:19:07.861 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 342 k 2024-03-21T13:19:07.861 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 1.5 M 2024-03-21T13:19:07.861 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-03-21T13:19:07.985 INFO:teuthology.orchestra.run.smithi144.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 556 kB/s | 69 kB 00:00 2024-03-21T13:19:08.135 INFO:teuthology.orchestra.run.smithi144.stdout:(2/2): ceph-volume-19.0.0-2267.g960fbbe4.el9.no 998 kB/s | 273 kB 00:00 2024-03-21T13:19:08.136 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-03-21T13:19:08.136 INFO:teuthology.orchestra.run.smithi144.stdout:Total 1.2 MB/s | 342 kB 00:00 2024-03-21T13:19:08.136 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-03-21T13:19:08.145 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-03-21T13:19:08.145 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-03-21T13:19:08.177 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-03-21T13:19:08.178 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-03-21T13:19:08.335 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-03-21T13:19:08.516 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-03-21T13:19:08.554 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-volume-2:19.0.0-2267.g960fbbe4.el9.noarch 2/2 2024-03-21T13:19:08.570 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-volume-2:19.0.0-2267.g960fbbe4.el9.noarch 2/2 2024-03-21T13:19:08.570 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T13:19:08.570 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-21T13:19:08.570 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:19:09.298 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-volume-2:19.0.0-2267.g960fbbe4.el9.noarch 1/2 2024-03-21T13:19:09.631 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-03-21T13:19:09.631 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:19:09.631 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-03-21T13:19:09.632 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-volume-2:19.0.0-2267.g960fbbe4.el9.noarch 2024-03-21T13:19:09.632 INFO:teuthology.orchestra.run.smithi144.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-21T13:19:09.632 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:19:09.632 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-03-21T13:19:09.997 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install librados-devel 2024-03-21T13:19:10.581 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:42 ago on Thu 21 Mar 2024 01:17:28 PM UTC. 2024-03-21T13:19:10.736 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-03-21T13:19:10.736 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-03-21T13:19:10.736 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repo Size 2024-03-21T13:19:10.736 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-03-21T13:19:10.736 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-03-21T13:19:10.736 INFO:teuthology.orchestra.run.smithi144.stdout: librados-devel x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 127 k 2024-03-21T13:19:10.736 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:19:10.736 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-03-21T13:19:10.737 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-03-21T13:19:10.737 INFO:teuthology.orchestra.run.smithi144.stdout:Install 1 Package 2024-03-21T13:19:10.737 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:19:10.737 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 127 k 2024-03-21T13:19:10.737 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 456 k 2024-03-21T13:19:10.737 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-03-21T13:19:10.941 INFO:teuthology.orchestra.run.smithi144.stdout:librados-devel-19.0.0-2267.g960fbbe4.el9.x86_64 621 kB/s | 127 kB 00:00 2024-03-21T13:19:10.942 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-03-21T13:19:10.942 INFO:teuthology.orchestra.run.smithi144.stdout:Total 617 kB/s | 127 kB 00:00 2024-03-21T13:19:10.942 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-03-21T13:19:10.947 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-03-21T13:19:10.947 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-03-21T13:19:11.008 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-03-21T13:19:11.008 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-03-21T13:19:11.210 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-03-21T13:19:11.307 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librados-devel-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T13:19:11.762 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librados-devel-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T13:19:12.038 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librados-devel-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T13:19:12.039 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:19:12.039 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-03-21T13:19:12.039 INFO:teuthology.orchestra.run.smithi144.stdout: librados-devel-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:19:12.039 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:19:12.039 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-03-21T13:19:12.269 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install libcephfs2 2024-03-21T13:19:12.854 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:44 ago on Thu 21 Mar 2024 01:17:28 PM UTC. 2024-03-21T13:19:12.955 INFO:teuthology.orchestra.run.smithi144.stdout:Package libcephfs2-2:19.0.0-2267.g960fbbe4.el9.x86_64 is already installed. 2024-03-21T13:19:13.006 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-03-21T13:19:13.007 INFO:teuthology.orchestra.run.smithi144.stdout:Nothing to do. 2024-03-21T13:19:13.007 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-03-21T13:19:13.071 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install libcephfs-devel 2024-03-21T13:19:13.641 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:45 ago on Thu 21 Mar 2024 01:17:28 PM UTC. 2024-03-21T13:19:13.790 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-03-21T13:19:13.791 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-03-21T13:19:13.791 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repo Size 2024-03-21T13:19:13.791 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-03-21T13:19:13.792 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-03-21T13:19:13.792 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs-devel x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 31 k 2024-03-21T13:19:13.792 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:19:13.792 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-03-21T13:19:13.792 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-03-21T13:19:13.792 INFO:teuthology.orchestra.run.smithi144.stdout:Install 1 Package 2024-03-21T13:19:13.792 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:19:13.792 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 31 k 2024-03-21T13:19:13.792 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 140 k 2024-03-21T13:19:13.792 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-03-21T13:19:13.953 INFO:teuthology.orchestra.run.smithi144.stdout:libcephfs-devel-19.0.0-2267.g960fbbe4.el9.x86_6 194 kB/s | 31 kB 00:00 2024-03-21T13:19:13.953 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-03-21T13:19:13.953 INFO:teuthology.orchestra.run.smithi144.stdout:Total 193 kB/s | 31 kB 00:00 2024-03-21T13:19:13.954 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-03-21T13:19:13.957 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-03-21T13:19:13.957 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-03-21T13:19:13.980 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-03-21T13:19:13.981 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-03-21T13:19:14.074 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-03-21T13:19:14.201 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libcephfs-devel-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T13:19:14.595 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T13:19:14.846 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libcephfs-devel-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T13:19:14.846 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:19:14.846 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-03-21T13:19:14.846 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs-devel-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:19:14.846 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:19:14.846 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-03-21T13:19:15.017 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install librados2 2024-03-21T13:19:15.582 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:47 ago on Thu 21 Mar 2024 01:17:28 PM UTC. 2024-03-21T13:19:15.680 INFO:teuthology.orchestra.run.smithi144.stdout:Package librados2-2:19.0.0-2267.g960fbbe4.el9.x86_64 is already installed. 2024-03-21T13:19:15.732 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-03-21T13:19:15.733 INFO:teuthology.orchestra.run.smithi144.stdout:Nothing to do. 2024-03-21T13:19:15.733 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-03-21T13:19:15.786 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install librbd1 2024-03-21T13:19:16.345 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:48 ago on Thu 21 Mar 2024 01:17:28 PM UTC. 2024-03-21T13:19:16.443 INFO:teuthology.orchestra.run.smithi144.stdout:Package librbd1-2:19.0.0-2267.g960fbbe4.el9.x86_64 is already installed. 2024-03-21T13:19:16.494 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-03-21T13:19:16.495 INFO:teuthology.orchestra.run.smithi144.stdout:Nothing to do. 2024-03-21T13:19:16.495 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-03-21T13:19:16.554 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install python3-rados 2024-03-21T13:19:17.123 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:49 ago on Thu 21 Mar 2024 01:17:28 PM UTC. 2024-03-21T13:19:17.220 INFO:teuthology.orchestra.run.smithi144.stdout:Package python3-rados-2:19.0.0-2267.g960fbbe4.el9.x86_64 is already installed. 2024-03-21T13:19:17.272 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-03-21T13:19:17.273 INFO:teuthology.orchestra.run.smithi144.stdout:Nothing to do. 2024-03-21T13:19:17.273 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-03-21T13:19:17.327 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install python3-rgw 2024-03-21T13:19:17.896 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:49 ago on Thu 21 Mar 2024 01:17:28 PM UTC. 2024-03-21T13:19:17.992 INFO:teuthology.orchestra.run.smithi144.stdout:Package python3-rgw-2:19.0.0-2267.g960fbbe4.el9.x86_64 is already installed. 2024-03-21T13:19:18.043 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-03-21T13:19:18.044 INFO:teuthology.orchestra.run.smithi144.stdout:Nothing to do. 2024-03-21T13:19:18.044 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-03-21T13:19:18.116 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install python3-cephfs 2024-03-21T13:19:18.676 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:50 ago on Thu 21 Mar 2024 01:17:28 PM UTC. 2024-03-21T13:19:18.773 INFO:teuthology.orchestra.run.smithi144.stdout:Package python3-cephfs-2:19.0.0-2267.g960fbbe4.el9.x86_64 is already installed. 2024-03-21T13:19:18.825 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-03-21T13:19:18.826 INFO:teuthology.orchestra.run.smithi144.stdout:Nothing to do. 2024-03-21T13:19:18.826 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-03-21T13:19:18.887 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install python3-rbd 2024-03-21T13:19:19.454 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:51 ago on Thu 21 Mar 2024 01:17:28 PM UTC. 2024-03-21T13:19:19.551 INFO:teuthology.orchestra.run.smithi144.stdout:Package python3-rbd-2:19.0.0-2267.g960fbbe4.el9.x86_64 is already installed. 2024-03-21T13:19:19.602 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-03-21T13:19:19.604 INFO:teuthology.orchestra.run.smithi144.stdout:Nothing to do. 2024-03-21T13:19:19.604 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-03-21T13:19:19.660 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install rbd-fuse 2024-03-21T13:19:20.229 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:52 ago on Thu 21 Mar 2024 01:17:28 PM UTC. 2024-03-21T13:19:20.388 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-03-21T13:19:20.388 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-03-21T13:19:20.388 INFO:teuthology.orchestra.run.smithi144.stdout: Package Architecture Version Repository Size 2024-03-21T13:19:20.388 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-03-21T13:19:20.388 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-03-21T13:19:20.388 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-fuse x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 85 k 2024-03-21T13:19:20.389 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:19:20.389 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-03-21T13:19:20.389 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-03-21T13:19:20.389 INFO:teuthology.orchestra.run.smithi144.stdout:Install 1 Package 2024-03-21T13:19:20.389 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:19:20.389 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 85 k 2024-03-21T13:19:20.389 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 231 k 2024-03-21T13:19:20.390 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-03-21T13:19:20.598 INFO:teuthology.orchestra.run.smithi144.stdout:rbd-fuse-19.0.0-2267.g960fbbe4.el9.x86_64.rpm 409 kB/s | 85 kB 00:00 2024-03-21T13:19:20.598 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-03-21T13:19:20.599 INFO:teuthology.orchestra.run.smithi144.stdout:Total 406 kB/s | 85 kB 00:00 2024-03-21T13:19:20.599 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-03-21T13:19:20.606 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-03-21T13:19:20.606 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-03-21T13:19:20.664 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-03-21T13:19:20.664 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-03-21T13:19:20.806 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-03-21T13:19:20.912 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : rbd-fuse-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T13:19:21.347 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: rbd-fuse-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T13:19:21.663 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : rbd-fuse-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T13:19:21.663 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:19:21.663 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-03-21T13:19:21.664 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-fuse-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:19:21.664 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:19:21.664 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-03-21T13:19:21.901 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install rbd-mirror 2024-03-21T13:19:22.483 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:54 ago on Thu 21 Mar 2024 01:17:28 PM UTC. 2024-03-21T13:19:22.639 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-03-21T13:19:22.639 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-03-21T13:19:22.639 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repo Size 2024-03-21T13:19:22.639 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-03-21T13:19:22.639 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-03-21T13:19:22.639 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-mirror x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 3.1 M 2024-03-21T13:19:22.639 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:19:22.639 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-03-21T13:19:22.639 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-03-21T13:19:22.639 INFO:teuthology.orchestra.run.smithi144.stdout:Install 1 Package 2024-03-21T13:19:22.639 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:19:22.640 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 3.1 M 2024-03-21T13:19:22.640 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 13 M 2024-03-21T13:19:22.640 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-03-21T13:19:23.984 INFO:teuthology.orchestra.run.smithi144.stdout:rbd-mirror-19.0.0-2267.g960fbbe4.el9.x86_64.rpm 2.3 MB/s | 3.1 MB 00:01 2024-03-21T13:19:23.985 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-03-21T13:19:23.985 INFO:teuthology.orchestra.run.smithi144.stdout:Total 2.3 MB/s | 3.1 MB 00:01 2024-03-21T13:19:23.985 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-03-21T13:19:23.995 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-03-21T13:19:23.995 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-03-21T13:19:24.069 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-03-21T13:19:24.070 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-03-21T13:19:24.493 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-03-21T13:19:24.568 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : rbd-mirror-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T13:19:24.593 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T13:19:24.594 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T13:19:24.594 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-21T13:19:24.594 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-21T13:19:24.594 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-21T13:19:24.594 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:19:25.738 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : rbd-mirror-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T13:19:25.738 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:19:25.738 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-03-21T13:19:25.738 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-mirror-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:19:25.738 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:19:25.738 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-03-21T13:19:26.000 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install rbd-nbd 2024-03-21T13:19:26.578 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:58 ago on Thu 21 Mar 2024 01:17:28 PM UTC. 2024-03-21T13:19:26.729 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-03-21T13:19:26.730 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-03-21T13:19:26.730 INFO:teuthology.orchestra.run.smithi144.stdout: Package Architecture Version Repository Size 2024-03-21T13:19:26.730 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-03-21T13:19:26.730 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-03-21T13:19:26.730 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-nbd x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 170 k 2024-03-21T13:19:26.730 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:19:26.730 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-03-21T13:19:26.730 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-03-21T13:19:26.730 INFO:teuthology.orchestra.run.smithi144.stdout:Install 1 Package 2024-03-21T13:19:26.730 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:19:26.730 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 170 k 2024-03-21T13:19:26.731 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 490 k 2024-03-21T13:19:26.731 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-03-21T13:19:26.954 INFO:teuthology.orchestra.run.smithi144.stdout:rbd-nbd-19.0.0-2267.g960fbbe4.el9.x86_64.rpm 762 kB/s | 170 kB 00:00 2024-03-21T13:19:26.955 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-03-21T13:19:26.955 INFO:teuthology.orchestra.run.smithi144.stdout:Total 757 kB/s | 170 kB 00:00 2024-03-21T13:19:26.955 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-03-21T13:19:26.963 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-03-21T13:19:26.963 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-03-21T13:19:27.018 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-03-21T13:19:27.018 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-03-21T13:19:27.167 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-03-21T13:19:27.261 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : rbd-nbd-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T13:19:27.761 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: rbd-nbd-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T13:19:28.052 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : rbd-nbd-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T13:19:28.052 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:19:28.052 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-03-21T13:19:28.052 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-nbd-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T13:19:28.052 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:19:28.052 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-03-21T13:19:28.285 DEBUG:teuthology.parallel:result is None 2024-03-21T13:19:28.285 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=960fbbe4645c2becc8084669be194c95183c7753 2024-03-21T13:19:28.506 DEBUG:teuthology.orchestra.run.smithi097:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-21T13:19:28.530 INFO:teuthology.orchestra.run.smithi097.stdout:19.0.0-2267.g960fbbe4.el9 2024-03-21T13:19:28.531 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2267.g960fbbe4.el9 2024-03-21T13:19:28.531 INFO:teuthology.task.install:The correct ceph version 19.0.0-2267.g960fbbe4 is installed. 2024-03-21T13:19:28.532 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=960fbbe4645c2becc8084669be194c95183c7753 2024-03-21T13:19:28.677 DEBUG:teuthology.orchestra.run.smithi144:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-21T13:19:28.698 INFO:teuthology.orchestra.run.smithi144.stdout:19.0.0-2267.g960fbbe4.el9 2024-03-21T13:19:28.698 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2267.g960fbbe4.el9 2024-03-21T13:19:28.698 INFO:teuthology.task.install:The correct ceph version 19.0.0-2267.g960fbbe4 is installed. 2024-03-21T13:19:28.700 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-21T13:19:28.700 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-21T13:19:28.700 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-21T13:19:28.732 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-21T13:19:28.732 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-21T13:19:28.765 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-21T13:19:28.765 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-21T13:19:28.765 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/daemon-helper 2024-03-21T13:19:28.803 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-21T13:19:28.875 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-21T13:19:28.875 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/daemon-helper 2024-03-21T13:19:28.909 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-21T13:19:28.976 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-21T13:19:28.977 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-21T13:19:28.977 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-21T13:19:29.006 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-21T13:19:29.075 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-21T13:19:29.075 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-21T13:19:29.102 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-21T13:19:29.168 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-21T13:19:29.169 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-21T13:19:29.169 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/stdin-killer 2024-03-21T13:19:29.196 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-21T13:19:29.263 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-21T13:19:29.263 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/stdin-killer 2024-03-21T13:19:29.290 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-21T13:19:29.356 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-21T13:19:29.445 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'client': {'debug ms': 1}, 'global': {'mon election default strategy': 1, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms type': 'async'}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'sha1': '960fbbe4645c2becc8084669be194c95183c7753', 'cephadm_mode': 'cephadm-package'} 2024-03-21T13:19:29.445 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 2024-03-21T13:19:29.446 INFO:tasks.cephadm:Cluster fsid is a5dd1f32-e785-11ee-95cd-87774f69a715 2024-03-21T13:19:29.446 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-21T13:19:29.446 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.97', 'mon.c': '[v2:172.21.15.97:3301,v1:172.21.15.97:6790]', 'mon.b': '172.21.15.144'} 2024-03-21T13:19:29.446 INFO:tasks.cephadm:First mon is mon.a on smithi097 2024-03-21T13:19:29.446 INFO:tasks.cephadm:First mgr is y 2024-03-21T13:19:29.446 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-21T13:19:29.447 DEBUG:teuthology.orchestra.run.smithi097:> sudo hostname $(hostname -s) 2024-03-21T13:19:29.474 DEBUG:teuthology.orchestra.run.smithi144:> sudo hostname $(hostname -s) 2024-03-21T13:19:29.510 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 on all hosts... 2024-03-21T13:19:29.510 DEBUG:teuthology.orchestra.run.smithi097:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 pull 2024-03-21T13:19:29.516 DEBUG:teuthology.orchestra.run.smithi144:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 pull 2024-03-21T13:19:30.316 INFO:teuthology.orchestra.run.smithi097.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753... 2024-03-21T13:19:30.432 INFO:teuthology.orchestra.run.smithi144.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753... 2024-03-21T13:20:13.792 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-21T13:20:13.792 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 19.0.0-2267-g960fbbe4 (960fbbe4645c2becc8084669be194c95183c7753) squid (dev)", 2024-03-21T13:20:13.793 INFO:teuthology.orchestra.run.smithi097.stdout: "image_id": "19e12293512ff7978c8adb5f72a829060dd47f07bfd6903b6b3178d2ebc29e4e", 2024-03-21T13:20:13.793 INFO:teuthology.orchestra.run.smithi097.stdout: "repo_digests": [ 2024-03-21T13:20:13.793 INFO:teuthology.orchestra.run.smithi097.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93" 2024-03-21T13:20:13.793 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-03-21T13:20:13.793 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-21T13:20:14.987 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-03-21T13:20:14.987 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_version": "ceph version 19.0.0-2267-g960fbbe4 (960fbbe4645c2becc8084669be194c95183c7753) squid (dev)", 2024-03-21T13:20:14.987 INFO:teuthology.orchestra.run.smithi144.stdout: "image_id": "19e12293512ff7978c8adb5f72a829060dd47f07bfd6903b6b3178d2ebc29e4e", 2024-03-21T13:20:14.988 INFO:teuthology.orchestra.run.smithi144.stdout: "repo_digests": [ 2024-03-21T13:20:14.988 INFO:teuthology.orchestra.run.smithi144.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93" 2024-03-21T13:20:14.988 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-03-21T13:20:14.988 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-03-21T13:20:15.011 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /etc/ceph 2024-03-21T13:20:15.046 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /etc/ceph 2024-03-21T13:20:15.075 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 777 /etc/ceph 2024-03-21T13:20:15.115 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 777 /etc/ceph 2024-03-21T13:20:15.142 INFO:tasks.cephadm:Writing seed config... 2024-03-21T13:20:15.143 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-21T13:20:15.143 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-21T13:20:15.143 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-21T13:20:15.143 INFO:tasks.cephadm: override: [global] mon election default strategy = 1 2024-03-21T13:20:15.143 INFO:tasks.cephadm: override: [global] ms bind msgr1 = False 2024-03-21T13:20:15.143 INFO:tasks.cephadm: override: [global] ms bind msgr2 = True 2024-03-21T13:20:15.143 INFO:tasks.cephadm: override: [global] ms type = async 2024-03-21T13:20:15.143 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-21T13:20:15.144 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-21T13:20:15.144 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-21T13:20:15.144 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-03-21T13:20:15.144 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-21T13:20:15.144 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-21T13:20:15.144 INFO:tasks.cephadm: override: [osd] osd class default list = * 2024-03-21T13:20:15.144 INFO:tasks.cephadm: override: [osd] osd class load list = * 2024-03-21T13:20:15.144 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-03-21T13:20:15.144 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-21T13:20:15.144 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-21T13:20:15.171 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = a5dd1f32-e785-11ee-95cd-87774f69a715 mon election default strategy = 1 ms bind msgr1 = False ms bind msgr2 = True ms type = async [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 osd class default list = * osd class load list = * [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] debug ms = 1 2024-03-21T13:20:15.171 DEBUG:teuthology.orchestra.run.smithi097:mon.a> sudo journalctl -f -n 0 -u ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@mon.a.service 2024-03-21T13:20:15.213 DEBUG:teuthology.orchestra.run.smithi097:mgr.y> sudo journalctl -f -n 0 -u ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@mgr.y.service 2024-03-21T13:20:15.258 INFO:tasks.cephadm:Bootstrapping... 2024-03-21T13:20:15.258 DEBUG:teuthology.orchestra.run.smithi097:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 -v bootstrap --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.97 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-21T13:20:15.450 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-03-21T13:20:15.450 INFO:teuthology.orchestra.run.smithi097.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753', '-v', 'bootstrap', '--fsid', 'a5dd1f32-e785-11ee-95cd-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.97', '--skip-admin-label'] 2024-03-21T13:20:15.451 INFO:teuthology.orchestra.run.smithi097.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-21T13:20:15.451 INFO:teuthology.orchestra.run.smithi097.stdout:Verifying podman|docker is present... 2024-03-21T13:20:15.478 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-21T13:20:15.478 INFO:teuthology.orchestra.run.smithi097.stdout:Verifying lvm2 is present... 2024-03-21T13:20:15.478 INFO:teuthology.orchestra.run.smithi097.stdout:Verifying time synchronization is in place... 2024-03-21T13:20:15.485 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-21T13:20:15.486 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-21T13:20:15.491 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-21T13:20:15.491 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout inactive 2024-03-21T13:20:15.498 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout enabled 2024-03-21T13:20:15.504 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout active 2024-03-21T13:20:15.504 INFO:teuthology.orchestra.run.smithi097.stdout:Unit chronyd.service is enabled and running 2024-03-21T13:20:15.504 INFO:teuthology.orchestra.run.smithi097.stdout:Repeating the final host check... 2024-03-21T13:20:15.527 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-21T13:20:15.527 INFO:teuthology.orchestra.run.smithi097.stdout:podman (/bin/podman) version 4.9.3 is present 2024-03-21T13:20:15.527 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl is present 2024-03-21T13:20:15.527 INFO:teuthology.orchestra.run.smithi097.stdout:lvcreate is present 2024-03-21T13:20:15.533 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-21T13:20:15.533 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-21T13:20:15.538 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-21T13:20:15.538 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout inactive 2024-03-21T13:20:15.544 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout enabled 2024-03-21T13:20:15.551 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout active 2024-03-21T13:20:15.551 INFO:teuthology.orchestra.run.smithi097.stdout:Unit chronyd.service is enabled and running 2024-03-21T13:20:15.551 INFO:teuthology.orchestra.run.smithi097.stdout:Host looks OK 2024-03-21T13:20:15.551 INFO:teuthology.orchestra.run.smithi097.stdout:Cluster fsid: a5dd1f32-e785-11ee-95cd-87774f69a715 2024-03-21T13:20:15.551 INFO:teuthology.orchestra.run.smithi097.stdout:Acquiring lock 140226123781072 on /run/cephadm/a5dd1f32-e785-11ee-95cd-87774f69a715.lock 2024-03-21T13:20:15.551 INFO:teuthology.orchestra.run.smithi097.stdout:Lock 140226123781072 acquired on /run/cephadm/a5dd1f32-e785-11ee-95cd-87774f69a715.lock 2024-03-21T13:20:15.551 INFO:teuthology.orchestra.run.smithi097.stdout:Verifying IP 172.21.15.97 port 3300 ... 2024-03-21T13:20:15.552 INFO:teuthology.orchestra.run.smithi097.stdout:Verifying IP 172.21.15.97 port 6789 ... 2024-03-21T13:20:15.552 INFO:teuthology.orchestra.run.smithi097.stdout:Base mon IP(s) is [172.21.15.97:3300, 172.21.15.97:6789], mon addrv is [v2:172.21.15.97:3300,v1:172.21.15.97:6789] 2024-03-21T13:20:15.554 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.97 metric 100 2024-03-21T13:20:15.554 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.97 metric 100 2024-03-21T13:20:15.557 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-21T13:20:15.557 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-03-21T13:20:15.557 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1790sec hoplimit 64 pref medium 2024-03-21T13:20:15.559 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-21T13:20:15.559 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-21T13:20:15.560 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-21T13:20:15.560 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-03-21T13:20:15.560 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-03-21T13:20:15.560 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-21T13:20:15.560 INFO:teuthology.orchestra.run.smithi097.stdout:Mon IP `172.21.15.97` is in CIDR network `172.21.0.0/20` 2024-03-21T13:20:15.560 INFO:teuthology.orchestra.run.smithi097.stdout:Mon IP `172.21.15.97` is in CIDR network `172.21.0.0/20` 2024-03-21T13:20:15.560 INFO:teuthology.orchestra.run.smithi097.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-21T13:20:15.561 INFO:teuthology.orchestra.run.smithi097.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-21T13:20:15.561 INFO:teuthology.orchestra.run.smithi097.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753... 2024-03-21T13:20:16.028 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stdout 19e12293512ff7978c8adb5f72a829060dd47f07bfd6903b6b3178d2ebc29e4e 2024-03-21T13:20:16.028 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753... 2024-03-21T13:20:16.028 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stderr Getting image source signatures 2024-03-21T13:20:16.028 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stderr Copying blob sha256:9516588f908cdb14b5e4062a6063db2d9291b355c3f32e2a99864608a4a7b649 2024-03-21T13:20:16.028 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stderr Copying blob sha256:42ed92a376f0fc24f900533b2c051b7c28b467ca85a1aa456c3a11e39d805f31 2024-03-21T13:20:16.028 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stderr Copying config sha256:19e12293512ff7978c8adb5f72a829060dd47f07bfd6903b6b3178d2ebc29e4e 2024-03-21T13:20:16.028 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-21T13:20:16.993 INFO:teuthology.orchestra.run.smithi097.stdout:ceph: stdout ceph version 19.0.0-2267-g960fbbe4 (960fbbe4645c2becc8084669be194c95183c7753) squid (dev) 2024-03-21T13:20:16.994 INFO:teuthology.orchestra.run.smithi097.stdout:Ceph version: ceph version 19.0.0-2267-g960fbbe4 (960fbbe4645c2becc8084669be194c95183c7753) squid (dev) 2024-03-21T13:20:16.994 INFO:teuthology.orchestra.run.smithi097.stdout:Extracting ceph user uid/gid from container image... 2024-03-21T13:20:18.032 INFO:teuthology.orchestra.run.smithi097.stdout:stat: stdout 167 167 2024-03-21T13:20:18.032 INFO:teuthology.orchestra.run.smithi097.stdout:Creating initial keys... 2024-03-21T13:20:19.113 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph-authtool: stdout AQASNPxlKWJtGBAAO5tlcM5PETUwb8re1wYAlA== 2024-03-21T13:20:25.152 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph-authtool: stdout AQATNPxlBJtfHRAA96my/gSq9NzHjdaKHfSWTA== 2024-03-21T13:20:33.320 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph-authtool: stdout AQAbNPxlQ8c7KhAAn5dN7gITVVBJ8IPbsWYS8A== 2024-03-21T13:20:33.321 INFO:teuthology.orchestra.run.smithi097.stdout:Creating initial monmap... 2024-03-21T13:20:34.293 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-21T13:20:34.293 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-03-21T13:20:34.293 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to a5dd1f32-e785-11ee-95cd-87774f69a715 2024-03-21T13:20:34.293 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-21T13:20:34.293 INFO:teuthology.orchestra.run.smithi097.stdout:monmaptool for a [v2:172.21.15.97:3300,v1:172.21.15.97:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-21T13:20:34.293 INFO:teuthology.orchestra.run.smithi097.stdout:setting min_mon_release = quincy 2024-03-21T13:20:34.293 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/monmaptool: set fsid to a5dd1f32-e785-11ee-95cd-87774f69a715 2024-03-21T13:20:34.293 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-21T13:20:34.293 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:20:34.293 INFO:teuthology.orchestra.run.smithi097.stdout:Creating mon... 2024-03-21T13:20:35.407 INFO:teuthology.orchestra.run.smithi097.stdout:create mon.a on 2024-03-21T13:20:35.601 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-21T13:20:35.740 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-21T13:20:35.916 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-a5dd1f32-e785-11ee-95cd-87774f69a715.target → /etc/systemd/system/ceph-a5dd1f32-e785-11ee-95cd-87774f69a715.target. 2024-03-21T13:20:35.917 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-a5dd1f32-e785-11ee-95cd-87774f69a715.target → /etc/systemd/system/ceph-a5dd1f32-e785-11ee-95cd-87774f69a715.target. 2024-03-21T13:20:36.154 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@mon.a 2024-03-21T13:20:36.154 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Failed to reset failed state of unit ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@mon.a.service: Unit ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@mon.a.service not loaded. 2024-03-21T13:20:36.312 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-a5dd1f32-e785-11ee-95cd-87774f69a715.target.wants/ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@mon.a.service → /etc/systemd/system/ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@.service. 2024-03-21T13:20:36.726 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:36 smithi097 podman[29523]: 2024-03-21 13:20:36.45564708 +0000 UTC m=+0.017755383 image pull 19e12293512ff7978c8adb5f72a829060dd47f07bfd6903b6b3178d2ebc29e4e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 2024-03-21T13:20:36.726 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:36 smithi097 podman[29523]: 2024-03-21 13:20:36.726224205 +0000 UTC m=+0.288332482 container init 8abcf391f46dd02b2af889281ec2defd48d068e0206e9a90ab3e6567397e1dd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mon-a, ceph=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T13:20:36.831 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-21T13:20:36.831 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout disabled 2024-03-21T13:20:36.837 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-21T13:20:36.838 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout inactive 2024-03-21T13:20:36.838 INFO:teuthology.orchestra.run.smithi097.stdout:firewalld.service is not enabled 2024-03-21T13:20:36.838 INFO:teuthology.orchestra.run.smithi097.stdout:Not possible to enable service . firewalld.service is not available 2024-03-21T13:20:36.838 INFO:teuthology.orchestra.run.smithi097.stdout:Waiting for mon to start... 2024-03-21T13:20:36.838 INFO:teuthology.orchestra.run.smithi097.stdout:Waiting for mon... 2024-03-21T13:20:37.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:36 smithi097 podman[29523]: 2024-03-21 13:20:36.731596979 +0000 UTC m=+0.293705257 container start 8abcf391f46dd02b2af889281ec2defd48d068e0206e9a90ab3e6567397e1dd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mon-a, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T13:20:37.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:36 smithi097 ceph-mon[29561]: mkfs a5dd1f32-e785-11ee-95cd-87774f69a715 2024-03-21T13:20:37.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:36 smithi097 ceph-mon[29561]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-21T13:20:37.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:36 smithi097 bash[29523]: 8abcf391f46dd02b2af889281ec2defd48d068e0206e9a90ab3e6567397e1dd6 2024-03-21T13:20:37.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:36 smithi097 systemd[1]: Started Ceph mon.a for a5dd1f32-e785-11ee-95cd-87774f69a715. 2024-03-21T13:20:38.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:37 smithi097 ceph-mon[29561]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-21T13:20:38.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:37 smithi097 ceph-mon[29561]: monmap e1: 1 mons at {a=[v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T13:20:38.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:37 smithi097 ceph-mon[29561]: fsmap 2024-03-21T13:20:38.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:37 smithi097 ceph-mon[29561]: osdmap e1: 0 total, 0 up, 0 in 2024-03-21T13:20:38.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:37 smithi097 ceph-mon[29561]: mgrmap e1: no daemons active 2024-03-21T13:20:38.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:37 smithi097 ceph-mon[29561]: from='client.? 172.21.15.97:0/1377653673' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-21T13:20:38.439 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout cluster: 2024-03-21T13:20:38.439 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout id: a5dd1f32-e785-11ee-95cd-87774f69a715 2024-03-21T13:20:38.439 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-21T13:20:38.439 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-03-21T13:20:38.439 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout services: 2024-03-21T13:20:38.439 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.623756s) 2024-03-21T13:20:38.439 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-21T13:20:38.440 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-21T13:20:38.440 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-03-21T13:20:38.440 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout data: 2024-03-21T13:20:38.440 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-21T13:20:38.440 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-21T13:20:38.440 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-21T13:20:38.440 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout pgs: 2024-03-21T13:20:38.440 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-03-21T13:20:38.440 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:37.328+0000 7fea0c3b5700 1 Processor -- start 2024-03-21T13:20:38.440 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:37.328+0000 7fea0c3b5700 1 -- start start 2024-03-21T13:20:38.440 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:37.328+0000 7fea0c3b5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea04106180 0x7fea041065a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:20:38.440 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:37.328+0000 7fea0c3b5700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea04106b70 con 0x7fea04106180 2024-03-21T13:20:38.440 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:37.328+0000 7fea0a151700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea04106180 0x7fea041065a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:20:38.440 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:37.328+0000 7fea0a151700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea04106180 0x7fea041065a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33300/0 (socket says 172.21.15.97:33300) 2024-03-21T13:20:38.441 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:37.328+0000 7fea0a151700 1 -- 172.21.15.97:0/1218123412 learned_addr learned my addr 172.21.15.97:0/1218123412 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:20:38.441 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:37.328+0000 7fea0a151700 1 -- 172.21.15.97:0/1218123412 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea04107380 con 0x7fea04106180 2024-03-21T13:20:38.441 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:37.329+0000 7fea0a151700 1 --2- 172.21.15.97:0/1218123412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea04106180 0x7fea041065a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe9f40098a0 tx=0x7fe9f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fab63c2db9ad3b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:20:38.441 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:37.329+0000 7fea0914f700 1 -- 172.21.15.97:0/1218123412 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe9f4004160 con 0x7fea04106180 2024-03-21T13:20:38.441 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:37.329+0000 7fea0914f700 1 -- 172.21.15.97:0/1218123412 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f4004300 con 0x7fea04106180 2024-03-21T13:20:38.441 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:37.329+0000 7fea0914f700 1 -- 172.21.15.97:0/1218123412 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe9f40036a0 con 0x7fea04106180 2024-03-21T13:20:38.441 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:37.329+0000 7fea0c3b5700 1 -- 172.21.15.97:0/1218123412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea04106180 msgr2=0x7fea041065a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:20:38.441 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:37.329+0000 7fea0c3b5700 1 --2- 172.21.15.97:0/1218123412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea04106180 0x7fea041065a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe9f40098a0 tx=0x7fe9f4009bb0 comp rx=0 tx=0).stop 2024-03-21T13:20:38.441 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:37.329+0000 7fea0c3b5700 1 -- 172.21.15.97:0/1218123412 shutdown_connections 2024-03-21T13:20:38.441 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:37.329+0000 7fea0c3b5700 1 --2- 172.21.15.97:0/1218123412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea04106180 0x7fea041065a0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:20:38.441 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:37.329+0000 7fea0c3b5700 1 -- 172.21.15.97:0/1218123412 >> 172.21.15.97:0/1218123412 conn(0x7fea04101740 msgr2=0x7fea04103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:20:38.441 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:37.330+0000 7fea0c3b5700 1 -- 172.21.15.97:0/1218123412 shutdown_connections 2024-03-21T13:20:38.441 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:37.330+0000 7fea0c3b5700 1 -- 172.21.15.97:0/1218123412 wait complete. 2024-03-21T13:20:38.441 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:37.330+0000 7fea0c3b5700 1 Processor -- start 2024-03-21T13:20:38.441 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:37.330+0000 7fea0c3b5700 1 -- start start 2024-03-21T13:20:38.442 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:37.330+0000 7fea0c3b5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea04106180 0x7fea04199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:20:38.442 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:37.330+0000 7fea0c3b5700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea041078b0 con 0x7fea04106180 2024-03-21T13:20:38.442 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:37.330+0000 7fea0a151700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea04106180 0x7fea04199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:20:38.442 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:37.330+0000 7fea0a151700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea04106180 0x7fea04199890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33312/0 (socket says 172.21.15.97:33312) 2024-03-21T13:20:38.442 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:37.330+0000 7fea0a151700 1 -- 172.21.15.97:0/1377653673 learned_addr learned my addr 172.21.15.97:0/1377653673 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:20:38.442 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:37.331+0000 7fea0a151700 1 -- 172.21.15.97:0/1377653673 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea04199dd0 con 0x7fea04106180 2024-03-21T13:20:38.442 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:37.331+0000 7fea0a151700 1 --2- 172.21.15.97:0/1377653673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea04106180 0x7fea04199890 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4000c00 tx=0x7fe9f4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:20:38.442 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:37.331+0000 7fe9fb7fe700 1 -- 172.21.15.97:0/1377653673 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe9f4003db0 con 0x7fea04106180 2024-03-21T13:20:38.442 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:37.331+0000 7fe9fb7fe700 1 -- 172.21.15.97:0/1377653673 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f4011630 con 0x7fea04106180 2024-03-21T13:20:38.442 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:37.331+0000 7fea0c3b5700 1 -- 172.21.15.97:0/1377653673 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea0419a060 con 0x7fea04106180 2024-03-21T13:20:38.442 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:37.331+0000 7fe9fb7fe700 1 -- 172.21.15.97:0/1377653673 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe9f4011950 con 0x7fea04106180 2024-03-21T13:20:38.442 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:37.331+0000 7fea0c3b5700 1 -- 172.21.15.97:0/1377653673 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea0419cd50 con 0x7fea04106180 2024-03-21T13:20:38.442 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:37.331+0000 7fe9fb7fe700 1 -- 172.21.15.97:0/1377653673 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fe9f4011af0 con 0x7fea04106180 2024-03-21T13:20:38.442 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:37.331+0000 7fe9fb7fe700 1 -- 172.21.15.97:0/1377653673 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe9f401c450 con 0x7fea04106180 2024-03-21T13:20:38.442 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:37.332+0000 7fea0c3b5700 1 -- 172.21.15.97:0/1377653673 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea040637c0 con 0x7fea04106180 2024-03-21T13:20:38.443 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:37.334+0000 7fe9fb7fe700 1 -- 172.21.15.97:0/1377653673 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fe9f4032020 con 0x7fea04106180 2024-03-21T13:20:38.443 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:37.403+0000 7fea0c3b5700 1 -- 172.21.15.97:0/1377653673 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fea0402da30 con 0x7fea04106180 2024-03-21T13:20:38.443 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:37.403+0000 7fe9fb7fe700 1 -- 172.21.15.97:0/1377653673 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7fe9f4004260 con 0x7fea04106180 2024-03-21T13:20:38.443 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:37.404+0000 7fea0c3b5700 1 -- 172.21.15.97:0/1377653673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea04106180 msgr2=0x7fea04199890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:20:38.443 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:37.404+0000 7fea0c3b5700 1 --2- 172.21.15.97:0/1377653673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea04106180 0x7fea04199890 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4000c00 tx=0x7fe9f4004750 comp rx=0 tx=0).stop 2024-03-21T13:20:38.443 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:37.404+0000 7fea0c3b5700 1 -- 172.21.15.97:0/1377653673 shutdown_connections 2024-03-21T13:20:38.443 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:37.404+0000 7fea0c3b5700 1 --2- 172.21.15.97:0/1377653673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea04106180 0x7fea04199890 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:20:38.443 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:37.404+0000 7fea0c3b5700 1 -- 172.21.15.97:0/1377653673 >> 172.21.15.97:0/1377653673 conn(0x7fea04101740 msgr2=0x7fea04190530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:20:38.443 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:37.404+0000 7fea0c3b5700 1 -- 172.21.15.97:0/1377653673 shutdown_connections 2024-03-21T13:20:38.443 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:37.404+0000 7fea0c3b5700 1 -- 172.21.15.97:0/1377653673 wait complete. 2024-03-21T13:20:38.443 INFO:teuthology.orchestra.run.smithi097.stdout:mon is available 2024-03-21T13:20:38.443 INFO:teuthology.orchestra.run.smithi097.stdout:Assimilating anything we can from ceph.conf... 2024-03-21T13:20:39.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:38 smithi097 ceph-mon[29561]: from='client.? 172.21.15.97:0/236949032' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-21T13:20:39.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:38 smithi097 ceph-mon[29561]: from='client.? 172.21.15.97:0/236949032' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-21T13:20:39.452 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-03-21T13:20:39.453 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout [global] 2024-03-21T13:20:39.453 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout fsid = a5dd1f32-e785-11ee-95cd-87774f69a715 2024-03-21T13:20:39.453 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.97:3300,v1:172.21.15.97:6789] 2024-03-21T13:20:39.453 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-21T13:20:39.453 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-21T13:20:39.453 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-21T13:20:39.453 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-21T13:20:39.453 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-03-21T13:20:39.453 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-21T13:20:39.453 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-21T13:20:39.453 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-03-21T13:20:39.453 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout [osd] 2024-03-21T13:20:39.453 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-21T13:20:39.453 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-21T13:20:39.454 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:38.835+0000 7f8cad465700 1 Processor -- start 2024-03-21T13:20:39.454 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:38.835+0000 7f8cad465700 1 -- start start 2024-03-21T13:20:39.454 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:38.835+0000 7f8cad465700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ca8106180 0x7f8ca81065a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:20:39.454 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:38.835+0000 7f8cad465700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca8106b70 con 0x7f8ca8106180 2024-03-21T13:20:39.454 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:38.836+0000 7f8ca6ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ca8106180 0x7f8ca81065a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:20:39.454 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:38.836+0000 7f8ca6ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ca8106180 0x7f8ca81065a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33314/0 (socket says 172.21.15.97:33314) 2024-03-21T13:20:39.454 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:38.836+0000 7f8ca6ffd700 1 -- 172.21.15.97:0/3615061587 learned_addr learned my addr 172.21.15.97:0/3615061587 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:20:39.454 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:38.836+0000 7f8ca6ffd700 1 -- 172.21.15.97:0/3615061587 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca8107380 con 0x7f8ca8106180 2024-03-21T13:20:39.454 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:38.836+0000 7f8ca6ffd700 1 --2- 172.21.15.97:0/3615061587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ca8106180 0x7f8ca81065a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f8c980098a0 tx=0x7f8c98009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b73bafe082438189 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:20:39.454 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:38.836+0000 7f8ca5ffb700 1 -- 172.21.15.97:0/3615061587 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c98004160 con 0x7f8ca8106180 2024-03-21T13:20:39.454 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:38.836+0000 7f8ca5ffb700 1 -- 172.21.15.97:0/3615061587 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c98004300 con 0x7f8ca8106180 2024-03-21T13:20:39.454 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:38.836+0000 7f8ca5ffb700 1 -- 172.21.15.97:0/3615061587 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c980036a0 con 0x7f8ca8106180 2024-03-21T13:20:39.454 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:38.837+0000 7f8cad465700 1 -- 172.21.15.97:0/3615061587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ca8106180 msgr2=0x7f8ca81065a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:20:39.455 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:38.837+0000 7f8cad465700 1 --2- 172.21.15.97:0/3615061587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ca8106180 0x7f8ca81065a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f8c980098a0 tx=0x7f8c98009bb0 comp rx=0 tx=0).stop 2024-03-21T13:20:39.455 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:38.837+0000 7f8cad465700 1 -- 172.21.15.97:0/3615061587 shutdown_connections 2024-03-21T13:20:39.455 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:38.837+0000 7f8cad465700 1 --2- 172.21.15.97:0/3615061587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ca8106180 0x7f8ca81065a0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:20:39.455 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:38.837+0000 7f8cad465700 1 -- 172.21.15.97:0/3615061587 >> 172.21.15.97:0/3615061587 conn(0x7f8ca8101740 msgr2=0x7f8ca8103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:20:39.455 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:38.837+0000 7f8cad465700 1 -- 172.21.15.97:0/3615061587 shutdown_connections 2024-03-21T13:20:39.455 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:38.837+0000 7f8cad465700 1 -- 172.21.15.97:0/3615061587 wait complete. 2024-03-21T13:20:39.455 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:38.837+0000 7f8cad465700 1 Processor -- start 2024-03-21T13:20:39.455 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:38.838+0000 7f8cad465700 1 -- start start 2024-03-21T13:20:39.455 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:38.838+0000 7f8cad465700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ca8106180 0x7f8ca8199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:20:39.455 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:38.838+0000 7f8cad465700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca81078b0 con 0x7f8ca8106180 2024-03-21T13:20:39.455 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:38.838+0000 7f8ca6ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ca8106180 0x7f8ca8199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:20:39.455 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:38.838+0000 7f8ca6ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ca8106180 0x7f8ca8199890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33328/0 (socket says 172.21.15.97:33328) 2024-03-21T13:20:39.455 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:38.838+0000 7f8ca6ffd700 1 -- 172.21.15.97:0/236949032 learned_addr learned my addr 172.21.15.97:0/236949032 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:20:39.455 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:38.838+0000 7f8ca6ffd700 1 -- 172.21.15.97:0/236949032 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca8199dd0 con 0x7f8ca8106180 2024-03-21T13:20:39.455 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:38.838+0000 7f8ca6ffd700 1 --2- 172.21.15.97:0/236949032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ca8106180 0x7f8ca8199890 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f8c98000c00 tx=0x7f8c98004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:20:39.455 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:38.838+0000 7f8c8ffff700 1 -- 172.21.15.97:0/236949032 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c98003db0 con 0x7f8ca8106180 2024-03-21T13:20:39.456 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:38.838+0000 7f8c8ffff700 1 -- 172.21.15.97:0/236949032 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c98011630 con 0x7f8ca8106180 2024-03-21T13:20:39.456 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:38.838+0000 7f8cad465700 1 -- 172.21.15.97:0/236949032 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ca819a060 con 0x7f8ca8106180 2024-03-21T13:20:39.456 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:38.838+0000 7f8c8ffff700 1 -- 172.21.15.97:0/236949032 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c98011950 con 0x7f8ca8106180 2024-03-21T13:20:39.456 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:38.839+0000 7f8cad465700 1 -- 172.21.15.97:0/236949032 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ca819cd50 con 0x7f8ca8106180 2024-03-21T13:20:39.456 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:38.839+0000 7f8c8ffff700 1 -- 172.21.15.97:0/236949032 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f8c98011af0 con 0x7f8ca8106180 2024-03-21T13:20:39.456 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:38.839+0000 7f8c8ffff700 1 -- 172.21.15.97:0/236949032 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8c9801c450 con 0x7f8ca8106180 2024-03-21T13:20:39.456 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:38.839+0000 7f8cad465700 1 -- 172.21.15.97:0/236949032 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ca80637c0 con 0x7f8ca8106180 2024-03-21T13:20:39.456 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:38.842+0000 7f8c8ffff700 1 -- 172.21.15.97:0/236949032 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f8c98032020 con 0x7f8ca8106180 2024-03-21T13:20:39.456 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:38.910+0000 7f8cad465700 1 -- 172.21.15.97:0/236949032 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f8ca8192420 con 0x7f8ca8106180 2024-03-21T13:20:39.456 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:38.913+0000 7f8c8ffff700 1 -- 172.21.15.97:0/236949032 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+338 (secure 0 0 0) 0x7f8c98004260 con 0x7f8ca8106180 2024-03-21T13:20:39.456 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:38.913+0000 7f8c8ffff700 1 -- 172.21.15.97:0/236949032 <== mon.0 v2:172.21.15.97:3300/0 8 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f8c9801c6a0 con 0x7f8ca8106180 2024-03-21T13:20:39.456 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:38.914+0000 7f8cad465700 1 -- 172.21.15.97:0/236949032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ca8106180 msgr2=0x7f8ca8199890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:20:39.456 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:38.914+0000 7f8cad465700 1 --2- 172.21.15.97:0/236949032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ca8106180 0x7f8ca8199890 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f8c98000c00 tx=0x7f8c98004750 comp rx=0 tx=0).stop 2024-03-21T13:20:39.456 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:38.914+0000 7f8cad465700 1 -- 172.21.15.97:0/236949032 shutdown_connections 2024-03-21T13:20:39.456 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:38.914+0000 7f8cad465700 1 --2- 172.21.15.97:0/236949032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ca8106180 0x7f8ca8199890 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:20:39.457 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:38.914+0000 7f8cad465700 1 -- 172.21.15.97:0/236949032 >> 172.21.15.97:0/236949032 conn(0x7f8ca8101740 msgr2=0x7f8ca8190530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:20:39.457 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:38.914+0000 7f8cad465700 1 -- 172.21.15.97:0/236949032 shutdown_connections 2024-03-21T13:20:39.457 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:38.914+0000 7f8cad465700 1 -- 172.21.15.97:0/236949032 wait complete. 2024-03-21T13:20:39.457 INFO:teuthology.orchestra.run.smithi097.stdout:Generating new minimal ceph.conf... 2024-03-21T13:20:40.275 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:40 smithi097 ceph-mon[29561]: from='client.? 172.21.15.97:0/2278247654' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:20:40.549 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:39.912+0000 7f9423a44700 1 Processor -- start 2024-03-21T13:20:40.550 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:39.912+0000 7f9423a44700 1 -- start start 2024-03-21T13:20:40.550 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:39.912+0000 7f9423a44700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f941c104400 0x7f941c104820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:20:40.550 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:39.912+0000 7f9423a44700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941c104df0 con 0x7f941c104400 2024-03-21T13:20:40.550 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:39.913+0000 7f94217e0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f941c104400 0x7f941c104820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:20:40.550 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:39.913+0000 7f94217e0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f941c104400 0x7f941c104820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33332/0 (socket says 172.21.15.97:33332) 2024-03-21T13:20:40.550 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:39.913+0000 7f94217e0700 1 -- 172.21.15.97:0/1012692559 learned_addr learned my addr 172.21.15.97:0/1012692559 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:20:40.550 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:39.913+0000 7f94217e0700 1 -- 172.21.15.97:0/1012692559 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f941c1073e0 con 0x7f941c104400 2024-03-21T13:20:40.550 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:39.913+0000 7f94217e0700 1 --2- 172.21.15.97:0/1012692559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f941c104400 0x7f941c104820 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9418009e80 tx=0x7f941800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a657543ac55b0039 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:20:40.550 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:39.913+0000 7f9413fff700 1 -- 172.21.15.97:0/1012692559 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9418017070 con 0x7f941c104400 2024-03-21T13:20:40.550 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:39.913+0000 7f9413fff700 1 -- 172.21.15.97:0/1012692559 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f941800b8c0 con 0x7f941c104400 2024-03-21T13:20:40.550 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:39.913+0000 7f9413fff700 1 -- 172.21.15.97:0/1012692559 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9418004470 con 0x7f941c104400 2024-03-21T13:20:40.550 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:39.914+0000 7f9423a44700 1 -- 172.21.15.97:0/1012692559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f941c104400 msgr2=0x7f941c104820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:20:40.551 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:39.914+0000 7f9423a44700 1 --2- 172.21.15.97:0/1012692559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f941c104400 0x7f941c104820 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9418009e80 tx=0x7f941800b330 comp rx=0 tx=0).stop 2024-03-21T13:20:40.551 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:39.914+0000 7f9423a44700 1 -- 172.21.15.97:0/1012692559 shutdown_connections 2024-03-21T13:20:40.551 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:39.914+0000 7f9423a44700 1 --2- 172.21.15.97:0/1012692559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f941c104400 0x7f941c104820 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:20:40.551 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:39.914+0000 7f9423a44700 1 -- 172.21.15.97:0/1012692559 >> 172.21.15.97:0/1012692559 conn(0x7f941c0ffd40 msgr2=0x7f941c1021a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:20:40.551 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:39.914+0000 7f9423a44700 1 -- 172.21.15.97:0/1012692559 shutdown_connections 2024-03-21T13:20:40.551 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:39.914+0000 7f9423a44700 1 -- 172.21.15.97:0/1012692559 wait complete. 2024-03-21T13:20:40.551 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:39.914+0000 7f9423a44700 1 Processor -- start 2024-03-21T13:20:40.551 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:39.915+0000 7f9423a44700 1 -- start start 2024-03-21T13:20:40.551 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:39.915+0000 7f9423a44700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f941c104400 0x7f941c19a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:20:40.551 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:39.915+0000 7f9423a44700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941c10a790 con 0x7f941c104400 2024-03-21T13:20:40.551 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:39.915+0000 7f94217e0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f941c104400 0x7f941c19a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:20:40.551 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:39.915+0000 7f94217e0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f941c104400 0x7f941c19a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33346/0 (socket says 172.21.15.97:33346) 2024-03-21T13:20:40.552 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:39.915+0000 7f94217e0700 1 -- 172.21.15.97:0/2278247654 learned_addr learned my addr 172.21.15.97:0/2278247654 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:20:40.552 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:39.915+0000 7f94217e0700 1 -- 172.21.15.97:0/2278247654 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f941c19a5b0 con 0x7f941c104400 2024-03-21T13:20:40.552 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:39.915+0000 7f94217e0700 1 --2- 172.21.15.97:0/2278247654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f941c104400 0x7f941c19a070 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f94180072a0 tx=0x7f94180095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:20:40.552 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:39.915+0000 7f94127fc700 1 -- 172.21.15.97:0/2278247654 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9418021020 con 0x7f941c104400 2024-03-21T13:20:40.552 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:39.915+0000 7f94127fc700 1 -- 172.21.15.97:0/2278247654 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f9418003d40 con 0x7f941c104400 2024-03-21T13:20:40.552 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:39.915+0000 7f9423a44700 1 -- 172.21.15.97:0/2278247654 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f941c19a840 con 0x7f941c104400 2024-03-21T13:20:40.552 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:39.915+0000 7f94127fc700 1 -- 172.21.15.97:0/2278247654 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9418017040 con 0x7f941c104400 2024-03-21T13:20:40.552 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:39.915+0000 7f9423a44700 1 -- 172.21.15.97:0/2278247654 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f941c19d530 con 0x7f941c104400 2024-03-21T13:20:40.552 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:39.915+0000 7f94127fc700 1 -- 172.21.15.97:0/2278247654 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f9418024020 con 0x7f941c104400 2024-03-21T13:20:40.552 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:39.916+0000 7f94127fc700 1 -- 172.21.15.97:0/2278247654 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f941800f050 con 0x7f941c104400 2024-03-21T13:20:40.552 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:39.916+0000 7f9423a44700 1 -- 172.21.15.97:0/2278247654 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f941c0637c0 con 0x7f941c104400 2024-03-21T13:20:40.552 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:39.918+0000 7f94127fc700 1 -- 172.21.15.97:0/2278247654 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f941802b020 con 0x7f941c104400 2024-03-21T13:20:40.552 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:39.986+0000 7f9423a44700 1 -- 172.21.15.97:0/2278247654 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f941c19d910 con 0x7f941c104400 2024-03-21T13:20:40.552 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:39.987+0000 7f94127fc700 1 -- 172.21.15.97:0/2278247654 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f941800b860 con 0x7f941c104400 2024-03-21T13:20:40.553 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:39.987+0000 7f9423a44700 1 -- 172.21.15.97:0/2278247654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f941c104400 msgr2=0x7f941c19a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:20:40.553 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:39.987+0000 7f9423a44700 1 --2- 172.21.15.97:0/2278247654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f941c104400 0x7f941c19a070 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f94180072a0 tx=0x7f94180095a0 comp rx=0 tx=0).stop 2024-03-21T13:20:40.553 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:39.987+0000 7f9423a44700 1 -- 172.21.15.97:0/2278247654 shutdown_connections 2024-03-21T13:20:40.553 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:39.987+0000 7f9423a44700 1 --2- 172.21.15.97:0/2278247654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f941c104400 0x7f941c19a070 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:20:40.553 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:39.987+0000 7f9423a44700 1 -- 172.21.15.97:0/2278247654 >> 172.21.15.97:0/2278247654 conn(0x7f941c0ffd40 msgr2=0x7f941c1007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:20:40.553 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:39.987+0000 7f9423a44700 1 -- 172.21.15.97:0/2278247654 shutdown_connections 2024-03-21T13:20:40.553 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:39.988+0000 7f9423a44700 1 -- 172.21.15.97:0/2278247654 wait complete. 2024-03-21T13:20:40.553 INFO:teuthology.orchestra.run.smithi097.stdout:Restarting the monitor... 2024-03-21T13:20:40.799 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:40 smithi097 systemd[1]: Stopping Ceph mon.a for a5dd1f32-e785-11ee-95cd-87774f69a715... 2024-03-21T13:20:41.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:40 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mon-a[29538]: 2024-03-21T13:20:40.798+0000 7f2dd1da4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T13:20:41.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:40 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mon-a[29538]: 2024-03-21T13:20:40.798+0000 7f2dd1da4700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-21T13:20:41.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:40 smithi097 podman[29883]: 2024-03-21 13:20:40.983277752 +0000 UTC m=+0.281612810 container died 8abcf391f46dd02b2af889281ec2defd48d068e0206e9a90ab3e6567397e1dd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mon-a, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe) 2024-03-21T13:20:41.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:41 smithi097 podman[29883]: 2024-03-21 13:20:41.345509143 +0000 UTC m=+0.643844203 container cleanup 8abcf391f46dd02b2af889281ec2defd48d068e0206e9a90ab3e6567397e1dd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mon-a, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240319) 2024-03-21T13:20:41.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:41 smithi097 bash[29883]: ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mon-a 2024-03-21T13:20:41.902 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:41 smithi097 podman[29902]: 2024-03-21 13:20:41.554661765 +0000 UTC m=+0.573628539 container remove 8abcf391f46dd02b2af889281ec2defd48d068e0206e9a90ab3e6567397e1dd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mon-a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-21T13:20:41.927 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:41 smithi097 systemd[1]: ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-21T13:20:41.927 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:41 smithi097 systemd[1]: Stopped Ceph mon.a for a5dd1f32-e785-11ee-95cd-87774f69a715. 2024-03-21T13:20:41.927 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:41 smithi097 systemd[1]: Starting Ceph mon.a for a5dd1f32-e785-11ee-95cd-87774f69a715... 2024-03-21T13:20:42.184 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:41 smithi097 podman[30002]: 2024-03-21 13:20:41.803955751 +0000 UTC m=+0.017760203 image pull 19e12293512ff7978c8adb5f72a829060dd47f07bfd6903b6b3178d2ebc29e4e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 2024-03-21T13:20:42.184 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:41 smithi097 podman[30002]: 2024-03-21 13:20:41.905667477 +0000 UTC m=+0.119471971 container create d6b35fa2e2973371234c332ac19ae07729d357ab56ff0c4b075b0092a84f4b5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mon-a, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-21T13:20:42.184 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 podman[30002]: 2024-03-21 13:20:42.075218774 +0000 UTC m=+0.289023236 container init d6b35fa2e2973371234c332ac19ae07729d357ab56ff0c4b075b0092a84f4b5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-21T13:20:42.184 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 podman[30002]: 2024-03-21 13:20:42.080309705 +0000 UTC m=+0.294114160 container start d6b35fa2e2973371234c332ac19ae07729d357ab56ff0c4b075b0092a84f4b5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mon-a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-21T13:20:42.184 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: set uid:gid to 167:167 (ceph:ceph) 2024-03-21T13:20:42.185 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: ceph version 19.0.0-2267-g960fbbe4 (960fbbe4645c2becc8084669be194c95183c7753) squid (dev), process ceph-mon, pid 7 2024-03-21T13:20:42.185 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: pidfile_write: ignore empty --pid-file 2024-03-21T13:20:42.185 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: load: jerasure load: lrc 2024-03-21T13:20:42.185 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: RocksDB version: 7.9.2 2024-03-21T13:20:42.185 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Git sha 0 2024-03-21T13:20:42.185 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Compile date 2024-03-21 08:53:26 2024-03-21T13:20:42.185 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: DB SUMMARY 2024-03-21T13:20:42.185 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: DB Session ID: HCSJ9D9B7TYDEWCMYS0R 2024-03-21T13:20:42.185 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: CURRENT file: CURRENT 2024-03-21T13:20:42.185 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: IDENTITY file: IDENTITY 2024-03-21T13:20:42.185 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-21T13:20:42.185 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-03-21T13:20:42.185 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 88029 ; 2024-03-21T13:20:42.187 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.error_if_exists: 0 2024-03-21T13:20:42.187 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.create_if_missing: 0 2024-03-21T13:20:42.187 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.paranoid_checks: 1 2024-03-21T13:20:42.187 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-21T13:20:42.187 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-21T13:20:42.187 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-21T13:20:42.188 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.env: 0x559d2b7a3b20 2024-03-21T13:20:42.188 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.fs: PosixFileSystem 2024-03-21T13:20:42.188 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.info_log: 0x559d2cf6f920 2024-03-21T13:20:42.188 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.max_file_opening_threads: 16 2024-03-21T13:20:42.188 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.statistics: (nil) 2024-03-21T13:20:42.188 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.use_fsync: 0 2024-03-21T13:20:42.188 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.max_log_file_size: 0 2024-03-21T13:20:42.188 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-21T13:20:42.188 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-21T13:20:42.188 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.keep_log_file_num: 1000 2024-03-21T13:20:42.188 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.recycle_log_file_num: 0 2024-03-21T13:20:42.188 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.allow_fallocate: 1 2024-03-21T13:20:42.188 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.allow_mmap_reads: 0 2024-03-21T13:20:42.188 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.allow_mmap_writes: 0 2024-03-21T13:20:42.188 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.use_direct_reads: 0 2024-03-21T13:20:42.189 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-21T13:20:42.189 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.create_missing_column_families: 0 2024-03-21T13:20:42.189 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.db_log_dir: 2024-03-21T13:20:42.189 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.wal_dir: 2024-03-21T13:20:42.189 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-21T13:20:42.189 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-21T13:20:42.189 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-21T13:20:42.189 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-21T13:20:42.189 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-21T13:20:42.189 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-21T13:20:42.189 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.advise_random_on_open: 1 2024-03-21T13:20:42.189 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.db_write_buffer_size: 0 2024-03-21T13:20:42.189 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.write_buffer_manager: 0x559d2cf401e0 2024-03-21T13:20:42.190 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-21T13:20:42.190 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-21T13:20:42.190 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-21T13:20:42.190 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.rate_limiter: (nil) 2024-03-21T13:20:42.190 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-21T13:20:42.190 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.wal_recovery_mode: 2 2024-03-21T13:20:42.190 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.enable_thread_tracking: 0 2024-03-21T13:20:42.190 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.enable_pipelined_write: 0 2024-03-21T13:20:42.190 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.unordered_write: 0 2024-03-21T13:20:42.190 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-21T13:20:42.190 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-21T13:20:42.191 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-21T13:20:42.191 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-21T13:20:42.191 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.row_cache: None 2024-03-21T13:20:42.191 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.wal_filter: None 2024-03-21T13:20:42.191 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-21T13:20:42.191 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.allow_ingest_behind: 0 2024-03-21T13:20:42.191 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.two_write_queues: 0 2024-03-21T13:20:42.191 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.manual_wal_flush: 0 2024-03-21T13:20:42.191 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.wal_compression: 0 2024-03-21T13:20:42.191 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.atomic_flush: 0 2024-03-21T13:20:42.191 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-21T13:20:42.191 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-21T13:20:42.191 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-21T13:20:42.191 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.log_readahead_size: 0 2024-03-21T13:20:42.191 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-21T13:20:42.191 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.best_efforts_recovery: 0 2024-03-21T13:20:42.191 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-21T13:20:42.192 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-21T13:20:42.192 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.allow_data_in_errors: 0 2024-03-21T13:20:42.192 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.db_host_id: __hostname__ 2024-03-21T13:20:42.192 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-21T13:20:42.192 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.max_background_jobs: 2 2024-03-21T13:20:42.192 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.max_background_compactions: -1 2024-03-21T13:20:42.192 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.max_subcompactions: 1 2024-03-21T13:20:42.192 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-21T13:20:42.192 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-21T13:20:42.192 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-21T13:20:42.192 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.max_total_wal_size: 0 2024-03-21T13:20:42.192 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-21T13:20:42.193 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-21T13:20:42.193 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-21T13:20:42.193 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-21T13:20:42.193 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.max_open_files: -1 2024-03-21T13:20:42.193 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.bytes_per_sync: 0 2024-03-21T13:20:42.193 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-21T13:20:42.193 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-21T13:20:42.193 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.compaction_readahead_size: 0 2024-03-21T13:20:42.193 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.max_background_flushes: -1 2024-03-21T13:20:42.193 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Compression algorithms supported: 2024-03-21T13:20:42.193 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-21T13:20:42.193 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: kZSTD supported: 0 2024-03-21T13:20:42.193 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: kXpressCompression supported: 0 2024-03-21T13:20:42.194 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: kLZ4HCCompression supported: 1 2024-03-21T13:20:42.194 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: kZlibCompression supported: 1 2024-03-21T13:20:42.194 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: kSnappyCompression supported: 1 2024-03-21T13:20:42.194 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: kLZ4Compression supported: 1 2024-03-21T13:20:42.194 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: kBZip2Compression supported: 0 2024-03-21T13:20:42.194 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-21T13:20:42.194 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-21T13:20:42.194 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-21T13:20:42.194 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-21T13:20:42.194 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-21T13:20:42.194 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.merge_operator: 2024-03-21T13:20:42.194 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.compaction_filter: None 2024-03-21T13:20:42.194 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.compaction_filter_factory: None 2024-03-21T13:20:42.194 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.sst_partitioner_factory: None 2024-03-21T13:20:42.195 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-21T13:20:42.195 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-21T13:20:42.195 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x559d2cf6fa20) 2024-03-21T13:20:42.195 INFO:journalctl@ceph.mon.a.smithi097.stdout: cache_index_and_filter_blocks: 1 2024-03-21T13:20:42.195 INFO:journalctl@ceph.mon.a.smithi097.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-21T13:20:42.195 INFO:journalctl@ceph.mon.a.smithi097.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-21T13:20:42.195 INFO:journalctl@ceph.mon.a.smithi097.stdout: pin_top_level_index_and_filter: 1 2024-03-21T13:20:42.195 INFO:journalctl@ceph.mon.a.smithi097.stdout: index_type: 0 2024-03-21T13:20:42.195 INFO:journalctl@ceph.mon.a.smithi097.stdout: data_block_index_type: 0 2024-03-21T13:20:42.195 INFO:journalctl@ceph.mon.a.smithi097.stdout: index_shortening: 1 2024-03-21T13:20:42.195 INFO:journalctl@ceph.mon.a.smithi097.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-21T13:20:42.195 INFO:journalctl@ceph.mon.a.smithi097.stdout: checksum: 4 2024-03-21T13:20:42.196 INFO:journalctl@ceph.mon.a.smithi097.stdout: no_block_cache: 0 2024-03-21T13:20:42.196 INFO:journalctl@ceph.mon.a.smithi097.stdout: block_cache: 0x559d2cf73090 2024-03-21T13:20:42.196 INFO:journalctl@ceph.mon.a.smithi097.stdout: block_cache_name: BinnedLRUCache 2024-03-21T13:20:42.196 INFO:journalctl@ceph.mon.a.smithi097.stdout: block_cache_options: 2024-03-21T13:20:42.196 INFO:journalctl@ceph.mon.a.smithi097.stdout: capacity : 536870912 2024-03-21T13:20:42.196 INFO:journalctl@ceph.mon.a.smithi097.stdout: num_shard_bits : 4 2024-03-21T13:20:42.196 INFO:journalctl@ceph.mon.a.smithi097.stdout: strict_capacity_limit : 0 2024-03-21T13:20:42.196 INFO:journalctl@ceph.mon.a.smithi097.stdout: high_pri_pool_ratio: 0.000 2024-03-21T13:20:42.196 INFO:journalctl@ceph.mon.a.smithi097.stdout: block_cache_compressed: (nil) 2024-03-21T13:20:42.196 INFO:journalctl@ceph.mon.a.smithi097.stdout: persistent_cache: (nil) 2024-03-21T13:20:42.196 INFO:journalctl@ceph.mon.a.smithi097.stdout: block_size: 4096 2024-03-21T13:20:42.196 INFO:journalctl@ceph.mon.a.smithi097.stdout: block_size_deviation: 10 2024-03-21T13:20:42.196 INFO:journalctl@ceph.mon.a.smithi097.stdout: block_restart_interval: 16 2024-03-21T13:20:42.196 INFO:journalctl@ceph.mon.a.smithi097.stdout: index_block_restart_interval: 1 2024-03-21T13:20:42.197 INFO:journalctl@ceph.mon.a.smithi097.stdout: metadata_block_size: 4096 2024-03-21T13:20:42.197 INFO:journalctl@ceph.mon.a.smithi097.stdout: partition_filters: 0 2024-03-21T13:20:42.197 INFO:journalctl@ceph.mon.a.smithi097.stdout: use_delta_encoding: 1 2024-03-21T13:20:42.197 INFO:journalctl@ceph.mon.a.smithi097.stdout: filter_policy: bloomfilter 2024-03-21T13:20:42.197 INFO:journalctl@ceph.mon.a.smithi097.stdout: whole_key_filtering: 1 2024-03-21T13:20:42.197 INFO:journalctl@ceph.mon.a.smithi097.stdout: verify_compression: 0 2024-03-21T13:20:42.197 INFO:journalctl@ceph.mon.a.smithi097.stdout: read_amp_bytes_per_bit: 0 2024-03-21T13:20:42.197 INFO:journalctl@ceph.mon.a.smithi097.stdout: format_version: 5 2024-03-21T13:20:42.197 INFO:journalctl@ceph.mon.a.smithi097.stdout: enable_index_compression: 1 2024-03-21T13:20:42.197 INFO:journalctl@ceph.mon.a.smithi097.stdout: block_align: 0 2024-03-21T13:20:42.197 INFO:journalctl@ceph.mon.a.smithi097.stdout: max_auto_readahead_size: 262144 2024-03-21T13:20:42.197 INFO:journalctl@ceph.mon.a.smithi097.stdout: prepopulate_block_cache: 0 2024-03-21T13:20:42.197 INFO:journalctl@ceph.mon.a.smithi097.stdout: initial_auto_readahead_size: 8192 2024-03-21T13:20:42.197 INFO:journalctl@ceph.mon.a.smithi097.stdout: num_file_reads_for_auto_readahead: 2 2024-03-21T13:20:42.197 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.write_buffer_size: 33554432 2024-03-21T13:20:42.198 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.max_write_buffer_number: 2 2024-03-21T13:20:42.198 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.compression: NoCompression 2024-03-21T13:20:42.198 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.bottommost_compression: Disabled 2024-03-21T13:20:42.198 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.prefix_extractor: nullptr 2024-03-21T13:20:42.198 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-21T13:20:42.198 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.num_levels: 7 2024-03-21T13:20:42.198 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-21T13:20:42.198 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-21T13:20:42.198 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-21T13:20:42.198 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-21T13:20:42.198 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-21T13:20:42.199 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-21T13:20:42.199 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-21T13:20:42.199 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-21T13:20:42.199 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-21T13:20:42.199 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-21T13:20:42.199 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-21T13:20:42.199 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-21T13:20:42.199 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-21T13:20:42.199 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.compression_opts.level: 32767 2024-03-21T13:20:42.199 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.compression_opts.strategy: 0 2024-03-21T13:20:42.199 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-21T13:20:42.199 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-21T13:20:42.199 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-21T13:20:42.199 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-21T13:20:42.199 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.compression_opts.enabled: false 2024-03-21T13:20:42.199 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-21T13:20:42.200 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-21T13:20:42.200 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-21T13:20:42.200 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-21T13:20:42.200 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.target_file_size_base: 67108864 2024-03-21T13:20:42.200 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-21T13:20:42.200 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-21T13:20:42.200 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-21T13:20:42.200 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-21T13:20:42.200 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-21T13:20:42.200 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-21T13:20:42.200 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-21T13:20:42.200 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-21T13:20:42.200 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-21T13:20:42.200 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-21T13:20:42.200 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-21T13:20:42.201 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-21T13:20:42.201 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-21T13:20:42.201 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-21T13:20:42.201 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.arena_block_size: 1048576 2024-03-21T13:20:42.201 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-21T13:20:42.201 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-21T13:20:42.201 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.disable_auto_compactions: 0 2024-03-21T13:20:42.201 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-21T13:20:42.201 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-21T13:20:42.201 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-21T13:20:42.201 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-21T13:20:42.201 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-21T13:20:42.201 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-21T13:20:42.201 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-21T13:20:42.201 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-21T13:20:42.201 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-21T13:20:42.202 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-21T13:20:42.202 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.table_properties_collectors: 2024-03-21T13:20:42.202 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.inplace_update_support: 0 2024-03-21T13:20:42.202 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-21T13:20:42.202 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-21T13:20:42.202 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-21T13:20:42.202 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-21T13:20:42.202 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.bloom_locality: 0 2024-03-21T13:20:42.202 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.max_successive_merges: 0 2024-03-21T13:20:42.202 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-21T13:20:42.202 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.paranoid_file_checks: 0 2024-03-21T13:20:42.202 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.force_consistency_checks: 1 2024-03-21T13:20:42.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.report_bg_io_stats: 0 2024-03-21T13:20:42.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.ttl: 2592000 2024-03-21T13:20:42.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-21T13:20:42.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-21T13:20:42.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-21T13:20:42.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.enable_blob_files: false 2024-03-21T13:20:42.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.min_blob_size: 0 2024-03-21T13:20:42.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.blob_file_size: 268435456 2024-03-21T13:20:42.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-21T13:20:42.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-21T13:20:42.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-21T13:20:42.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-21T13:20:42.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-21T13:20:42.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.blob_file_starting_level: 0 2024-03-21T13:20:42.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-21T13:20:42.204 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-21T13:20:42.204 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-21T13:20:42.204 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 25c74a39-aaa9-4084-aee8-6c1e49e66e5e 2024-03-21T13:20:42.205 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711027242109309, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-21T13:20:42.205 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-21T13:20:42.205 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711027242110636, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 84277, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 266, "table_properties": {"data_size": 82456, "index_size": 231, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 10814, "raw_average_key_size": 48, "raw_value_size": 76433, "raw_average_value_size": 339, "num_data_blocks": 10, "num_entries": 225, "num_filter_entries": 225, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711027242, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "25c74a39-aaa9-4084-aee8-6c1e49e66e5e", "db_session_id": "HCSJ9D9B7TYDEWCMYS0R", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-21T13:20:42.205 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711027242110741, "job": 1, "event": "recovery_finished"} 2024-03-21T13:20:42.205 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-21T13:20:42.205 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-21T13:20:42.205 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x559d2d062000 2024-03-21T13:20:42.206 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: DB pointer 0x559d2d04c000 2024-03-21T13:20:42.206 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-21T13:20:42.206 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-21T13:20:42.206 INFO:journalctl@ceph.mon.a.smithi097.stdout: ** DB Stats ** 2024-03-21T13:20:42.206 INFO:journalctl@ceph.mon.a.smithi097.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-21T13:20:42.206 INFO:journalctl@ceph.mon.a.smithi097.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-21T13:20:42.206 INFO:journalctl@ceph.mon.a.smithi097.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-21T13:20:42.206 INFO:journalctl@ceph.mon.a.smithi097.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-21T13:20:42.206 INFO:journalctl@ceph.mon.a.smithi097.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-21T13:20:42.206 INFO:journalctl@ceph.mon.a.smithi097.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-21T13:20:42.206 INFO:journalctl@ceph.mon.a.smithi097.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-21T13:20:42.206 INFO:journalctl@ceph.mon.a.smithi097.stdout: 2024-03-21T13:20:42.206 INFO:journalctl@ceph.mon.a.smithi097.stdout: ** Compaction Stats [default] ** 2024-03-21T13:20:42.207 INFO:journalctl@ceph.mon.a.smithi097.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-21T13:20:42.207 INFO:journalctl@ceph.mon.a.smithi097.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-21T13:20:42.207 INFO:journalctl@ceph.mon.a.smithi097.stdout: L0 2/0 84.14 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 96.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-21T13:20:42.207 INFO:journalctl@ceph.mon.a.smithi097.stdout: Sum 2/0 84.14 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 96.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-21T13:20:42.207 INFO:journalctl@ceph.mon.a.smithi097.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 96.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-21T13:20:42.207 INFO:journalctl@ceph.mon.a.smithi097.stdout: 2024-03-21T13:20:42.207 INFO:journalctl@ceph.mon.a.smithi097.stdout: ** Compaction Stats [default] ** 2024-03-21T13:20:42.207 INFO:journalctl@ceph.mon.a.smithi097.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-21T13:20:42.207 INFO:journalctl@ceph.mon.a.smithi097.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-21T13:20:42.207 INFO:journalctl@ceph.mon.a.smithi097.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 96.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-21T13:20:42.207 INFO:journalctl@ceph.mon.a.smithi097.stdout: 2024-03-21T13:20:42.207 INFO:journalctl@ceph.mon.a.smithi097.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-21T13:20:42.207 INFO:journalctl@ceph.mon.a.smithi097.stdout: 2024-03-21T13:20:42.207 INFO:journalctl@ceph.mon.a.smithi097.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-21T13:20:42.208 INFO:journalctl@ceph.mon.a.smithi097.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-21T13:20:42.208 INFO:journalctl@ceph.mon.a.smithi097.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-21T13:20:42.208 INFO:journalctl@ceph.mon.a.smithi097.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-21T13:20:42.208 INFO:journalctl@ceph.mon.a.smithi097.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-21T13:20:42.208 INFO:journalctl@ceph.mon.a.smithi097.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-21T13:20:42.208 INFO:journalctl@ceph.mon.a.smithi097.stdout: Cumulative compaction: 0.00 GB write, 19.43 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-21T13:20:42.208 INFO:journalctl@ceph.mon.a.smithi097.stdout: Interval compaction: 0.00 GB write, 19.43 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-21T13:20:42.208 INFO:journalctl@ceph.mon.a.smithi097.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-21T13:20:42.208 INFO:journalctl@ceph.mon.a.smithi097.stdout: Block cache BinnedLRUCache@0x559d2cf73090#7 capacity: 512.00 MB usage: 1.19 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-03-21T13:20:42.208 INFO:journalctl@ceph.mon.a.smithi097.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.77 KB,0.000146031%) IndexBlock(2,0.42 KB,8.04663e-05%) Misc(1,0.00 KB,0%) 2024-03-21T13:20:42.208 INFO:journalctl@ceph.mon.a.smithi097.stdout: 2024-03-21T13:20:42.208 INFO:journalctl@ceph.mon.a.smithi097.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-21T13:20:42.208 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: starting mon.a rank 0 at public addrs [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] at bind addrs [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid a5dd1f32-e785-11ee-95cd-87774f69a715 2024-03-21T13:20:42.208 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: mon.a@-1(???) e1 preinit fsid a5dd1f32-e785-11ee-95cd-87774f69a715 2024-03-21T13:20:42.208 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: mon.a@-1(???).mds e1 new map 2024-03-21T13:20:42.209 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: mon.a@-1(???).mds e1 print_map 2024-03-21T13:20:42.209 INFO:journalctl@ceph.mon.a.smithi097.stdout: e1 2024-03-21T13:20:42.209 INFO:journalctl@ceph.mon.a.smithi097.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T13:20:42.209 INFO:journalctl@ceph.mon.a.smithi097.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-21T13:20:42.209 INFO:journalctl@ceph.mon.a.smithi097.stdout: legacy client fscid: -1 2024-03-21T13:20:42.209 INFO:journalctl@ceph.mon.a.smithi097.stdout: 2024-03-21T13:20:42.209 INFO:journalctl@ceph.mon.a.smithi097.stdout: No filesystems configured 2024-03-21T13:20:42.209 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-21T13:20:42.209 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T13:20:42.209 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T13:20:42.209 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T13:20:42.209 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-21T13:20:42.209 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-21T13:20:42.209 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: monmap e1: 1 mons at {a=[v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T13:20:42.210 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: fsmap 2024-03-21T13:20:42.210 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: osdmap e1: 0 total, 0 up, 0 in 2024-03-21T13:20:42.210 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 ceph-mon[30040]: mgrmap e1: no daemons active 2024-03-21T13:20:42.213 INFO:teuthology.orchestra.run.smithi097.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-21T13:20:42.438 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 bash[30002]: d6b35fa2e2973371234c332ac19ae07729d357ab56ff0c4b075b0092a84f4b5d 2024-03-21T13:20:42.438 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:42 smithi097 systemd[1]: Started Ceph mon.a for a5dd1f32-e785-11ee-95cd-87774f69a715. 2024-03-21T13:20:43.446 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:42.680+0000 7f4688869700 1 Processor -- start 2024-03-21T13:20:43.446 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:42.680+0000 7f4688869700 1 -- start start 2024-03-21T13:20:43.446 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:42.680+0000 7f4688869700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4684106180 0x7f46841065a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:20:43.446 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:42.680+0000 7f4688869700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4684106b70 con 0x7f4684106180 2024-03-21T13:20:43.446 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:42.681+0000 7f468259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4684106180 0x7f46841065a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:20:43.446 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:42.681+0000 7f468259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4684106180 0x7f46841065a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49254/0 (socket says 172.21.15.97:49254) 2024-03-21T13:20:43.447 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:42.681+0000 7f468259c700 1 -- 172.21.15.97:0/494734123 learned_addr learned my addr 172.21.15.97:0/494734123 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:20:43.447 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:42.681+0000 7f468259c700 1 -- 172.21.15.97:0/494734123 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4684107380 con 0x7f4684106180 2024-03-21T13:20:43.447 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:42.681+0000 7f468259c700 1 --2- 172.21.15.97:0/494734123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4684106180 0x7f46841065a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f467c009e80 tx=0x7f467c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4b59cf9abd6d9c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:20:43.447 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:42.681+0000 7f468159a700 1 -- 172.21.15.97:0/494734123 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f467c017070 con 0x7f4684106180 2024-03-21T13:20:43.447 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:42.681+0000 7f468159a700 1 -- 172.21.15.97:0/494734123 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f467c00b8c0 con 0x7f4684106180 2024-03-21T13:20:43.447 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:42.681+0000 7f468159a700 1 -- 172.21.15.97:0/494734123 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f467c004470 con 0x7f4684106180 2024-03-21T13:20:43.447 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:42.682+0000 7f4688869700 1 -- 172.21.15.97:0/494734123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4684106180 msgr2=0x7f46841065a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:20:43.447 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:42.682+0000 7f4688869700 1 --2- 172.21.15.97:0/494734123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4684106180 0x7f46841065a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f467c009e80 tx=0x7f467c00b330 comp rx=0 tx=0).stop 2024-03-21T13:20:43.447 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:42.682+0000 7f4688869700 1 -- 172.21.15.97:0/494734123 shutdown_connections 2024-03-21T13:20:43.447 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:42.682+0000 7f4688869700 1 --2- 172.21.15.97:0/494734123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4684106180 0x7f46841065a0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:20:43.447 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:42.682+0000 7f4688869700 1 -- 172.21.15.97:0/494734123 >> 172.21.15.97:0/494734123 conn(0x7f4684101760 msgr2=0x7f4684103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:20:43.447 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:42.682+0000 7f4688869700 1 -- 172.21.15.97:0/494734123 shutdown_connections 2024-03-21T13:20:43.447 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:42.682+0000 7f4688869700 1 -- 172.21.15.97:0/494734123 wait complete. 2024-03-21T13:20:43.447 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:42.683+0000 7f4688869700 1 Processor -- start 2024-03-21T13:20:43.447 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:42.683+0000 7f4688869700 1 -- start start 2024-03-21T13:20:43.448 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:42.683+0000 7f4688869700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4684106180 0x7f468419a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:20:43.448 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:42.683+0000 7f4688869700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468410a730 con 0x7f4684106180 2024-03-21T13:20:43.448 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:42.683+0000 7f468259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4684106180 0x7f468419a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:20:43.448 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:42.683+0000 7f468259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4684106180 0x7f468419a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49268/0 (socket says 172.21.15.97:49268) 2024-03-21T13:20:43.448 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:42.683+0000 7f468259c700 1 -- 172.21.15.97:0/3795975653 learned_addr learned my addr 172.21.15.97:0/3795975653 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:20:43.448 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:42.683+0000 7f468259c700 1 -- 172.21.15.97:0/3795975653 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f468419a5c0 con 0x7f4684106180 2024-03-21T13:20:43.448 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:42.683+0000 7f468259c700 1 --2- 172.21.15.97:0/3795975653 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4684106180 0x7f468419a080 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f467c009fd0 tx=0x7f467c00be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:20:43.448 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:42.683+0000 7f46737fe700 1 -- 172.21.15.97:0/3795975653 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f467c021020 con 0x7f4684106180 2024-03-21T13:20:43.448 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:42.684+0000 7f46737fe700 1 -- 172.21.15.97:0/3795975653 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f467c003d00 con 0x7f4684106180 2024-03-21T13:20:43.448 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:42.684+0000 7f4688869700 1 -- 172.21.15.97:0/3795975653 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f468419a850 con 0x7f4684106180 2024-03-21T13:20:43.448 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:42.684+0000 7f46737fe700 1 -- 172.21.15.97:0/3795975653 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f467c017040 con 0x7f4684106180 2024-03-21T13:20:43.448 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:42.684+0000 7f4688869700 1 -- 172.21.15.97:0/3795975653 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f468419d540 con 0x7f4684106180 2024-03-21T13:20:43.448 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:42.684+0000 7f46737fe700 1 -- 172.21.15.97:0/3795975653 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f467c024020 con 0x7f4684106180 2024-03-21T13:20:43.448 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:42.684+0000 7f46737fe700 1 -- 172.21.15.97:0/3795975653 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f467c00f050 con 0x7f4684106180 2024-03-21T13:20:43.448 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:42.685+0000 7f4688869700 1 -- 172.21.15.97:0/3795975653 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46840637c0 con 0x7f4684106180 2024-03-21T13:20:43.448 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:42.687+0000 7f46737fe700 1 -- 172.21.15.97:0/3795975653 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f467c02b020 con 0x7f4684106180 2024-03-21T13:20:43.449 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:42.753+0000 7f4688869700 1 -- 172.21.15.97:0/3795975653 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f468419d9b0 con 0x7f4684106180 2024-03-21T13:20:43.449 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:42.755+0000 7f46737fe700 1 -- 172.21.15.97:0/3795975653 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f467c026d60 con 0x7f4684106180 2024-03-21T13:20:43.449 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:42.756+0000 7f4688869700 1 -- 172.21.15.97:0/3795975653 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4684106180 msgr2=0x7f468419a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:20:43.449 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:42.756+0000 7f4688869700 1 --2- 172.21.15.97:0/3795975653 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4684106180 0x7f468419a080 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f467c009fd0 tx=0x7f467c00be90 comp rx=0 tx=0).stop 2024-03-21T13:20:43.449 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:42.756+0000 7f4688869700 1 -- 172.21.15.97:0/3795975653 shutdown_connections 2024-03-21T13:20:43.449 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:42.756+0000 7f4688869700 1 --2- 172.21.15.97:0/3795975653 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4684106180 0x7f468419a080 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:20:43.449 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:42.756+0000 7f4688869700 1 -- 172.21.15.97:0/3795975653 >> 172.21.15.97:0/3795975653 conn(0x7f4684101760 msgr2=0x7f4684102220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:20:43.449 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:42.756+0000 7f4688869700 1 -- 172.21.15.97:0/3795975653 shutdown_connections 2024-03-21T13:20:43.449 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:42.756+0000 7f4688869700 1 -- 172.21.15.97:0/3795975653 wait complete. 2024-03-21T13:20:43.449 INFO:teuthology.orchestra.run.smithi097.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-21T13:20:43.472 INFO:teuthology.orchestra.run.smithi097.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-21T13:20:43.472 INFO:teuthology.orchestra.run.smithi097.stdout:Creating mgr... 2024-03-21T13:20:43.473 INFO:teuthology.orchestra.run.smithi097.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-21T13:20:43.473 INFO:teuthology.orchestra.run.smithi097.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-21T13:20:43.730 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@mgr.y 2024-03-21T13:20:43.731 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Failed to reset failed state of unit ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@mgr.y.service: Unit ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@mgr.y.service not loaded. 2024-03-21T13:20:43.883 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-a5dd1f32-e785-11ee-95cd-87774f69a715.target.wants/ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@mgr.y.service → /etc/systemd/system/ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@.service. 2024-03-21T13:20:44.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:43 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3795975653' entity='client.admin' 2024-03-21T13:20:44.053 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:20:43 smithi097 systemd[1]: Starting Ceph mgr.y for a5dd1f32-e785-11ee-95cd-87774f69a715... 2024-03-21T13:20:44.377 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:20:44 smithi097 podman[30282]: 2024-03-21 13:20:44.018031292 +0000 UTC m=+0.017148598 image pull 19e12293512ff7978c8adb5f72a829060dd47f07bfd6903b6b3178d2ebc29e4e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 2024-03-21T13:20:44.377 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:20:44 smithi097 podman[30282]: 2024-03-21 13:20:44.116269869 +0000 UTC m=+0.115387174 container create 6c7ef8cfbe19127a76518a69d0acabf8a5ce304c0734df85b6a001986eba5de1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319) 2024-03-21T13:20:44.377 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:20:44 smithi097 podman[30282]: 2024-03-21 13:20:44.310329606 +0000 UTC m=+0.309446911 container init 6c7ef8cfbe19127a76518a69d0acabf8a5ce304c0734df85b6a001986eba5de1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-21T13:20:44.377 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:20:44 smithi097 podman[30282]: 2024-03-21 13:20:44.315785793 +0000 UTC m=+0.314903101 container start 6c7ef8cfbe19127a76518a69d0acabf8a5ce304c0734df85b6a001986eba5de1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.schema-version=1.0) 2024-03-21T13:20:44.377 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:20:44 smithi097 bash[30282]: 6c7ef8cfbe19127a76518a69d0acabf8a5ce304c0734df85b6a001986eba5de1 2024-03-21T13:20:44.389 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-21T13:20:44.389 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout disabled 2024-03-21T13:20:44.396 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-21T13:20:44.396 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout inactive 2024-03-21T13:20:44.396 INFO:teuthology.orchestra.run.smithi097.stdout:firewalld.service is not enabled 2024-03-21T13:20:44.397 INFO:teuthology.orchestra.run.smithi097.stdout:Not possible to enable service . firewalld.service is not available 2024-03-21T13:20:44.404 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-21T13:20:44.404 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout disabled 2024-03-21T13:20:44.411 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-21T13:20:44.412 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout inactive 2024-03-21T13:20:44.412 INFO:teuthology.orchestra.run.smithi097.stdout:firewalld.service is not enabled 2024-03-21T13:20:44.412 INFO:teuthology.orchestra.run.smithi097.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-03-21T13:20:44.412 INFO:teuthology.orchestra.run.smithi097.stdout:Waiting for mgr to start... 2024-03-21T13:20:44.412 INFO:teuthology.orchestra.run.smithi097.stdout:Waiting for mgr... 2024-03-21T13:20:44.628 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:20:44 smithi097 systemd[1]: Started Ceph mgr.y for a5dd1f32-e785-11ee-95cd-87774f69a715. 2024-03-21T13:20:44.878 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:20:44 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:20:44.872+0000 7f947e89c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-21T13:20:45.266 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:45 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/947709467' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-21T13:20:45.267 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:20:44 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:20:44.971+0000 7f947e89c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-21T13:20:45.552 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-03-21T13:20:45.555 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout { 2024-03-21T13:20:45.555 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "fsid": "a5dd1f32-e785-11ee-95cd-87774f69a715", 2024-03-21T13:20:45.555 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "health": { 2024-03-21T13:20:45.555 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-21T13:20:45.555 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-21T13:20:45.555 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-21T13:20:45.555 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-21T13:20:45.555 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-21T13:20:45.555 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-21T13:20:45.556 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 0 2024-03-21T13:20:45.556 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-03-21T13:20:45.556 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-21T13:20:45.556 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "a" 2024-03-21T13:20:45.556 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-03-21T13:20:45.556 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-03-21T13:20:45.556 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-21T13:20:45.556 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T13:20:45.556 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-21T13:20:45.556 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-21T13:20:45.556 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-21T13:20:45.556 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-21T13:20:45.556 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T13:20:45.556 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-21T13:20:45.557 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-21T13:20:45.557 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-21T13:20:45.564 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-21T13:20:45.564 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-21T13:20:45.564 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-21T13:20:45.564 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-21T13:20:45.564 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-21T13:20:45.564 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-21T13:20:45.564 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-21T13:20:45.564 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-21T13:20:45.564 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-21T13:20:45.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-21T13:20:45.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-21T13:20:45.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-21T13:20:45.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-21T13:20:45.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-21T13:20:45.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-21T13:20:45.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T13:20:45.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-21T13:20:45.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-21T13:20:45.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-21T13:20:45.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-21T13:20:45.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-21T13:20:45.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-21T13:20:45.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-21T13:20:45.566 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-21T13:20:45.566 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-21T13:20:45.566 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "restful" 2024-03-21T13:20:45.566 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-03-21T13:20:45.566 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T13:20:45.566 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-21T13:20:45.566 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-21T13:20:45.566 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T13:20:45.566 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "modified": "2024-03-21T13:20:36.782623+0000", 2024-03-21T13:20:45.566 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T13:20:45.566 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-21T13:20:45.566 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-21T13:20:45.566 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout } 2024-03-21T13:20:45.566 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:44.874+0000 7f89a2dcc700 1 Processor -- start 2024-03-21T13:20:45.567 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:44.874+0000 7f89a2dcc700 1 -- start start 2024-03-21T13:20:45.567 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:44.874+0000 7f89a2dcc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f899c106140 0x7f899c106560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:20:45.567 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:44.874+0000 7f89a2dcc700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f899c106b30 con 0x7f899c106140 2024-03-21T13:20:45.567 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:44.875+0000 7f89a0b68700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f899c106140 0x7f899c106560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:20:45.567 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:44.875+0000 7f89a0b68700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f899c106140 0x7f899c106560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49282/0 (socket says 172.21.15.97:49282) 2024-03-21T13:20:45.567 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:44.875+0000 7f89a0b68700 1 -- 172.21.15.97:0/237839245 learned_addr learned my addr 172.21.15.97:0/237839245 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:20:45.567 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:44.875+0000 7f89a0b68700 1 -- 172.21.15.97:0/237839245 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f899c107340 con 0x7f899c106140 2024-03-21T13:20:45.567 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:44.875+0000 7f89a0b68700 1 --2- 172.21.15.97:0/237839245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f899c106140 0x7f899c106560 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f89900098a0 tx=0x7f8990009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86b74e1ebdf8860b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:20:45.567 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:44.875+0000 7f899b7fe700 1 -- 172.21.15.97:0/237839245 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8990004160 con 0x7f899c106140 2024-03-21T13:20:45.567 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:44.876+0000 7f899b7fe700 1 -- 172.21.15.97:0/237839245 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f8990004300 con 0x7f899c106140 2024-03-21T13:20:45.567 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:44.876+0000 7f899b7fe700 1 -- 172.21.15.97:0/237839245 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8990011a60 con 0x7f899c106140 2024-03-21T13:20:45.567 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:44.876+0000 7f89a2dcc700 1 -- 172.21.15.97:0/237839245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f899c106140 msgr2=0x7f899c106560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:20:45.567 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:44.876+0000 7f89a2dcc700 1 --2- 172.21.15.97:0/237839245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f899c106140 0x7f899c106560 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f89900098a0 tx=0x7f8990009bb0 comp rx=0 tx=0).stop 2024-03-21T13:20:45.568 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:44.876+0000 7f89a2dcc700 1 -- 172.21.15.97:0/237839245 shutdown_connections 2024-03-21T13:20:45.568 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:44.876+0000 7f89a2dcc700 1 --2- 172.21.15.97:0/237839245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f899c106140 0x7f899c106560 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:20:45.568 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:44.876+0000 7f89a2dcc700 1 -- 172.21.15.97:0/237839245 >> 172.21.15.97:0/237839245 conn(0x7f899c1016a0 msgr2=0x7f899c103b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:20:45.568 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:44.876+0000 7f89a2dcc700 1 -- 172.21.15.97:0/237839245 shutdown_connections 2024-03-21T13:20:45.568 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:44.876+0000 7f89a2dcc700 1 -- 172.21.15.97:0/237839245 wait complete. 2024-03-21T13:20:45.568 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:44.877+0000 7f89a2dcc700 1 Processor -- start 2024-03-21T13:20:45.568 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:44.877+0000 7f89a2dcc700 1 -- start start 2024-03-21T13:20:45.568 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:44.877+0000 7f89a2dcc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f899c106140 0x7f899c19a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:20:45.568 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:44.877+0000 7f89a2dcc700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f899c10a6f0 con 0x7f899c106140 2024-03-21T13:20:45.568 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:44.877+0000 7f89a0b68700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f899c106140 0x7f899c19a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:20:45.568 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:44.877+0000 7f89a0b68700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f899c106140 0x7f899c19a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49290/0 (socket says 172.21.15.97:49290) 2024-03-21T13:20:45.568 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:44.877+0000 7f89a0b68700 1 -- 172.21.15.97:0/947709467 learned_addr learned my addr 172.21.15.97:0/947709467 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:20:45.568 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:44.877+0000 7f89a0b68700 1 -- 172.21.15.97:0/947709467 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f899c19a570 con 0x7f899c106140 2024-03-21T13:20:45.568 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:44.878+0000 7f89a0b68700 1 --2- 172.21.15.97:0/947709467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f899c106140 0x7f899c19a030 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8990011700 tx=0x7f89900046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:20:45.568 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:44.878+0000 7f8999ffb700 1 -- 172.21.15.97:0/947709467 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8990003b20 con 0x7f899c106140 2024-03-21T13:20:45.569 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:44.878+0000 7f8999ffb700 1 -- 172.21.15.97:0/947709467 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f899001a8c0 con 0x7f899c106140 2024-03-21T13:20:45.569 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:44.878+0000 7f8999ffb700 1 -- 172.21.15.97:0/947709467 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f89900249a0 con 0x7f899c106140 2024-03-21T13:20:45.569 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:44.878+0000 7f89a2dcc700 1 -- 172.21.15.97:0/947709467 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f899c19a800 con 0x7f899c106140 2024-03-21T13:20:45.569 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:44.878+0000 7f89a2dcc700 1 -- 172.21.15.97:0/947709467 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f899c19d4f0 con 0x7f899c106140 2024-03-21T13:20:45.569 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:44.878+0000 7f8999ffb700 1 -- 172.21.15.97:0/947709467 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f8990022020 con 0x7f899c106140 2024-03-21T13:20:45.569 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:44.878+0000 7f8999ffb700 1 -- 172.21.15.97:0/947709467 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f89900193c0 con 0x7f899c106140 2024-03-21T13:20:45.569 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:44.879+0000 7f89a2dcc700 1 -- 172.21.15.97:0/947709467 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f899c0637c0 con 0x7f899c106140 2024-03-21T13:20:45.569 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:44.881+0000 7f8999ffb700 1 -- 172.21.15.97:0/947709467 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f8990004300 con 0x7f899c106140 2024-03-21T13:20:45.569 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:44.953+0000 7f89a2dcc700 1 -- 172.21.15.97:0/947709467 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f899c19d8d0 con 0x7f899c106140 2024-03-21T13:20:45.569 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:44.954+0000 7f8999ffb700 1 -- 172.21.15.97:0/947709467 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f899001aa70 con 0x7f899c106140 2024-03-21T13:20:45.569 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:44.955+0000 7f89a2dcc700 1 -- 172.21.15.97:0/947709467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f899c106140 msgr2=0x7f899c19a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:20:45.569 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:44.955+0000 7f89a2dcc700 1 --2- 172.21.15.97:0/947709467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f899c106140 0x7f899c19a030 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8990011700 tx=0x7f89900046a0 comp rx=0 tx=0).stop 2024-03-21T13:20:45.569 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:44.955+0000 7f89a2dcc700 1 -- 172.21.15.97:0/947709467 shutdown_connections 2024-03-21T13:20:45.569 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:44.955+0000 7f89a2dcc700 1 --2- 172.21.15.97:0/947709467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f899c106140 0x7f899c19a030 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:20:45.569 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:44.955+0000 7f89a2dcc700 1 -- 172.21.15.97:0/947709467 >> 172.21.15.97:0/947709467 conn(0x7f899c1016a0 msgr2=0x7f899c102130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:20:45.569 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:44.955+0000 7f89a2dcc700 1 -- 172.21.15.97:0/947709467 shutdown_connections 2024-03-21T13:20:45.570 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:44.955+0000 7f89a2dcc700 1 -- 172.21.15.97:0/947709467 wait complete. 2024-03-21T13:20:45.570 INFO:teuthology.orchestra.run.smithi097.stdout:mgr not available, waiting (1/15)... 2024-03-21T13:20:45.803 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:20:45 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:20:45.626+0000 7f947e89c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-21T13:20:45.803 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:20:45 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:20:45.717+0000 7f947e89c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-21T13:20:46.291 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:20:45 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:20:45.857+0000 7f947e89c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-21T13:20:46.291 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:20:45 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:20:45.950+0000 7f947e89c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-21T13:20:46.291 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:20:46 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:20:46.032+0000 7f947e89c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-21T13:20:46.291 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:20:46 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:20:46.291+0000 7f947e89c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-21T13:20:46.553 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:20:46 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:20:46.398+0000 7f947e89c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-21T13:20:46.553 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:20:46 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:20:46.482+0000 7f947e89c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-21T13:20:47.053 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:20:46 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:20:46.575+0000 7f947e89c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-21T13:20:47.053 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:20:46 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:20:46.750+0000 7f947e89c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-21T13:20:47.933 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:20:47 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:20:47.714+0000 7f947e89c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-21T13:20:47.934 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:20:47 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:20:47.850+0000 7f947e89c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-21T13:20:48.194 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:48 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1633386904' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-21T13:20:48.482 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:20:48 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:20:48.193+0000 7f947e89c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-21T13:20:48.483 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:20:48 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:20:48.283+0000 7f947e89c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-21T13:20:48.483 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:20:48 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:20:48.482+0000 7f947e89c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-21T13:20:48.790 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:20:48 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:20:48.667+0000 7f947e89c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-21T13:20:48.802 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-03-21T13:20:48.802 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout { 2024-03-21T13:20:48.803 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "fsid": "a5dd1f32-e785-11ee-95cd-87774f69a715", 2024-03-21T13:20:48.803 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "health": { 2024-03-21T13:20:48.803 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-21T13:20:48.803 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-21T13:20:48.803 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-21T13:20:48.803 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-21T13:20:48.803 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-21T13:20:48.803 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-21T13:20:48.803 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 0 2024-03-21T13:20:48.803 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-03-21T13:20:48.803 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-21T13:20:48.803 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "a" 2024-03-21T13:20:48.803 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-03-21T13:20:48.803 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-03-21T13:20:48.804 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-21T13:20:48.804 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T13:20:48.804 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-21T13:20:48.804 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-21T13:20:48.804 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-21T13:20:48.804 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-21T13:20:48.804 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T13:20:48.804 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-21T13:20:48.804 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-21T13:20:48.804 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-21T13:20:48.804 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-21T13:20:48.804 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-21T13:20:48.804 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-21T13:20:48.804 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-21T13:20:48.804 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-21T13:20:48.805 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-21T13:20:48.805 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-21T13:20:48.805 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-21T13:20:48.805 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-21T13:20:48.805 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-21T13:20:48.805 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-21T13:20:48.805 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-21T13:20:48.805 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-21T13:20:48.805 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-21T13:20:48.805 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-21T13:20:48.805 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T13:20:48.805 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-21T13:20:48.805 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-21T13:20:48.805 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-21T13:20:48.805 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-21T13:20:48.806 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-21T13:20:48.806 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-21T13:20:48.806 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-21T13:20:48.806 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-21T13:20:48.806 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-21T13:20:48.806 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "restful" 2024-03-21T13:20:48.806 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-03-21T13:20:48.806 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T13:20:48.806 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-21T13:20:48.806 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-21T13:20:48.806 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T13:20:48.806 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "modified": "2024-03-21T13:20:36.782623+0000", 2024-03-21T13:20:48.806 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T13:20:48.806 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-21T13:20:48.807 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-21T13:20:48.807 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout } 2024-03-21T13:20:48.807 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:48.029+0000 7f9578873700 1 Processor -- start 2024-03-21T13:20:48.807 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:48.029+0000 7f9578873700 1 -- start start 2024-03-21T13:20:48.807 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:48.029+0000 7f9578873700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95741065d0 0x7f95741069f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:20:48.807 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:48.029+0000 7f9578873700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9574106fc0 con 0x7f95741065d0 2024-03-21T13:20:48.807 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:48.030+0000 7f957259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95741065d0 0x7f95741069f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:20:48.807 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:48.030+0000 7f957259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95741065d0 0x7f95741069f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49306/0 (socket says 172.21.15.97:49306) 2024-03-21T13:20:48.807 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:48.030+0000 7f957259c700 1 -- 172.21.15.97:0/3670495982 learned_addr learned my addr 172.21.15.97:0/3670495982 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:20:48.807 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:48.030+0000 7f957259c700 1 -- 172.21.15.97:0/3670495982 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9574107140 con 0x7f95741065d0 2024-03-21T13:20:48.807 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:48.030+0000 7f957259c700 1 --2- 172.21.15.97:0/3670495982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95741065d0 0x7f95741069f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f95640098a0 tx=0x7f9564009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f145d90ffc47b83f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:20:48.807 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:48.030+0000 7f957159a700 1 -- 172.21.15.97:0/3670495982 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9564004160 con 0x7f95741065d0 2024-03-21T13:20:48.807 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:48.030+0000 7f957159a700 1 -- 172.21.15.97:0/3670495982 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f9564004300 con 0x7f95741065d0 2024-03-21T13:20:48.807 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:48.030+0000 7f957159a700 1 -- 172.21.15.97:0/3670495982 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9564011a60 con 0x7f95741065d0 2024-03-21T13:20:48.807 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:48.031+0000 7f9578873700 1 -- 172.21.15.97:0/3670495982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95741065d0 msgr2=0x7f95741069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:20:48.808 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:48.031+0000 7f9578873700 1 --2- 172.21.15.97:0/3670495982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95741065d0 0x7f95741069f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f95640098a0 tx=0x7f9564009bb0 comp rx=0 tx=0).stop 2024-03-21T13:20:48.808 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:48.031+0000 7f9578873700 1 -- 172.21.15.97:0/3670495982 shutdown_connections 2024-03-21T13:20:48.808 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:48.031+0000 7f9578873700 1 --2- 172.21.15.97:0/3670495982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95741065d0 0x7f95741069f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:20:48.808 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:48.031+0000 7f9578873700 1 -- 172.21.15.97:0/3670495982 >> 172.21.15.97:0/3670495982 conn(0x7f95740ffd40 msgr2=0x7f95741021a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:20:48.808 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:48.031+0000 7f9578873700 1 -- 172.21.15.97:0/3670495982 shutdown_connections 2024-03-21T13:20:48.808 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:48.031+0000 7f9578873700 1 -- 172.21.15.97:0/3670495982 wait complete. 2024-03-21T13:20:48.808 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:48.032+0000 7f9578873700 1 Processor -- start 2024-03-21T13:20:48.808 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:48.032+0000 7f9578873700 1 -- start start 2024-03-21T13:20:48.808 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:48.032+0000 7f9578873700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95741065d0 0x7f957419a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:20:48.808 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:48.032+0000 7f9578873700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9574109550 con 0x7f95741065d0 2024-03-21T13:20:48.808 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:48.032+0000 7f957259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95741065d0 0x7f957419a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:20:48.808 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:48.032+0000 7f957259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95741065d0 0x7f957419a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49314/0 (socket says 172.21.15.97:49314) 2024-03-21T13:20:48.808 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:48.032+0000 7f957259c700 1 -- 172.21.15.97:0/1633386904 learned_addr learned my addr 172.21.15.97:0/1633386904 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:20:48.808 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:48.032+0000 7f957259c700 1 -- 172.21.15.97:0/1633386904 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f957419a590 con 0x7f95741065d0 2024-03-21T13:20:48.808 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:48.033+0000 7f957259c700 1 --2- 172.21.15.97:0/1633386904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95741065d0 0x7f957419a050 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9564009fb0 tx=0x7f95640046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:20:48.809 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:48.033+0000 7f95637fe700 1 -- 172.21.15.97:0/1633386904 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f956401a430 con 0x7f95741065d0 2024-03-21T13:20:48.809 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:48.033+0000 7f9578873700 1 -- 172.21.15.97:0/1633386904 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f957419a820 con 0x7f95741065d0 2024-03-21T13:20:48.809 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:48.033+0000 7f95637fe700 1 -- 172.21.15.97:0/1633386904 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f956401aa60 con 0x7f95741065d0 2024-03-21T13:20:48.809 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:48.033+0000 7f95637fe700 1 -- 172.21.15.97:0/1633386904 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f95640249b0 con 0x7f95741065d0 2024-03-21T13:20:48.809 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:48.033+0000 7f9578873700 1 -- 172.21.15.97:0/1633386904 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f957419d510 con 0x7f95741065d0 2024-03-21T13:20:48.809 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:48.033+0000 7f95637fe700 1 -- 172.21.15.97:0/1633386904 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f9564022020 con 0x7f95741065d0 2024-03-21T13:20:48.809 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:48.033+0000 7f95637fe700 1 -- 172.21.15.97:0/1633386904 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f9564019710 con 0x7f95741065d0 2024-03-21T13:20:48.809 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:48.034+0000 7f9578873700 1 -- 172.21.15.97:0/1633386904 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95740637c0 con 0x7f95741065d0 2024-03-21T13:20:48.809 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:48.037+0000 7f95637fe700 1 -- 172.21.15.97:0/1633386904 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f956401a5d0 con 0x7f95741065d0 2024-03-21T13:20:48.809 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:48.108+0000 7f9578873700 1 -- 172.21.15.97:0/1633386904 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f957402da30 con 0x7f95741065d0 2024-03-21T13:20:48.809 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:48.109+0000 7f95637fe700 1 -- 172.21.15.97:0/1633386904 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f956401a7b0 con 0x7f95741065d0 2024-03-21T13:20:48.809 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:48.110+0000 7f9578873700 1 -- 172.21.15.97:0/1633386904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95741065d0 msgr2=0x7f957419a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:20:48.809 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:48.110+0000 7f9578873700 1 --2- 172.21.15.97:0/1633386904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95741065d0 0x7f957419a050 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9564009fb0 tx=0x7f95640046a0 comp rx=0 tx=0).stop 2024-03-21T13:20:48.809 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:48.110+0000 7f9578873700 1 -- 172.21.15.97:0/1633386904 shutdown_connections 2024-03-21T13:20:48.810 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:48.110+0000 7f9578873700 1 --2- 172.21.15.97:0/1633386904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95741065d0 0x7f957419a050 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:20:48.810 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:48.110+0000 7f9578873700 1 -- 172.21.15.97:0/1633386904 >> 172.21.15.97:0/1633386904 conn(0x7f95740ffd40 msgr2=0x7f9574100b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:20:48.810 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:48.110+0000 7f9578873700 1 -- 172.21.15.97:0/1633386904 shutdown_connections 2024-03-21T13:20:48.810 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:48.110+0000 7f9578873700 1 -- 172.21.15.97:0/1633386904 wait complete. 2024-03-21T13:20:48.810 INFO:teuthology.orchestra.run.smithi097.stdout:mgr not available, waiting (2/15)... 2024-03-21T13:20:49.053 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:20:48 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:20:48.950+0000 7f947e89c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-21T13:20:49.553 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:20:49 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:20:49.207+0000 7f947e89c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-21T13:20:49.553 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:20:49 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:20:49.443+0000 7f947e89c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-21T13:20:50.553 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:20:50 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:20:50.242+0000 7f947e89c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-21T13:20:51.082 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:20:50 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:20:50.808+0000 7f947e89c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-21T13:20:51.082 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:20:50 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:20:50.914+0000 7f947e89c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-21T13:20:51.392 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:20:51 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:20:51.130+0000 7f947e89c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-21T13:20:51.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:51 smithi097 ceph-mon[30040]: Activating manager daemon y 2024-03-21T13:20:51.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:51 smithi097 ceph-mon[30040]: mgrmap e2: y(active, starting, since 0.00296067s) 2024-03-21T13:20:51.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:51 smithi097 ceph-mon[30040]: from='mgr.14100 172.21.15.97:0/517029336' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-21T13:20:51.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:51 smithi097 ceph-mon[30040]: from='mgr.14100 172.21.15.97:0/517029336' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-21T13:20:51.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:51 smithi097 ceph-mon[30040]: from='mgr.14100 172.21.15.97:0/517029336' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-21T13:20:51.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:51 smithi097 ceph-mon[30040]: from='mgr.14100 172.21.15.97:0/517029336' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T13:20:51.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:51 smithi097 ceph-mon[30040]: from='mgr.14100 172.21.15.97:0/517029336' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-21T13:20:51.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:51 smithi097 ceph-mon[30040]: Manager daemon y is now available 2024-03-21T13:20:51.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:51 smithi097 ceph-mon[30040]: from='mgr.14100 172.21.15.97:0/517029336' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-21T13:20:51.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:51 smithi097 ceph-mon[30040]: from='mgr.14100 172.21.15.97:0/517029336' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-21T13:20:51.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:51 smithi097 ceph-mon[30040]: from='mgr.14100 172.21.15.97:0/517029336' entity='mgr.y' 2024-03-21T13:20:51.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:51 smithi097 ceph-mon[30040]: from='mgr.14100 172.21.15.97:0/517029336' entity='mgr.y' 2024-03-21T13:20:51.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:51 smithi097 ceph-mon[30040]: from='mgr.14100 172.21.15.97:0/517029336' entity='mgr.y' 2024-03-21T13:20:52.126 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-03-21T13:20:52.126 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout { 2024-03-21T13:20:52.127 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "fsid": "a5dd1f32-e785-11ee-95cd-87774f69a715", 2024-03-21T13:20:52.127 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "health": { 2024-03-21T13:20:52.127 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-21T13:20:52.127 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-21T13:20:52.127 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-21T13:20:52.127 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-21T13:20:52.127 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-21T13:20:52.127 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-21T13:20:52.127 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 0 2024-03-21T13:20:52.127 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-03-21T13:20:52.127 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-21T13:20:52.127 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "a" 2024-03-21T13:20:52.127 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-03-21T13:20:52.127 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-03-21T13:20:52.127 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-21T13:20:52.128 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T13:20:52.128 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-21T13:20:52.128 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-21T13:20:52.128 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-21T13:20:52.128 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-21T13:20:52.128 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T13:20:52.128 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-21T13:20:52.128 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-21T13:20:52.128 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-21T13:20:52.136 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-21T13:20:52.136 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-21T13:20:52.136 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-21T13:20:52.136 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-21T13:20:52.136 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-21T13:20:52.136 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-21T13:20:52.136 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-21T13:20:52.136 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-21T13:20:52.136 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-21T13:20:52.137 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-21T13:20:52.137 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-21T13:20:52.137 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-21T13:20:52.137 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-21T13:20:52.137 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-21T13:20:52.137 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-21T13:20:52.137 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T13:20:52.137 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-21T13:20:52.137 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-21T13:20:52.137 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-21T13:20:52.137 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-21T13:20:52.137 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-21T13:20:52.137 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-21T13:20:52.137 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-21T13:20:52.137 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-21T13:20:52.138 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-21T13:20:52.138 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "restful" 2024-03-21T13:20:52.138 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-03-21T13:20:52.138 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T13:20:52.138 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-21T13:20:52.138 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-21T13:20:52.138 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T13:20:52.138 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "modified": "2024-03-21T13:20:36.782623+0000", 2024-03-21T13:20:52.138 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T13:20:52.138 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-21T13:20:52.138 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-21T13:20:52.138 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout } 2024-03-21T13:20:52.138 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:51.407+0000 7efd5727e700 1 Processor -- start 2024-03-21T13:20:52.138 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:51.407+0000 7efd5727e700 1 -- start start 2024-03-21T13:20:52.138 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:51.407+0000 7efd5727e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd50106170 0x7efd50106590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:20:52.139 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:51.407+0000 7efd5727e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd50106b60 con 0x7efd50106170 2024-03-21T13:20:52.139 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:51.407+0000 7efd5501a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd50106170 0x7efd50106590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:20:52.139 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:51.407+0000 7efd5501a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd50106170 0x7efd50106590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45898/0 (socket says 172.21.15.97:45898) 2024-03-21T13:20:52.139 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:51.407+0000 7efd5501a700 1 -- 172.21.15.97:0/1981527228 learned_addr learned my addr 172.21.15.97:0/1981527228 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:20:52.139 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:51.407+0000 7efd5501a700 1 -- 172.21.15.97:0/1981527228 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd50107370 con 0x7efd50106170 2024-03-21T13:20:52.139 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:51.408+0000 7efd5501a700 1 --2- 172.21.15.97:0/1981527228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd50106170 0x7efd50106590 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7efd400098a0 tx=0x7efd40009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdab80c5367b6b66 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:20:52.139 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:51.408+0000 7efd47fff700 1 -- 172.21.15.97:0/1981527228 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd40004160 con 0x7efd50106170 2024-03-21T13:20:52.139 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:51.408+0000 7efd47fff700 1 -- 172.21.15.97:0/1981527228 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7efd40004300 con 0x7efd50106170 2024-03-21T13:20:52.139 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:51.408+0000 7efd47fff700 1 -- 172.21.15.97:0/1981527228 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd40011a60 con 0x7efd50106170 2024-03-21T13:20:52.139 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:51.408+0000 7efd5727e700 1 -- 172.21.15.97:0/1981527228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd50106170 msgr2=0x7efd50106590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:20:52.139 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:51.408+0000 7efd5727e700 1 --2- 172.21.15.97:0/1981527228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd50106170 0x7efd50106590 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7efd400098a0 tx=0x7efd40009bb0 comp rx=0 tx=0).stop 2024-03-21T13:20:52.139 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:51.408+0000 7efd5727e700 1 -- 172.21.15.97:0/1981527228 shutdown_connections 2024-03-21T13:20:52.139 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:51.408+0000 7efd5727e700 1 --2- 172.21.15.97:0/1981527228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd50106170 0x7efd50106590 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:20:52.139 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:51.408+0000 7efd5727e700 1 -- 172.21.15.97:0/1981527228 >> 172.21.15.97:0/1981527228 conn(0x7efd50101730 msgr2=0x7efd50103b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:20:52.139 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:51.408+0000 7efd5727e700 1 -- 172.21.15.97:0/1981527228 shutdown_connections 2024-03-21T13:20:52.140 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:51.408+0000 7efd5727e700 1 -- 172.21.15.97:0/1981527228 wait complete. 2024-03-21T13:20:52.140 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:51.409+0000 7efd5727e700 1 Processor -- start 2024-03-21T13:20:52.140 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:51.409+0000 7efd5727e700 1 -- start start 2024-03-21T13:20:52.140 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:51.409+0000 7efd5727e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd50106170 0x7efd5019a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:20:52.140 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:51.409+0000 7efd5727e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5010a720 con 0x7efd50106170 2024-03-21T13:20:52.140 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:51.409+0000 7efd5501a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd50106170 0x7efd5019a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:20:52.140 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:51.409+0000 7efd5501a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd50106170 0x7efd5019a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45902/0 (socket says 172.21.15.97:45902) 2024-03-21T13:20:52.140 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:51.409+0000 7efd5501a700 1 -- 172.21.15.97:0/610647936 learned_addr learned my addr 172.21.15.97:0/610647936 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:20:52.140 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:51.410+0000 7efd5501a700 1 -- 172.21.15.97:0/610647936 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd5019a570 con 0x7efd50106170 2024-03-21T13:20:52.140 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:51.410+0000 7efd5501a700 1 --2- 172.21.15.97:0/610647936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd50106170 0x7efd5019a030 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7efd40009fb0 tx=0x7efd400046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:20:52.140 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:51.410+0000 7efd467fc700 1 -- 172.21.15.97:0/610647936 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd4001a430 con 0x7efd50106170 2024-03-21T13:20:52.140 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:51.410+0000 7efd5727e700 1 -- 172.21.15.97:0/610647936 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd5019a800 con 0x7efd50106170 2024-03-21T13:20:52.140 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:51.410+0000 7efd467fc700 1 -- 172.21.15.97:0/610647936 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7efd4001aa60 con 0x7efd50106170 2024-03-21T13:20:52.140 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:51.410+0000 7efd467fc700 1 -- 172.21.15.97:0/610647936 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd400249b0 con 0x7efd50106170 2024-03-21T13:20:52.141 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:51.410+0000 7efd5727e700 1 -- 172.21.15.97:0/610647936 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd5007a1d0 con 0x7efd50106170 2024-03-21T13:20:52.141 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:51.411+0000 7efd467fc700 1 -- 172.21.15.97:0/610647936 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 2) v1 ==== 46050+0+0 (secure 0 0 0) 0x7efd40022020 con 0x7efd50106170 2024-03-21T13:20:52.141 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:51.411+0000 7efd5727e700 1 -- 172.21.15.97:0/610647936 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd500637c0 con 0x7efd50106170 2024-03-21T13:20:52.141 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:51.411+0000 7efd467fc700 1 -- 172.21.15.97:0/610647936 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7efd4004d2a0 con 0x7efd50106170 2024-03-21T13:20:52.141 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:51.413+0000 7efd467fc700 1 -- 172.21.15.97:0/610647936 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7efd4000f2e0 con 0x7efd50106170 2024-03-21T13:20:52.141 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:51.482+0000 7efd5727e700 1 -- 172.21.15.97:0/610647936 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7efd5007a610 con 0x7efd50106170 2024-03-21T13:20:52.141 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:51.482+0000 7efd467fc700 1 -- 172.21.15.97:0/610647936 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7efd4002d3f0 con 0x7efd50106170 2024-03-21T13:20:52.141 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:51.483+0000 7efd5727e700 1 -- 172.21.15.97:0/610647936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd50106170 msgr2=0x7efd5019a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:20:52.141 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:51.483+0000 7efd5727e700 1 --2- 172.21.15.97:0/610647936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd50106170 0x7efd5019a030 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7efd40009fb0 tx=0x7efd400046a0 comp rx=0 tx=0).stop 2024-03-21T13:20:52.141 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:51.483+0000 7efd5727e700 1 -- 172.21.15.97:0/610647936 shutdown_connections 2024-03-21T13:20:52.141 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:51.483+0000 7efd5727e700 1 --2- 172.21.15.97:0/610647936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd50106170 0x7efd5019a030 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:20:52.141 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:51.483+0000 7efd5727e700 1 -- 172.21.15.97:0/610647936 >> 172.21.15.97:0/610647936 conn(0x7efd50101730 msgr2=0x7efd50102180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:20:52.141 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:51.483+0000 7efd5727e700 1 -- 172.21.15.97:0/610647936 shutdown_connections 2024-03-21T13:20:52.141 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:51.483+0000 7efd5727e700 1 -- 172.21.15.97:0/610647936 wait complete. 2024-03-21T13:20:52.141 INFO:teuthology.orchestra.run.smithi097.stdout:mgr not available, waiting (3/15)... 2024-03-21T13:20:52.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:52 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/610647936' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-21T13:20:52.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:52 smithi097 ceph-mon[30040]: mgrmap e3: y(active, since 1.00632s) 2024-03-21T13:20:54.490 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:54 smithi097 ceph-mon[30040]: mgrmap e4: y(active, since 2s) 2024-03-21T13:20:55.427 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:55 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2325913321' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-21T13:20:55.431 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-03-21T13:20:55.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout { 2024-03-21T13:20:55.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "fsid": "a5dd1f32-e785-11ee-95cd-87774f69a715", 2024-03-21T13:20:55.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "health": { 2024-03-21T13:20:55.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-21T13:20:55.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-21T13:20:55.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-21T13:20:55.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-21T13:20:55.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-21T13:20:55.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-21T13:20:55.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 0 2024-03-21T13:20:55.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-03-21T13:20:55.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-21T13:20:55.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "a" 2024-03-21T13:20:55.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-03-21T13:20:55.433 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-03-21T13:20:55.433 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-21T13:20:55.433 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T13:20:55.433 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-21T13:20:55.433 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-21T13:20:55.433 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-21T13:20:55.433 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-21T13:20:55.433 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T13:20:55.433 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-21T13:20:55.433 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-21T13:20:55.441 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-21T13:20:55.441 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-21T13:20:55.442 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-21T13:20:55.442 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-21T13:20:55.442 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-21T13:20:55.442 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-21T13:20:55.442 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-21T13:20:55.442 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-21T13:20:55.442 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-21T13:20:55.442 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-21T13:20:55.442 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-21T13:20:55.442 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-21T13:20:55.442 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-21T13:20:55.442 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-21T13:20:55.442 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-21T13:20:55.442 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-21T13:20:55.443 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T13:20:55.443 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-21T13:20:55.443 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-21T13:20:55.443 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-21T13:20:55.443 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-21T13:20:55.443 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-21T13:20:55.443 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-21T13:20:55.443 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-21T13:20:55.443 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-21T13:20:55.443 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-21T13:20:55.443 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "restful" 2024-03-21T13:20:55.443 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-03-21T13:20:55.443 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T13:20:55.443 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-21T13:20:55.444 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-21T13:20:55.444 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T13:20:55.444 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "modified": "2024-03-21T13:20:36.782623+0000", 2024-03-21T13:20:55.444 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T13:20:55.444 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-21T13:20:55.444 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-21T13:20:55.444 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout } 2024-03-21T13:20:55.444 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:54.595+0000 7fb274eed700 1 Processor -- start 2024-03-21T13:20:55.444 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:54.595+0000 7fb274eed700 1 -- start start 2024-03-21T13:20:55.444 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:54.595+0000 7fb274eed700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb270106bb0 0x7fb270108fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:20:55.444 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:54.595+0000 7fb274eed700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb270079710 con 0x7fb270106bb0 2024-03-21T13:20:55.444 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:54.596+0000 7fb26e59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb270106bb0 0x7fb270108fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:20:55.444 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:54.596+0000 7fb26e59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb270106bb0 0x7fb270108fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45906/0 (socket says 172.21.15.97:45906) 2024-03-21T13:20:55.444 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:54.596+0000 7fb26e59c700 1 -- 172.21.15.97:0/3185084019 learned_addr learned my addr 172.21.15.97:0/3185084019 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:20:55.444 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:54.596+0000 7fb26e59c700 1 -- 172.21.15.97:0/3185084019 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2701094e0 con 0x7fb270106bb0 2024-03-21T13:20:55.445 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:54.596+0000 7fb26e59c700 1 --2- 172.21.15.97:0/3185084019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb270106bb0 0x7fb270108fa0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb268009e80 tx=0x7fb26800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fd995c32c89ce57 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:20:55.445 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:54.596+0000 7fb26d59a700 1 -- 172.21.15.97:0/3185084019 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb268017070 con 0x7fb270106bb0 2024-03-21T13:20:55.445 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:54.596+0000 7fb26d59a700 1 -- 172.21.15.97:0/3185084019 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7fb26800b8c0 con 0x7fb270106bb0 2024-03-21T13:20:55.445 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:54.596+0000 7fb26d59a700 1 -- 172.21.15.97:0/3185084019 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb268004470 con 0x7fb270106bb0 2024-03-21T13:20:55.445 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:54.597+0000 7fb274eed700 1 -- 172.21.15.97:0/3185084019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb270106bb0 msgr2=0x7fb270108fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:20:55.445 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:54.597+0000 7fb274eed700 1 --2- 172.21.15.97:0/3185084019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb270106bb0 0x7fb270108fa0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb268009e80 tx=0x7fb26800b330 comp rx=0 tx=0).stop 2024-03-21T13:20:55.445 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:54.597+0000 7fb274eed700 1 -- 172.21.15.97:0/3185084019 shutdown_connections 2024-03-21T13:20:55.445 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:54.597+0000 7fb274eed700 1 --2- 172.21.15.97:0/3185084019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb270106bb0 0x7fb270108fa0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:20:55.445 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:54.597+0000 7fb274eed700 1 -- 172.21.15.97:0/3185084019 >> 172.21.15.97:0/3185084019 conn(0x7fb2701005c0 msgr2=0x7fb270102a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:20:55.445 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:54.597+0000 7fb274eed700 1 -- 172.21.15.97:0/3185084019 shutdown_connections 2024-03-21T13:20:55.445 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:54.597+0000 7fb274eed700 1 -- 172.21.15.97:0/3185084019 wait complete. 2024-03-21T13:20:55.445 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:54.598+0000 7fb274eed700 1 Processor -- start 2024-03-21T13:20:55.445 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:54.598+0000 7fb274eed700 1 -- start start 2024-03-21T13:20:55.445 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:54.598+0000 7fb274eed700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb270106bb0 0x7fb270106430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:20:55.445 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:54.598+0000 7fb274eed700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27010a4b0 con 0x7fb270106bb0 2024-03-21T13:20:55.446 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:54.598+0000 7fb26e59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb270106bb0 0x7fb270106430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:20:55.446 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:54.598+0000 7fb26e59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb270106bb0 0x7fb270106430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45916/0 (socket says 172.21.15.97:45916) 2024-03-21T13:20:55.446 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:54.598+0000 7fb26e59c700 1 -- 172.21.15.97:0/2325913321 learned_addr learned my addr 172.21.15.97:0/2325913321 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:20:55.446 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:54.598+0000 7fb26e59c700 1 -- 172.21.15.97:0/2325913321 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb270106970 con 0x7fb270106bb0 2024-03-21T13:20:55.446 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:54.598+0000 7fb26e59c700 1 --2- 172.21.15.97:0/2325913321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb270106bb0 0x7fb270106430 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb2680072a0 tx=0x7fb268009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:20:55.446 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:54.599+0000 7fb25f7fe700 1 -- 172.21.15.97:0/2325913321 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb268021020 con 0x7fb270106bb0 2024-03-21T13:20:55.446 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:54.599+0000 7fb274eed700 1 -- 172.21.15.97:0/2325913321 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb270104b60 con 0x7fb270106bb0 2024-03-21T13:20:55.446 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:54.599+0000 7fb25f7fe700 1 -- 172.21.15.97:0/2325913321 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7fb268003d40 con 0x7fb270106bb0 2024-03-21T13:20:55.446 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:54.599+0000 7fb25f7fe700 1 -- 172.21.15.97:0/2325913321 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb268017040 con 0x7fb270106bb0 2024-03-21T13:20:55.446 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:54.599+0000 7fb274eed700 1 -- 172.21.15.97:0/2325913321 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2701050a0 con 0x7fb270106bb0 2024-03-21T13:20:55.446 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:54.599+0000 7fb25f7fe700 1 -- 172.21.15.97:0/2325913321 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 4) v1 ==== 46248+0+0 (secure 0 0 0) 0x7fb268024020 con 0x7fb270106bb0 2024-03-21T13:20:55.446 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:54.600+0000 7fb274eed700 1 -- 172.21.15.97:0/2325913321 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2700637c0 con 0x7fb270106bb0 2024-03-21T13:20:55.446 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:54.600+0000 7fb25f7fe700 1 --2- 172.21.15.97:0/2325913321 >> v2:172.21.15.97:6800/2447843320 conn(0x7fb2580390a0 0x7fb25803b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:20:55.446 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:54.600+0000 7fb25f7fe700 1 -- 172.21.15.97:0/2325913321 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb26800f050 con 0x7fb270106bb0 2024-03-21T13:20:55.447 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:54.600+0000 7fb26dd9b700 1 --2- 172.21.15.97:0/2325913321 >> v2:172.21.15.97:6800/2447843320 conn(0x7fb2580390a0 0x7fb25803b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:20:55.447 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:54.600+0000 7fb26dd9b700 1 --2- 172.21.15.97:0/2325913321 >> v2:172.21.15.97:6800/2447843320 conn(0x7fb2580390a0 0x7fb25803b560 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb264006fd0 tx=0x7fb264006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:20:55.447 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:54.605+0000 7fb25f7fe700 1 -- 172.21.15.97:0/2325913321 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb26802b020 con 0x7fb270106bb0 2024-03-21T13:20:55.447 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:54.879+0000 7fb274eed700 1 -- 172.21.15.97:0/2325913321 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fb2701056e0 con 0x7fb270106bb0 2024-03-21T13:20:55.447 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:54.879+0000 7fb25f7fe700 1 -- 172.21.15.97:0/2325913321 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fb268025770 con 0x7fb270106bb0 2024-03-21T13:20:55.447 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:54.881+0000 7fb274eed700 1 -- 172.21.15.97:0/2325913321 >> v2:172.21.15.97:6800/2447843320 conn(0x7fb2580390a0 msgr2=0x7fb25803b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:20:55.447 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:54.882+0000 7fb274eed700 1 --2- 172.21.15.97:0/2325913321 >> v2:172.21.15.97:6800/2447843320 conn(0x7fb2580390a0 0x7fb25803b560 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb264006fd0 tx=0x7fb264006e40 comp rx=0 tx=0).stop 2024-03-21T13:20:55.447 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:54.882+0000 7fb274eed700 1 -- 172.21.15.97:0/2325913321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb270106bb0 msgr2=0x7fb270106430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:20:55.447 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:54.882+0000 7fb274eed700 1 --2- 172.21.15.97:0/2325913321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb270106bb0 0x7fb270106430 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb2680072a0 tx=0x7fb268009610 comp rx=0 tx=0).stop 2024-03-21T13:20:55.447 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:54.882+0000 7fb274eed700 1 -- 172.21.15.97:0/2325913321 shutdown_connections 2024-03-21T13:20:55.447 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:54.882+0000 7fb274eed700 1 --2- 172.21.15.97:0/2325913321 >> v2:172.21.15.97:6800/2447843320 conn(0x7fb2580390a0 0x7fb25803b560 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:20:55.447 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:54.882+0000 7fb274eed700 1 --2- 172.21.15.97:0/2325913321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb270106bb0 0x7fb270106430 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:20:55.447 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:54.882+0000 7fb274eed700 1 -- 172.21.15.97:0/2325913321 >> 172.21.15.97:0/2325913321 conn(0x7fb2701005c0 msgr2=0x7fb270101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:20:55.447 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:54.882+0000 7fb274eed700 1 -- 172.21.15.97:0/2325913321 shutdown_connections 2024-03-21T13:20:55.448 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:54.882+0000 7fb274eed700 1 -- 172.21.15.97:0/2325913321 wait complete. 2024-03-21T13:20:55.448 INFO:teuthology.orchestra.run.smithi097.stdout:mgr is available 2024-03-21T13:20:56.350 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:56 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1221727708' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-21T13:20:56.618 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-03-21T13:20:56.618 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout [global] 2024-03-21T13:20:56.619 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout fsid = a5dd1f32-e785-11ee-95cd-87774f69a715 2024-03-21T13:20:56.619 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-21T13:20:56.619 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-21T13:20:56.619 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-21T13:20:56.619 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-21T13:20:56.619 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-03-21T13:20:56.619 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-21T13:20:56.619 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-21T13:20:56.619 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-03-21T13:20:56.619 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout [osd] 2024-03-21T13:20:56.619 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-21T13:20:56.619 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-21T13:20:56.619 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:55.853+0000 7fb83a6a0700 1 Processor -- start 2024-03-21T13:20:56.619 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:55.853+0000 7fb83a6a0700 1 -- start start 2024-03-21T13:20:56.620 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:55.854+0000 7fb83a6a0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb834106170 0x7fb834106590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:20:56.620 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:55.854+0000 7fb83a6a0700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb834106b60 con 0x7fb834106170 2024-03-21T13:20:56.620 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:55.854+0000 7fb833fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb834106170 0x7fb834106590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:20:56.620 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:55.854+0000 7fb833fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb834106170 0x7fb834106590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45918/0 (socket says 172.21.15.97:45918) 2024-03-21T13:20:56.620 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:55.854+0000 7fb833fff700 1 -- 172.21.15.97:0/320263904 learned_addr learned my addr 172.21.15.97:0/320263904 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:20:56.620 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:55.854+0000 7fb833fff700 1 -- 172.21.15.97:0/320263904 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb834107370 con 0x7fb834106170 2024-03-21T13:20:56.620 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:55.854+0000 7fb833fff700 1 --2- 172.21.15.97:0/320263904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb834106170 0x7fb834106590 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb8240098a0 tx=0x7fb824009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f6d6e5b151d2807 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:20:56.620 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:55.854+0000 7fb832ffd700 1 -- 172.21.15.97:0/320263904 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb824004160 con 0x7fb834106170 2024-03-21T13:20:56.620 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:55.854+0000 7fb832ffd700 1 -- 172.21.15.97:0/320263904 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7fb824004300 con 0x7fb834106170 2024-03-21T13:20:56.620 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:55.854+0000 7fb832ffd700 1 -- 172.21.15.97:0/320263904 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb824011a60 con 0x7fb834106170 2024-03-21T13:20:56.620 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:55.855+0000 7fb83a6a0700 1 -- 172.21.15.97:0/320263904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb834106170 msgr2=0x7fb834106590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:20:56.620 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:55.855+0000 7fb83a6a0700 1 --2- 172.21.15.97:0/320263904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb834106170 0x7fb834106590 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb8240098a0 tx=0x7fb824009bb0 comp rx=0 tx=0).stop 2024-03-21T13:20:56.620 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:55.855+0000 7fb83a6a0700 1 -- 172.21.15.97:0/320263904 shutdown_connections 2024-03-21T13:20:56.620 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:55.855+0000 7fb83a6a0700 1 --2- 172.21.15.97:0/320263904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb834106170 0x7fb834106590 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:20:56.620 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:55.855+0000 7fb83a6a0700 1 -- 172.21.15.97:0/320263904 >> 172.21.15.97:0/320263904 conn(0x7fb834101730 msgr2=0x7fb834103b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:20:56.620 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:55.855+0000 7fb83a6a0700 1 -- 172.21.15.97:0/320263904 shutdown_connections 2024-03-21T13:20:56.621 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:55.855+0000 7fb83a6a0700 1 -- 172.21.15.97:0/320263904 wait complete. 2024-03-21T13:20:56.621 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:55.855+0000 7fb83a6a0700 1 Processor -- start 2024-03-21T13:20:56.621 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:55.856+0000 7fb83a6a0700 1 -- start start 2024-03-21T13:20:56.621 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:55.856+0000 7fb83a6a0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb834106170 0x7fb83419a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:20:56.621 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:55.856+0000 7fb83a6a0700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb83410a720 con 0x7fb834106170 2024-03-21T13:20:56.621 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:55.856+0000 7fb833fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb834106170 0x7fb83419a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:20:56.621 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:55.856+0000 7fb833fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb834106170 0x7fb83419a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45924/0 (socket says 172.21.15.97:45924) 2024-03-21T13:20:56.621 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:55.856+0000 7fb833fff700 1 -- 172.21.15.97:0/1221727708 learned_addr learned my addr 172.21.15.97:0/1221727708 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:20:56.621 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:55.856+0000 7fb833fff700 1 -- 172.21.15.97:0/1221727708 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb83419a570 con 0x7fb834106170 2024-03-21T13:20:56.621 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:55.856+0000 7fb833fff700 1 --2- 172.21.15.97:0/1221727708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb834106170 0x7fb83419a030 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb824009fb0 tx=0x7fb8240046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:20:56.621 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:55.856+0000 7fb8317fa700 1 -- 172.21.15.97:0/1221727708 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb82401a430 con 0x7fb834106170 2024-03-21T13:20:56.621 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:55.857+0000 7fb8317fa700 1 -- 172.21.15.97:0/1221727708 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7fb82401aa60 con 0x7fb834106170 2024-03-21T13:20:56.621 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:55.857+0000 7fb83a6a0700 1 -- 172.21.15.97:0/1221727708 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb83419a800 con 0x7fb834106170 2024-03-21T13:20:56.621 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:55.857+0000 7fb8317fa700 1 -- 172.21.15.97:0/1221727708 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb8240249b0 con 0x7fb834106170 2024-03-21T13:20:56.621 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:55.857+0000 7fb83a6a0700 1 -- 172.21.15.97:0/1221727708 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb83419d4f0 con 0x7fb834106170 2024-03-21T13:20:56.622 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:55.858+0000 7fb8317fa700 1 -- 172.21.15.97:0/1221727708 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 4) v1 ==== 46248+0+0 (secure 0 0 0) 0x7fb824022020 con 0x7fb834106170 2024-03-21T13:20:56.622 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:55.858+0000 7fb8317fa700 1 --2- 172.21.15.97:0/1221727708 >> v2:172.21.15.97:6800/2447843320 conn(0x7fb81c0390f0 0x7fb81c03b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:20:56.622 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:55.858+0000 7fb8317fa700 1 -- 172.21.15.97:0/1221727708 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb82404d750 con 0x7fb834106170 2024-03-21T13:20:56.622 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:55.858+0000 7fb83a6a0700 1 -- 172.21.15.97:0/1221727708 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8340637c0 con 0x7fb834106170 2024-03-21T13:20:56.622 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:55.858+0000 7fb8337fe700 1 --2- 172.21.15.97:0/1221727708 >> v2:172.21.15.97:6800/2447843320 conn(0x7fb81c0390f0 0x7fb81c03b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:20:56.622 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:55.858+0000 7fb8337fe700 1 --2- 172.21.15.97:0/1221727708 >> v2:172.21.15.97:6800/2447843320 conn(0x7fb81c0390f0 0x7fb81c03b5b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb828006fd0 tx=0x7fb828006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:20:56.622 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:55.863+0000 7fb8317fa700 1 -- 172.21.15.97:0/1221727708 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb82400b3e0 con 0x7fb834106170 2024-03-21T13:20:56.622 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:56.054+0000 7fb83a6a0700 1 -- 172.21.15.97:0/1221727708 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fb83419ea50 con 0x7fb834106170 2024-03-21T13:20:56.622 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:56.055+0000 7fb8317fa700 1 -- 172.21.15.97:0/1221727708 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+282 (secure 0 0 0) 0x7fb824018e00 con 0x7fb834106170 2024-03-21T13:20:56.622 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:56.057+0000 7fb83a6a0700 1 -- 172.21.15.97:0/1221727708 >> v2:172.21.15.97:6800/2447843320 conn(0x7fb81c0390f0 msgr2=0x7fb81c03b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:20:56.622 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:56.057+0000 7fb83a6a0700 1 --2- 172.21.15.97:0/1221727708 >> v2:172.21.15.97:6800/2447843320 conn(0x7fb81c0390f0 0x7fb81c03b5b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb828006fd0 tx=0x7fb828006e40 comp rx=0 tx=0).stop 2024-03-21T13:20:56.622 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:56.057+0000 7fb83a6a0700 1 -- 172.21.15.97:0/1221727708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb834106170 msgr2=0x7fb83419a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:20:56.622 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:56.057+0000 7fb83a6a0700 1 --2- 172.21.15.97:0/1221727708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb834106170 0x7fb83419a030 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb824009fb0 tx=0x7fb8240046a0 comp rx=0 tx=0).stop 2024-03-21T13:20:56.623 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:56.058+0000 7fb83a6a0700 1 -- 172.21.15.97:0/1221727708 shutdown_connections 2024-03-21T13:20:56.623 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:56.058+0000 7fb83a6a0700 1 --2- 172.21.15.97:0/1221727708 >> v2:172.21.15.97:6800/2447843320 conn(0x7fb81c0390f0 0x7fb81c03b5b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:20:56.623 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:56.058+0000 7fb83a6a0700 1 --2- 172.21.15.97:0/1221727708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb834106170 0x7fb83419a030 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:20:56.623 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:56.058+0000 7fb83a6a0700 1 -- 172.21.15.97:0/1221727708 >> 172.21.15.97:0/1221727708 conn(0x7fb834101730 msgr2=0x7fb834102180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:20:56.623 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:56.058+0000 7fb83a6a0700 1 -- 172.21.15.97:0/1221727708 shutdown_connections 2024-03-21T13:20:56.623 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:56.058+0000 7fb83a6a0700 1 -- 172.21.15.97:0/1221727708 wait complete. 2024-03-21T13:20:56.623 INFO:teuthology.orchestra.run.smithi097.stdout:Enabling cephadm module... 2024-03-21T13:20:57.698 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:57 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1695182490' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-21T13:20:57.699 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:20:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ignoring --setuser ceph since I am not root 2024-03-21T13:20:57.699 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:20:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ignoring --setgroup ceph since I am not root 2024-03-21T13:20:57.950 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:20:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:20:57.941+0000 7f5e5ce04200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-21T13:20:57.956 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.071+0000 7f4f08719700 1 Processor -- start 2024-03-21T13:20:57.957 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.071+0000 7f4f08719700 1 -- start start 2024-03-21T13:20:57.957 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.071+0000 7f4f08719700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f00107780 0x7f4f00107ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:20:57.957 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.071+0000 7f4f08719700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f00108170 con 0x7f4f00107780 2024-03-21T13:20:57.957 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.071+0000 7f4f064b5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f00107780 0x7f4f00107ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:20:57.957 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.071+0000 7f4f064b5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f00107780 0x7f4f00107ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45938/0 (socket says 172.21.15.97:45938) 2024-03-21T13:20:57.957 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.071+0000 7f4f064b5700 1 -- 172.21.15.97:0/1189001111 learned_addr learned my addr 172.21.15.97:0/1189001111 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:20:57.957 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.071+0000 7f4f064b5700 1 -- 172.21.15.97:0/1189001111 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f001082f0 con 0x7f4f00107780 2024-03-21T13:20:57.957 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.072+0000 7f4f064b5700 1 --2- 172.21.15.97:0/1189001111 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f00107780 0x7f4f00107ba0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4efc0098a0 tx=0x7f4efc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dea22ba7d73b0b6b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:20:57.957 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.072+0000 7f4f054b3700 1 -- 172.21.15.97:0/1189001111 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4efc004160 con 0x7f4f00107780 2024-03-21T13:20:57.957 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.072+0000 7f4f054b3700 1 -- 172.21.15.97:0/1189001111 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f4efc004300 con 0x7f4f00107780 2024-03-21T13:20:57.957 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.072+0000 7f4f054b3700 1 -- 172.21.15.97:0/1189001111 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4efc011a60 con 0x7f4f00107780 2024-03-21T13:20:57.957 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.072+0000 7f4f08719700 1 -- 172.21.15.97:0/1189001111 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f00107780 msgr2=0x7f4f00107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:20:57.957 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.072+0000 7f4f08719700 1 --2- 172.21.15.97:0/1189001111 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f00107780 0x7f4f00107ba0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4efc0098a0 tx=0x7f4efc009bb0 comp rx=0 tx=0).stop 2024-03-21T13:20:57.958 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.072+0000 7f4f08719700 1 -- 172.21.15.97:0/1189001111 shutdown_connections 2024-03-21T13:20:57.958 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.072+0000 7f4f08719700 1 --2- 172.21.15.97:0/1189001111 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f00107780 0x7f4f00107ba0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:20:57.958 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.072+0000 7f4f08719700 1 -- 172.21.15.97:0/1189001111 >> 172.21.15.97:0/1189001111 conn(0x7f4f000ffd60 msgr2=0x7f4f001021c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:20:57.958 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.072+0000 7f4f08719700 1 -- 172.21.15.97:0/1189001111 shutdown_connections 2024-03-21T13:20:57.958 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.073+0000 7f4f08719700 1 -- 172.21.15.97:0/1189001111 wait complete. 2024-03-21T13:20:57.958 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.073+0000 7f4f08719700 1 Processor -- start 2024-03-21T13:20:57.958 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.073+0000 7f4f08719700 1 -- start start 2024-03-21T13:20:57.958 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.073+0000 7f4f08719700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f00107780 0x7f4f00106da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:20:57.958 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.073+0000 7f4f08719700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f00104f90 con 0x7f4f00107780 2024-03-21T13:20:57.958 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.073+0000 7f4f064b5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f00107780 0x7f4f00106da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:20:57.958 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.073+0000 7f4f064b5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f00107780 0x7f4f00106da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45950/0 (socket says 172.21.15.97:45950) 2024-03-21T13:20:57.958 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.073+0000 7f4f064b5700 1 -- 172.21.15.97:0/1695182490 learned_addr learned my addr 172.21.15.97:0/1695182490 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:20:57.958 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.074+0000 7f4f064b5700 1 -- 172.21.15.97:0/1695182490 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f001072e0 con 0x7f4f00107780 2024-03-21T13:20:57.959 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.074+0000 7f4f064b5700 1 --2- 172.21.15.97:0/1695182490 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f00107780 0x7f4f00106da0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4efc011700 tx=0x7f4efc0046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:20:57.959 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.074+0000 7f4ef77fe700 1 -- 172.21.15.97:0/1695182490 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4efc003b20 con 0x7f4f00107780 2024-03-21T13:20:57.959 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.074+0000 7f4f08719700 1 -- 172.21.15.97:0/1695182490 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f001054d0 con 0x7f4f00107780 2024-03-21T13:20:57.959 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.074+0000 7f4ef77fe700 1 -- 172.21.15.97:0/1695182490 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f4efc01a8c0 con 0x7f4f00107780 2024-03-21T13:20:57.959 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.074+0000 7f4ef77fe700 1 -- 172.21.15.97:0/1695182490 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4efc0249a0 con 0x7f4f00107780 2024-03-21T13:20:57.959 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.074+0000 7f4f08719700 1 -- 172.21.15.97:0/1695182490 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f00105a10 con 0x7f4f00107780 2024-03-21T13:20:57.959 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.075+0000 7f4f08719700 1 -- 172.21.15.97:0/1695182490 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f0010a750 con 0x7f4f00107780 2024-03-21T13:20:57.959 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.075+0000 7f4ef77fe700 1 -- 172.21.15.97:0/1695182490 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 4) v1 ==== 46248+0+0 (secure 0 0 0) 0x7f4efc022020 con 0x7f4f00107780 2024-03-21T13:20:57.959 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.075+0000 7f4ef77fe700 1 --2- 172.21.15.97:0/1695182490 >> v2:172.21.15.97:6800/2447843320 conn(0x7f4eec0390a0 0x7f4eec03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:20:57.959 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.075+0000 7f4ef77fe700 1 -- 172.21.15.97:0/1695182490 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4efc04d4f0 con 0x7f4f00107780 2024-03-21T13:20:57.959 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.075+0000 7f4f05cb4700 1 --2- 172.21.15.97:0/1695182490 >> v2:172.21.15.97:6800/2447843320 conn(0x7f4eec0390a0 0x7f4eec03b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:20:57.959 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.075+0000 7f4f05cb4700 1 --2- 172.21.15.97:0/1695182490 >> v2:172.21.15.97:6800/2447843320 conn(0x7f4eec0390a0 0x7f4eec03b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0006fd0 tx=0x7f4ef0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:20:57.959 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.080+0000 7f4ef77fe700 1 -- 172.21.15.97:0/1695182490 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4efc00be50 con 0x7f4f00107780 2024-03-21T13:20:57.959 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.335+0000 7f4f08719700 1 -- 172.21.15.97:0/1695182490 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f4f00106210 con 0x7f4f00107780 2024-03-21T13:20:57.960 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.386+0000 7f4ef77fe700 1 -- 172.21.15.97:0/1695182490 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f4efc00edf0 con 0x7f4f00107780 2024-03-21T13:20:57.960 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.388+0000 7f4ef77fe700 1 -- 172.21.15.97:0/1695182490 <== mon.0 v2:172.21.15.97:3300/0 8 ==== mgrmap(e 5) v1 ==== 46259+0+0 (secure 0 0 0) 0x7f4efc00b3e0 con 0x7f4f00107780 2024-03-21T13:20:57.960 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.389+0000 7f4f08719700 1 -- 172.21.15.97:0/1695182490 >> v2:172.21.15.97:6800/2447843320 conn(0x7f4eec0390a0 msgr2=0x7f4eec03b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:20:57.960 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.389+0000 7f4f08719700 1 --2- 172.21.15.97:0/1695182490 >> v2:172.21.15.97:6800/2447843320 conn(0x7f4eec0390a0 0x7f4eec03b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0006fd0 tx=0x7f4ef0006e40 comp rx=0 tx=0).stop 2024-03-21T13:20:57.960 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.389+0000 7f4f08719700 1 -- 172.21.15.97:0/1695182490 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f00107780 msgr2=0x7f4f00106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:20:57.960 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.389+0000 7f4f08719700 1 --2- 172.21.15.97:0/1695182490 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f00107780 0x7f4f00106da0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4efc011700 tx=0x7f4efc0046a0 comp rx=0 tx=0).stop 2024-03-21T13:20:57.960 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.389+0000 7f4f08719700 1 -- 172.21.15.97:0/1695182490 shutdown_connections 2024-03-21T13:20:57.960 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.389+0000 7f4f08719700 1 --2- 172.21.15.97:0/1695182490 >> v2:172.21.15.97:6800/2447843320 conn(0x7f4eec0390a0 0x7f4eec03b560 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:20:57.960 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.389+0000 7f4f08719700 1 --2- 172.21.15.97:0/1695182490 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f00107780 0x7f4f00106da0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:20:57.960 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.389+0000 7f4f08719700 1 -- 172.21.15.97:0/1695182490 >> 172.21.15.97:0/1695182490 conn(0x7f4f000ffd60 msgr2=0x7f4f00100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:20:57.960 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.390+0000 7f4f08719700 1 -- 172.21.15.97:0/1695182490 shutdown_connections 2024-03-21T13:20:57.960 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:57.390+0000 7f4f08719700 1 -- 172.21.15.97:0/1695182490 wait complete. 2024-03-21T13:20:58.277 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:20:58 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:20:58.040+0000 7f5e5ce04200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-21T13:20:58.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:58 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1695182490' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-21T13:20:58.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:58 smithi097 ceph-mon[30040]: mgrmap e5: y(active, since 6s) 2024-03-21T13:20:58.937 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:20:58 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:20:58.702+0000 7f5e5ce04200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-21T13:20:58.938 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:20:58 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:20:58.794+0000 7f5e5ce04200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-21T13:20:58.938 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:20:58 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:20:58.937+0000 7f5e5ce04200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-21T13:20:59.255 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:20:59 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:20:59.034+0000 7f5e5ce04200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-21T13:20:59.255 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:20:59 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:20:59.116+0000 7f5e5ce04200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-21T13:20:59.263 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout { 2024-03-21T13:20:59.263 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-21T13:20:59.263 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-21T13:20:59.263 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-03-21T13:20:59.263 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-21T13:20:59.263 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout } 2024-03-21T13:20:59.264 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.377+0000 7f8d52c2e700 1 Processor -- start 2024-03-21T13:20:59.264 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.377+0000 7f8d52c2e700 1 -- start start 2024-03-21T13:20:59.264 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.377+0000 7f8d52c2e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d4c106bd0 0x7f8d4c108fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:20:59.264 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.377+0000 7f8d52c2e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4c079fa0 con 0x7f8d4c106bd0 2024-03-21T13:20:59.264 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.377+0000 7f8d509ca700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d4c106bd0 0x7f8d4c108fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:20:59.264 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.377+0000 7f8d509ca700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d4c106bd0 0x7f8d4c108fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45982/0 (socket says 172.21.15.97:45982) 2024-03-21T13:20:59.264 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.377+0000 7f8d509ca700 1 -- 172.21.15.97:0/3355503301 learned_addr learned my addr 172.21.15.97:0/3355503301 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:20:59.264 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.377+0000 7f8d509ca700 1 -- 172.21.15.97:0/3355503301 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d4c109500 con 0x7f8d4c106bd0 2024-03-21T13:20:59.264 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.378+0000 7f8d509ca700 1 --2- 172.21.15.97:0/3355503301 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d4c106bd0 0x7f8d4c108fc0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8d400098a0 tx=0x7f8d40009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b708b8175ebc063b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:20:59.264 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.378+0000 7f8d4b7fe700 1 -- 172.21.15.97:0/3355503301 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d40004160 con 0x7f8d4c106bd0 2024-03-21T13:20:59.264 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.378+0000 7f8d4b7fe700 1 -- 172.21.15.97:0/3355503301 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f8d40004300 con 0x7f8d4c106bd0 2024-03-21T13:20:59.264 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.378+0000 7f8d4b7fe700 1 -- 172.21.15.97:0/3355503301 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d40011a60 con 0x7f8d4c106bd0 2024-03-21T13:20:59.264 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.378+0000 7f8d52c2e700 1 -- 172.21.15.97:0/3355503301 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d4c106bd0 msgr2=0x7f8d4c108fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:20:59.264 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.378+0000 7f8d52c2e700 1 --2- 172.21.15.97:0/3355503301 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d4c106bd0 0x7f8d4c108fc0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8d400098a0 tx=0x7f8d40009bb0 comp rx=0 tx=0).stop 2024-03-21T13:20:59.265 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.378+0000 7f8d52c2e700 1 -- 172.21.15.97:0/3355503301 shutdown_connections 2024-03-21T13:20:59.265 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.378+0000 7f8d52c2e700 1 --2- 172.21.15.97:0/3355503301 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d4c106bd0 0x7f8d4c108fc0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:20:59.265 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.378+0000 7f8d52c2e700 1 -- 172.21.15.97:0/3355503301 >> 172.21.15.97:0/3355503301 conn(0x7f8d4c100600 msgr2=0x7f8d4c102a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:20:59.265 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.378+0000 7f8d52c2e700 1 -- 172.21.15.97:0/3355503301 shutdown_connections 2024-03-21T13:20:59.265 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.379+0000 7f8d52c2e700 1 -- 172.21.15.97:0/3355503301 wait complete. 2024-03-21T13:20:59.265 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.379+0000 7f8d52c2e700 1 Processor -- start 2024-03-21T13:20:59.265 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.379+0000 7f8d52c2e700 1 -- start start 2024-03-21T13:20:59.265 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.379+0000 7f8d52c2e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d4c106bd0 0x7f8d4c199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:20:59.265 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.379+0000 7f8d52c2e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4c10a4d0 con 0x7f8d4c106bd0 2024-03-21T13:20:59.265 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.379+0000 7f8d509ca700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d4c106bd0 0x7f8d4c199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:20:59.265 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.379+0000 7f8d509ca700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d4c106bd0 0x7f8d4c199ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45992/0 (socket says 172.21.15.97:45992) 2024-03-21T13:20:59.265 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.379+0000 7f8d509ca700 1 -- 172.21.15.97:0/1738765889 learned_addr learned my addr 172.21.15.97:0/1738765889 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:20:59.265 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.380+0000 7f8d509ca700 1 -- 172.21.15.97:0/1738765889 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d4c19a530 con 0x7f8d4c106bd0 2024-03-21T13:20:59.265 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.380+0000 7f8d509ca700 1 --2- 172.21.15.97:0/1738765889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d4c106bd0 0x7f8d4c199ff0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8d40011700 tx=0x7f8d40011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:20:59.266 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.380+0000 7f8d49ffb700 1 -- 172.21.15.97:0/1738765889 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d4001a430 con 0x7f8d4c106bd0 2024-03-21T13:20:59.266 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.380+0000 7f8d49ffb700 1 -- 172.21.15.97:0/1738765889 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f8d4001aa60 con 0x7f8d4c106bd0 2024-03-21T13:20:59.266 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.380+0000 7f8d52c2e700 1 -- 172.21.15.97:0/1738765889 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d4c19a7c0 con 0x7f8d4c106bd0 2024-03-21T13:20:59.266 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.380+0000 7f8d49ffb700 1 -- 172.21.15.97:0/1738765889 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d40024970 con 0x7f8d4c106bd0 2024-03-21T13:20:59.266 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.380+0000 7f8d52c2e700 1 -- 172.21.15.97:0/1738765889 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d4c19d4b0 con 0x7f8d4c106bd0 2024-03-21T13:20:59.266 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.381+0000 7f8d377fe700 1 -- 172.21.15.97:0/1738765889 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d4c0637c0 con 0x7f8d4c106bd0 2024-03-21T13:20:59.266 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.381+0000 7f8d49ffb700 1 -- 172.21.15.97:0/1738765889 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 46259+0+0 (secure 0 0 0) 0x7f8d40022020 con 0x7f8d4c106bd0 2024-03-21T13:20:59.266 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.381+0000 7f8d49ffb700 1 --2- 172.21.15.97:0/1738765889 >> v2:172.21.15.97:6800/2447843320 conn(0x7f8d3c039140 0x7f8d3c03b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:20:59.266 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.382+0000 7f8d49ffb700 1 -- 172.21.15.97:0/1738765889 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8d4004d830 con 0x7f8d4c106bd0 2024-03-21T13:20:59.266 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.382+0000 7f8d4bfff700 1 -- 172.21.15.97:0/1738765889 >> v2:172.21.15.97:6800/2447843320 conn(0x7f8d3c039140 msgr2=0x7f8d3c03b600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/2447843320 2024-03-21T13:20:59.266 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.382+0000 7f8d4bfff700 1 --2- 172.21.15.97:0/1738765889 >> v2:172.21.15.97:6800/2447843320 conn(0x7f8d3c039140 0x7f8d3c03b600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T13:20:59.266 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.386+0000 7f8d49ffb700 1 -- 172.21.15.97:0/1738765889 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8d4000b3e0 con 0x7f8d4c106bd0 2024-03-21T13:20:59.266 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.583+0000 7f8d4bfff700 1 -- 172.21.15.97:0/1738765889 >> v2:172.21.15.97:6800/2447843320 conn(0x7f8d3c039140 msgr2=0x7f8d3c03b600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/2447843320 2024-03-21T13:20:59.266 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.583+0000 7f8d4bfff700 1 --2- 172.21.15.97:0/1738765889 >> v2:172.21.15.97:6800/2447843320 conn(0x7f8d3c039140 0x7f8d3c03b600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-21T13:20:59.267 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.648+0000 7f8d377fe700 1 -- 172.21.15.97:0/1738765889 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f8d4c19aca0 con 0x7f8d4c106bd0 2024-03-21T13:20:59.267 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.648+0000 7f8d49ffb700 1 -- 172.21.15.97:0/1738765889 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f8d4c19aca0 con 0x7f8d4c106bd0 2024-03-21T13:20:59.267 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.651+0000 7f8d377fe700 1 -- 172.21.15.97:0/1738765889 >> v2:172.21.15.97:6800/2447843320 conn(0x7f8d3c039140 msgr2=0x7f8d3c03b600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T13:20:59.267 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.651+0000 7f8d377fe700 1 --2- 172.21.15.97:0/1738765889 >> v2:172.21.15.97:6800/2447843320 conn(0x7f8d3c039140 0x7f8d3c03b600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:20:59.267 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.651+0000 7f8d377fe700 1 -- 172.21.15.97:0/1738765889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d4c106bd0 msgr2=0x7f8d4c199ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:20:59.267 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.651+0000 7f8d377fe700 1 --2- 172.21.15.97:0/1738765889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d4c106bd0 0x7f8d4c199ff0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8d40011700 tx=0x7f8d40011b50 comp rx=0 tx=0).stop 2024-03-21T13:20:59.267 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.651+0000 7f8d377fe700 1 -- 172.21.15.97:0/1738765889 shutdown_connections 2024-03-21T13:20:59.267 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.651+0000 7f8d377fe700 1 --2- 172.21.15.97:0/1738765889 >> v2:172.21.15.97:6800/2447843320 conn(0x7f8d3c039140 0x7f8d3c03b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:20:59.267 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.651+0000 7f8d377fe700 1 --2- 172.21.15.97:0/1738765889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d4c106bd0 0x7f8d4c199ff0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:20:59.267 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.651+0000 7f8d377fe700 1 -- 172.21.15.97:0/1738765889 >> 172.21.15.97:0/1738765889 conn(0x7f8d4c100600 msgr2=0x7f8d4c101030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:20:59.267 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.651+0000 7f8d377fe700 1 -- 172.21.15.97:0/1738765889 shutdown_connections 2024-03-21T13:20:59.267 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:58.651+0000 7f8d377fe700 1 -- 172.21.15.97:0/1738765889 wait complete. 2024-03-21T13:20:59.267 INFO:teuthology.orchestra.run.smithi097.stdout:Waiting for the mgr to restart... 2024-03-21T13:20:59.267 INFO:teuthology.orchestra.run.smithi097.stdout:Waiting for mgr epoch 5... 2024-03-21T13:20:59.512 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:20:59 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1738765889' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-21T13:20:59.512 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:20:59 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:20:59.381+0000 7f5e5ce04200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-21T13:20:59.512 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:20:59 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:20:59.494+0000 7f5e5ce04200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-21T13:20:59.861 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:20:59 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:20:59.583+0000 7f5e5ce04200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-21T13:20:59.861 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:20:59 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:20:59.682+0000 7f5e5ce04200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-21T13:21:00.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:20:59 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:20:59.860+0000 7f5e5ce04200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-21T13:21:01.127 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:00 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:21:00.810+0000 7f5e5ce04200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-21T13:21:01.127 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:00 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:21:00.939+0000 7f5e5ce04200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-21T13:21:01.549 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:01 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:21:01.267+0000 7f5e5ce04200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-21T13:21:01.549 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:01 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:21:01.355+0000 7f5e5ce04200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-21T13:21:01.549 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:01 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:21:01.548+0000 7f5e5ce04200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-21T13:21:01.803 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:01 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:21:01.730+0000 7f5e5ce04200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-21T13:21:02.264 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:02 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:21:02.009+0000 7f5e5ce04200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-21T13:21:02.553 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:02 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:21:02.264+0000 7f5e5ce04200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-21T13:21:02.553 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:02 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:21:02.496+0000 7f5e5ce04200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-21T13:21:03.553 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:03 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:21:03.287+0000 7f5e5ce04200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-21T13:21:04.158 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:03 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:21:03.850+0000 7f5e5ce04200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-21T13:21:04.158 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:03 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:21:03.945+0000 7f5e5ce04200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-21T13:21:04.158 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:04 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:21:04.157+0000 7f5e5ce04200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-21T13:21:04.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:04 smithi097 ceph-mon[30040]: Active manager daemon y restarted 2024-03-21T13:21:04.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:04 smithi097 ceph-mon[30040]: Activating manager daemon y 2024-03-21T13:21:04.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:04 smithi097 ceph-mon[30040]: osdmap e2: 0 total, 0 up, 0 in 2024-03-21T13:21:04.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:04 smithi097 ceph-mon[30040]: mgrmap e6: y(active, starting, since 0.00328758s) 2024-03-21T13:21:04.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:04 smithi097 ceph-mon[30040]: from='mgr.14120 172.21.15.97:0/366686223' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T13:21:04.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:04 smithi097 ceph-mon[30040]: from='mgr.14120 172.21.15.97:0/366686223' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-21T13:21:04.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:04 smithi097 ceph-mon[30040]: from='mgr.14120 172.21.15.97:0/366686223' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-21T13:21:04.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:04 smithi097 ceph-mon[30040]: from='mgr.14120 172.21.15.97:0/366686223' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-21T13:21:04.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:04 smithi097 ceph-mon[30040]: from='mgr.14120 172.21.15.97:0/366686223' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-21T13:21:04.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:04 smithi097 ceph-mon[30040]: Manager daemon y is now available 2024-03-21T13:21:04.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:04 smithi097 ceph-mon[30040]: from='mgr.14120 172.21.15.97:0/366686223' entity='mgr.y' 2024-03-21T13:21:04.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:04 smithi097 ceph-mon[30040]: from='mgr.14120 172.21.15.97:0/366686223' entity='mgr.y' 2024-03-21T13:21:04.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:04 smithi097 ceph-mon[30040]: from='mgr.14120 172.21.15.97:0/366686223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:21:04.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:04 smithi097 ceph-mon[30040]: from='mgr.14120 172.21.15.97:0/366686223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:21:05.455 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:05 smithi097 ceph-mon[30040]: Found migration_current of "None". Setting to last migration. 2024-03-21T13:21:05.455 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:05 smithi097 ceph-mon[30040]: from='mgr.14120 172.21.15.97:0/366686223' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-21T13:21:05.455 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:05 smithi097 ceph-mon[30040]: from='mgr.14120 172.21.15.97:0/366686223' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-21T13:21:05.455 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:05 smithi097 ceph-mon[30040]: from='mgr.14120 172.21.15.97:0/366686223' entity='mgr.y' 2024-03-21T13:21:05.455 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:05 smithi097 ceph-mon[30040]: from='mgr.14120 172.21.15.97:0/366686223' entity='mgr.y' 2024-03-21T13:21:05.456 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:05 smithi097 ceph-mon[30040]: mgrmap e7: y(active, since 1.00589s) 2024-03-21T13:21:05.797 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout { 2024-03-21T13:21:05.797 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-21T13:21:05.797 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-21T13:21:05.797 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout } 2024-03-21T13:21:05.797 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:59.719+0000 7f3e1fae3700 1 Processor -- start 2024-03-21T13:21:05.797 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:59.720+0000 7f3e1fae3700 1 -- start start 2024-03-21T13:21:05.797 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:59.720+0000 7f3e1fae3700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e181049c0 0x7f3e18104de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:05.797 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:59.720+0000 7f3e1fae3700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e18079fa0 con 0x7f3e181049c0 2024-03-21T13:21:05.797 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:59.720+0000 7f3e1d87f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e181049c0 0x7f3e18104de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:05.797 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:59.720+0000 7f3e1d87f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e181049c0 0x7f3e18104de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45998/0 (socket says 172.21.15.97:45998) 2024-03-21T13:21:05.798 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:59.720+0000 7f3e1d87f700 1 -- 172.21.15.97:0/3511004143 learned_addr learned my addr 172.21.15.97:0/3511004143 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:21:05.798 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:59.720+0000 7f3e1d87f700 1 -- 172.21.15.97:0/3511004143 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e18105320 con 0x7f3e181049c0 2024-03-21T13:21:05.798 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:59.720+0000 7f3e1d87f700 1 --2- 172.21.15.97:0/3511004143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e181049c0 0x7f3e18104de0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3e140098a0 tx=0x7f3e14009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=456f1f53ef28581a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:05.798 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:59.721+0000 7f3e1c87d700 1 -- 172.21.15.97:0/3511004143 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e14004160 con 0x7f3e181049c0 2024-03-21T13:21:05.798 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:59.721+0000 7f3e1c87d700 1 -- 172.21.15.97:0/3511004143 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f3e14004300 con 0x7f3e181049c0 2024-03-21T13:21:05.798 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:59.721+0000 7f3e1c87d700 1 -- 172.21.15.97:0/3511004143 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e14011a60 con 0x7f3e181049c0 2024-03-21T13:21:05.798 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:59.721+0000 7f3e1fae3700 1 -- 172.21.15.97:0/3511004143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e181049c0 msgr2=0x7f3e18104de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:05.798 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:59.721+0000 7f3e1fae3700 1 --2- 172.21.15.97:0/3511004143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e181049c0 0x7f3e18104de0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3e140098a0 tx=0x7f3e14009bb0 comp rx=0 tx=0).stop 2024-03-21T13:21:05.798 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:59.721+0000 7f3e1fae3700 1 -- 172.21.15.97:0/3511004143 shutdown_connections 2024-03-21T13:21:05.798 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:59.721+0000 7f3e1fae3700 1 --2- 172.21.15.97:0/3511004143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e181049c0 0x7f3e18104de0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:05.798 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:59.721+0000 7f3e1fae3700 1 -- 172.21.15.97:0/3511004143 >> 172.21.15.97:0/3511004143 conn(0x7f3e18100600 msgr2=0x7f3e18102a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:21:05.798 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:59.721+0000 7f3e1fae3700 1 -- 172.21.15.97:0/3511004143 shutdown_connections 2024-03-21T13:21:05.798 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:59.721+0000 7f3e1fae3700 1 -- 172.21.15.97:0/3511004143 wait complete. 2024-03-21T13:21:05.798 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:59.722+0000 7f3e1fae3700 1 Processor -- start 2024-03-21T13:21:05.799 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:59.722+0000 7f3e1fae3700 1 -- start start 2024-03-21T13:21:05.799 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:59.722+0000 7f3e1fae3700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e181049c0 0x7f3e1819c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:05.799 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:59.722+0000 7f3e1fae3700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e18106350 con 0x7f3e181049c0 2024-03-21T13:21:05.799 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:59.722+0000 7f3e1d87f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e181049c0 0x7f3e1819c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:05.799 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:59.722+0000 7f3e1d87f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e181049c0 0x7f3e1819c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46014/0 (socket says 172.21.15.97:46014) 2024-03-21T13:21:05.799 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:59.722+0000 7f3e1d87f700 1 -- 172.21.15.97:0/784973323 learned_addr learned my addr 172.21.15.97:0/784973323 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:21:05.799 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:59.722+0000 7f3e1d87f700 1 -- 172.21.15.97:0/784973323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e1819c720 con 0x7f3e181049c0 2024-03-21T13:21:05.799 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:59.723+0000 7f3e1d87f700 1 --2- 172.21.15.97:0/784973323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e181049c0 0x7f3e1819c1e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3e14009870 tx=0x7f3e14011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:05.799 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:59.723+0000 7f3e0effd700 1 -- 172.21.15.97:0/784973323 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e1401a430 con 0x7f3e181049c0 2024-03-21T13:21:05.799 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:59.723+0000 7f3e0effd700 1 -- 172.21.15.97:0/784973323 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f3e1401aa60 con 0x7f3e181049c0 2024-03-21T13:21:05.799 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:59.723+0000 7f3e1fae3700 1 -- 172.21.15.97:0/784973323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e1819c9b0 con 0x7f3e181049c0 2024-03-21T13:21:05.799 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:59.723+0000 7f3e0effd700 1 -- 172.21.15.97:0/784973323 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e140199a0 con 0x7f3e181049c0 2024-03-21T13:21:05.799 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:59.723+0000 7f3e1fae3700 1 -- 172.21.15.97:0/784973323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e1819f6a0 con 0x7f3e181049c0 2024-03-21T13:21:05.799 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:59.724+0000 7f3e0effd700 1 -- 172.21.15.97:0/784973323 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 46259+0+0 (secure 0 0 0) 0x7f3e14022020 con 0x7f3e181049c0 2024-03-21T13:21:05.799 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:59.724+0000 7f3e0effd700 1 --2- 172.21.15.97:0/784973323 >> v2:172.21.15.97:6800/2447843320 conn(0x7f3e04039140 0x7f3e0403b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:05.800 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:59.724+0000 7f3e0effd700 1 -- 172.21.15.97:0/784973323 --> v2:172.21.15.97:6800/2447843320 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3e0403bcd0 con 0x7f3e04039140 2024-03-21T13:21:05.800 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:59.724+0000 7f3e0effd700 1 -- 172.21.15.97:0/784973323 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3e1404d2d0 con 0x7f3e181049c0 2024-03-21T13:21:05.800 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:59.724+0000 7f3e1d07e700 1 -- 172.21.15.97:0/784973323 >> v2:172.21.15.97:6800/2447843320 conn(0x7f3e04039140 msgr2=0x7f3e0403b600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/2447843320 2024-03-21T13:21:05.800 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:59.724+0000 7f3e1d07e700 1 --2- 172.21.15.97:0/784973323 >> v2:172.21.15.97:6800/2447843320 conn(0x7f3e04039140 0x7f3e0403b600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T13:21:05.800 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:59.925+0000 7f3e1d07e700 1 -- 172.21.15.97:0/784973323 >> v2:172.21.15.97:6800/2447843320 conn(0x7f3e04039140 msgr2=0x7f3e0403b600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/2447843320 2024-03-21T13:21:05.800 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:20:59.925+0000 7f3e1d07e700 1 --2- 172.21.15.97:0/784973323 >> v2:172.21.15.97:6800/2447843320 conn(0x7f3e04039140 0x7f3e0403b600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-21T13:21:05.800 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:00.326+0000 7f3e1d07e700 1 -- 172.21.15.97:0/784973323 >> v2:172.21.15.97:6800/2447843320 conn(0x7f3e04039140 msgr2=0x7f3e0403b600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/2447843320 2024-03-21T13:21:05.800 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:00.326+0000 7f3e1d07e700 1 --2- 172.21.15.97:0/784973323 >> v2:172.21.15.97:6800/2447843320 conn(0x7f3e04039140 0x7f3e0403b600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-21T13:21:05.800 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:01.127+0000 7f3e1d07e700 1 -- 172.21.15.97:0/784973323 >> v2:172.21.15.97:6800/2447843320 conn(0x7f3e04039140 msgr2=0x7f3e0403b600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/2447843320 2024-03-21T13:21:05.800 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:01.127+0000 7f3e1d07e700 1 --2- 172.21.15.97:0/784973323 >> v2:172.21.15.97:6800/2447843320 conn(0x7f3e04039140 0x7f3e0403b600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-21T13:21:05.800 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:02.728+0000 7f3e1d07e700 1 -- 172.21.15.97:0/784973323 >> v2:172.21.15.97:6800/2447843320 conn(0x7f3e04039140 msgr2=0x7f3e0403b600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/2447843320 2024-03-21T13:21:05.800 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:02.728+0000 7f3e1d07e700 1 --2- 172.21.15.97:0/784973323 >> v2:172.21.15.97:6800/2447843320 conn(0x7f3e04039140 0x7f3e0403b600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-21T13:21:05.800 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:04.172+0000 7f3e0effd700 1 -- 172.21.15.97:0/784973323 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mgrmap(e 6) v1 ==== 46061+0+0 (secure 0 0 0) 0x7f3e14019c50 con 0x7f3e181049c0 2024-03-21T13:21:05.800 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:04.172+0000 7f3e0effd700 1 -- 172.21.15.97:0/784973323 >> v2:172.21.15.97:6800/2447843320 conn(0x7f3e04039140 msgr2=0x7f3e0403b600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T13:21:05.800 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:04.172+0000 7f3e0effd700 1 --2- 172.21.15.97:0/784973323 >> v2:172.21.15.97:6800/2447843320 conn(0x7f3e04039140 0x7f3e0403b600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:05.801 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:05.173+0000 7f3e0effd700 1 -- 172.21.15.97:0/784973323 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mgrmap(e 7) v1 ==== 46153+0+0 (secure 0 0 0) 0x7f3e1404f0a0 con 0x7f3e181049c0 2024-03-21T13:21:05.801 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:05.174+0000 7f3e0effd700 1 --2- 172.21.15.97:0/784973323 >> v2:172.21.15.97:6800/4140723216 conn(0x7f3e0403ca10 0x7f3e0403ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:05.801 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:05.174+0000 7f3e0effd700 1 -- 172.21.15.97:0/784973323 --> v2:172.21.15.97:6800/4140723216 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3e1401dec0 con 0x7f3e0403ca10 2024-03-21T13:21:05.801 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:05.174+0000 7f3e1d07e700 1 --2- 172.21.15.97:0/784973323 >> v2:172.21.15.97:6800/4140723216 conn(0x7f3e0403ca10 0x7f3e0403ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:05.801 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:05.175+0000 7f3e1d07e700 1 --2- 172.21.15.97:0/784973323 >> v2:172.21.15.97:6800/4140723216 conn(0x7f3e0403ca10 0x7f3e0403ee00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3e08003a10 tx=0x7f3e080092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:05.801 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:05.175+0000 7f3e0effd700 1 -- 172.21.15.97:0/784973323 <== mgr.14120 v2:172.21.15.97:6800/4140723216 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f3e1401dec0 con 0x7f3e0403ca10 2024-03-21T13:21:05.801 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:05.182+0000 7f3e1fae3700 1 -- 172.21.15.97:0/784973323 --> v2:172.21.15.97:6800/4140723216 -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f3e180637c0 con 0x7f3e0403ca10 2024-03-21T13:21:05.801 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:05.182+0000 7f3e0effd700 1 -- 172.21.15.97:0/784973323 <== mgr.14120 v2:172.21.15.97:6800/4140723216 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f3e180637c0 con 0x7f3e0403ca10 2024-03-21T13:21:05.801 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:05.182+0000 7f3e1fae3700 1 -- 172.21.15.97:0/784973323 >> v2:172.21.15.97:6800/4140723216 conn(0x7f3e0403ca10 msgr2=0x7f3e0403ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:05.801 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:05.182+0000 7f3e1fae3700 1 --2- 172.21.15.97:0/784973323 >> v2:172.21.15.97:6800/4140723216 conn(0x7f3e0403ca10 0x7f3e0403ee00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3e08003a10 tx=0x7f3e080092b0 comp rx=0 tx=0).stop 2024-03-21T13:21:05.801 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:05.182+0000 7f3e1fae3700 1 -- 172.21.15.97:0/784973323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e181049c0 msgr2=0x7f3e1819c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:05.801 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:05.182+0000 7f3e1fae3700 1 --2- 172.21.15.97:0/784973323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e181049c0 0x7f3e1819c1e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3e14009870 tx=0x7f3e14011ae0 comp rx=0 tx=0).stop 2024-03-21T13:21:05.801 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:05.182+0000 7f3e1fae3700 1 -- 172.21.15.97:0/784973323 shutdown_connections 2024-03-21T13:21:05.801 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:05.182+0000 7f3e1fae3700 1 --2- 172.21.15.97:0/784973323 >> v2:172.21.15.97:6800/4140723216 conn(0x7f3e0403ca10 0x7f3e0403ee00 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:05.801 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:05.182+0000 7f3e1fae3700 1 --2- 172.21.15.97:0/784973323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e181049c0 0x7f3e1819c1e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:05.802 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:05.182+0000 7f3e1fae3700 1 --2- 172.21.15.97:0/784973323 >> v2:172.21.15.97:6800/2447843320 conn(0x7f3e04039140 0x7f3e0403b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:05.802 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:05.182+0000 7f3e1fae3700 1 -- 172.21.15.97:0/784973323 >> 172.21.15.97:0/784973323 conn(0x7f3e18100600 msgr2=0x7f3e18192de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:21:05.802 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:05.182+0000 7f3e1fae3700 1 -- 172.21.15.97:0/784973323 shutdown_connections 2024-03-21T13:21:05.802 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:05.182+0000 7f3e1fae3700 1 -- 172.21.15.97:0/784973323 wait complete. 2024-03-21T13:21:05.802 INFO:teuthology.orchestra.run.smithi097.stdout:mgr epoch 5 is available 2024-03-21T13:21:05.802 INFO:teuthology.orchestra.run.smithi097.stdout:Setting orchestrator backend to cephadm... 2024-03-21T13:21:06.455 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:06 smithi097 ceph-mon[30040]: [21/Mar/2024:13:21:05] ENGINE Bus STARTING 2024-03-21T13:21:06.455 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:06 smithi097 ceph-mon[30040]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-21T13:21:06.455 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:06 smithi097 ceph-mon[30040]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-21T13:21:06.455 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:06 smithi097 ceph-mon[30040]: [21/Mar/2024:13:21:05] ENGINE Serving on http://172.21.15.97:8765 2024-03-21T13:21:06.455 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:06 smithi097 ceph-mon[30040]: [21/Mar/2024:13:21:05] ENGINE Serving on https://172.21.15.97:7150 2024-03-21T13:21:06.455 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:06 smithi097 ceph-mon[30040]: [21/Mar/2024:13:21:05] ENGINE Bus STARTED 2024-03-21T13:21:06.455 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:06 smithi097 ceph-mon[30040]: from='mgr.14120 172.21.15.97:0/366686223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:21:07.051 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:06.237+0000 7f4111cc1700 1 Processor -- start 2024-03-21T13:21:07.051 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:06.237+0000 7f4111cc1700 1 -- start start 2024-03-21T13:21:07.051 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:06.237+0000 7f4111cc1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f410c106180 0x7f410c1065a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:07.051 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:06.237+0000 7f4111cc1700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f410c106b70 con 0x7f410c106180 2024-03-21T13:21:07.051 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:06.237+0000 7f410b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f410c106180 0x7f410c1065a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:07.051 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:06.237+0000 7f410b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f410c106180 0x7f410c1065a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52576/0 (socket says 172.21.15.97:52576) 2024-03-21T13:21:07.051 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:06.237+0000 7f410b7fe700 1 -- 172.21.15.97:0/3819429129 learned_addr learned my addr 172.21.15.97:0/3819429129 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:21:07.051 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:06.238+0000 7f410b7fe700 1 -- 172.21.15.97:0/3819429129 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f410c107380 con 0x7f410c106180 2024-03-21T13:21:07.051 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:06.238+0000 7f410b7fe700 1 --2- 172.21.15.97:0/3819429129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f410c106180 0x7f410c1065a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f40fc0098a0 tx=0x7f40fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4245e8f06e696ff3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:07.051 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:06.238+0000 7f410a7fc700 1 -- 172.21.15.97:0/3819429129 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f40fc004160 con 0x7f410c106180 2024-03-21T13:21:07.051 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:06.238+0000 7f410a7fc700 1 -- 172.21.15.97:0/3819429129 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f40fc004300 con 0x7f410c106180 2024-03-21T13:21:07.052 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:06.238+0000 7f410a7fc700 1 -- 172.21.15.97:0/3819429129 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f40fc011a60 con 0x7f410c106180 2024-03-21T13:21:07.052 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:06.238+0000 7f4111cc1700 1 -- 172.21.15.97:0/3819429129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f410c106180 msgr2=0x7f410c1065a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:07.052 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:06.238+0000 7f4111cc1700 1 --2- 172.21.15.97:0/3819429129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f410c106180 0x7f410c1065a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f40fc0098a0 tx=0x7f40fc009bb0 comp rx=0 tx=0).stop 2024-03-21T13:21:07.052 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:06.238+0000 7f4111cc1700 1 -- 172.21.15.97:0/3819429129 shutdown_connections 2024-03-21T13:21:07.052 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:06.238+0000 7f4111cc1700 1 --2- 172.21.15.97:0/3819429129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f410c106180 0x7f410c1065a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:07.052 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:06.238+0000 7f4111cc1700 1 -- 172.21.15.97:0/3819429129 >> 172.21.15.97:0/3819429129 conn(0x7f410c101700 msgr2=0x7f410c103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:21:07.052 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:06.239+0000 7f4111cc1700 1 -- 172.21.15.97:0/3819429129 shutdown_connections 2024-03-21T13:21:07.052 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:06.239+0000 7f4111cc1700 1 -- 172.21.15.97:0/3819429129 wait complete. 2024-03-21T13:21:07.052 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:06.239+0000 7f4111cc1700 1 Processor -- start 2024-03-21T13:21:07.052 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:06.239+0000 7f4111cc1700 1 -- start start 2024-03-21T13:21:07.052 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:06.239+0000 7f4111cc1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f410c106180 0x7f410c19a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:07.052 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:06.239+0000 7f4111cc1700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f410c10a730 con 0x7f410c106180 2024-03-21T13:21:07.052 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:06.239+0000 7f410b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f410c106180 0x7f410c19a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:07.052 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:06.239+0000 7f410b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f410c106180 0x7f410c19a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52588/0 (socket says 172.21.15.97:52588) 2024-03-21T13:21:07.052 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:06.239+0000 7f410b7fe700 1 -- 172.21.15.97:0/708461037 learned_addr learned my addr 172.21.15.97:0/708461037 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:21:07.053 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:06.240+0000 7f410b7fe700 1 -- 172.21.15.97:0/708461037 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f410c19a560 con 0x7f410c106180 2024-03-21T13:21:07.053 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:06.240+0000 7f410b7fe700 1 --2- 172.21.15.97:0/708461037 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f410c106180 0x7f410c19a020 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f40fc011700 tx=0x7f40fc0046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:07.053 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:06.240+0000 7f4108ff9700 1 -- 172.21.15.97:0/708461037 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f40fc003b20 con 0x7f410c106180 2024-03-21T13:21:07.053 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:06.240+0000 7f4111cc1700 1 -- 172.21.15.97:0/708461037 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f410c19a7f0 con 0x7f410c106180 2024-03-21T13:21:07.053 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:06.240+0000 7f4108ff9700 1 -- 172.21.15.97:0/708461037 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f40fc01a8c0 con 0x7f410c106180 2024-03-21T13:21:07.053 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:06.240+0000 7f4108ff9700 1 -- 172.21.15.97:0/708461037 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f40fc0249a0 con 0x7f410c106180 2024-03-21T13:21:07.053 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:06.240+0000 7f4111cc1700 1 -- 172.21.15.97:0/708461037 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f410c19d4e0 con 0x7f410c106180 2024-03-21T13:21:07.053 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:06.241+0000 7f4108ff9700 1 -- 172.21.15.97:0/708461037 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 46259+0+0 (secure 0 0 0) 0x7f40fc022020 con 0x7f410c106180 2024-03-21T13:21:07.053 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:06.241+0000 7f4111cc1700 1 -- 172.21.15.97:0/708461037 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f410c0637c0 con 0x7f410c106180 2024-03-21T13:21:07.053 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:06.241+0000 7f4108ff9700 1 --2- 172.21.15.97:0/708461037 >> v2:172.21.15.97:6800/4140723216 conn(0x7f40f40390f0 0x7f40f403b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:07.053 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:06.241+0000 7f4108ff9700 1 -- 172.21.15.97:0/708461037 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (secure 0 0 0) 0x7f40fc04d790 con 0x7f410c106180 2024-03-21T13:21:07.053 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:06.241+0000 7f410affd700 1 --2- 172.21.15.97:0/708461037 >> v2:172.21.15.97:6800/4140723216 conn(0x7f40f40390f0 0x7f40f403b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:07.053 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:06.242+0000 7f410affd700 1 --2- 172.21.15.97:0/708461037 >> v2:172.21.15.97:6800/4140723216 conn(0x7f40f40390f0 0x7f40f403b5b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4100006fd0 tx=0x7f4100006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:07.053 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:06.246+0000 7f4108ff9700 1 -- 172.21.15.97:0/708461037 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f40fc00b3e0 con 0x7f410c106180 2024-03-21T13:21:07.053 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:06.454+0000 7f4111cc1700 1 -- 172.21.15.97:0/708461037 --> v2:172.21.15.97:6800/4140723216 -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f410c102880 con 0x7f40f40390f0 2024-03-21T13:21:07.054 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:06.463+0000 7f4108ff9700 1 -- 172.21.15.97:0/708461037 <== mgr.14120 v2:172.21.15.97:6800/4140723216 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f410c102880 con 0x7f40f40390f0 2024-03-21T13:21:07.054 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:06.465+0000 7f4111cc1700 1 -- 172.21.15.97:0/708461037 >> v2:172.21.15.97:6800/4140723216 conn(0x7f40f40390f0 msgr2=0x7f40f403b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:07.054 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:06.465+0000 7f4111cc1700 1 --2- 172.21.15.97:0/708461037 >> v2:172.21.15.97:6800/4140723216 conn(0x7f40f40390f0 0x7f40f403b5b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4100006fd0 tx=0x7f4100006e40 comp rx=0 tx=0).stop 2024-03-21T13:21:07.054 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:06.465+0000 7f4111cc1700 1 -- 172.21.15.97:0/708461037 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f410c106180 msgr2=0x7f410c19a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:07.054 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:06.465+0000 7f4111cc1700 1 --2- 172.21.15.97:0/708461037 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f410c106180 0x7f410c19a020 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f40fc011700 tx=0x7f40fc0046a0 comp rx=0 tx=0).stop 2024-03-21T13:21:07.054 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:06.465+0000 7f4111cc1700 1 -- 172.21.15.97:0/708461037 shutdown_connections 2024-03-21T13:21:07.054 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:06.465+0000 7f4111cc1700 1 --2- 172.21.15.97:0/708461037 >> v2:172.21.15.97:6800/4140723216 conn(0x7f40f40390f0 0x7f40f403b5b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:07.054 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:06.465+0000 7f4111cc1700 1 --2- 172.21.15.97:0/708461037 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f410c106180 0x7f410c19a020 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:07.054 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:06.465+0000 7f4111cc1700 1 -- 172.21.15.97:0/708461037 >> 172.21.15.97:0/708461037 conn(0x7f410c101700 msgr2=0x7f410c102170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:21:07.054 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:06.465+0000 7f4111cc1700 1 -- 172.21.15.97:0/708461037 shutdown_connections 2024-03-21T13:21:07.054 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:06.466+0000 7f4111cc1700 1 -- 172.21.15.97:0/708461037 wait complete. 2024-03-21T13:21:07.298 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:07 smithi097 ceph-mon[30040]: mgrmap e8: y(active, since 2s) 2024-03-21T13:21:07.298 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:07 smithi097 ceph-mon[30040]: from='mgr.14120 172.21.15.97:0/366686223' entity='mgr.y' 2024-03-21T13:21:07.298 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:07 smithi097 ceph-mon[30040]: from='mgr.14120 172.21.15.97:0/366686223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:21:08.274 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:08 smithi097 ceph-mon[30040]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:21:08.281 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-21T13:21:08.281 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:07.499+0000 7f28a7cc2700 1 Processor -- start 2024-03-21T13:21:08.281 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:07.499+0000 7f28a7cc2700 1 -- start start 2024-03-21T13:21:08.282 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:07.499+0000 7f28a7cc2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28a0106120 0x7f28a0106540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:08.282 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:07.499+0000 7f28a7cc2700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a0106b10 con 0x7f28a0106120 2024-03-21T13:21:08.282 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:07.499+0000 7f28a5a5e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28a0106120 0x7f28a0106540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:08.282 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:07.499+0000 7f28a5a5e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28a0106120 0x7f28a0106540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52604/0 (socket says 172.21.15.97:52604) 2024-03-21T13:21:08.282 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:07.499+0000 7f28a5a5e700 1 -- 172.21.15.97:0/4106802287 learned_addr learned my addr 172.21.15.97:0/4106802287 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:21:08.282 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:07.500+0000 7f28a5a5e700 1 -- 172.21.15.97:0/4106802287 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a0107320 con 0x7f28a0106120 2024-03-21T13:21:08.282 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:07.500+0000 7f28a5a5e700 1 --2- 172.21.15.97:0/4106802287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28a0106120 0x7f28a0106540 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f28900098a0 tx=0x7f2890009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0e48538123cecd3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:08.282 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:07.500+0000 7f28a4a5c700 1 -- 172.21.15.97:0/4106802287 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2890004160 con 0x7f28a0106120 2024-03-21T13:21:08.282 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:07.500+0000 7f28a4a5c700 1 -- 172.21.15.97:0/4106802287 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f2890004300 con 0x7f28a0106120 2024-03-21T13:21:08.282 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:07.500+0000 7f28a4a5c700 1 -- 172.21.15.97:0/4106802287 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2890011a60 con 0x7f28a0106120 2024-03-21T13:21:08.282 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:07.501+0000 7f28a7cc2700 1 -- 172.21.15.97:0/4106802287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28a0106120 msgr2=0x7f28a0106540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:08.282 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:07.501+0000 7f28a7cc2700 1 --2- 172.21.15.97:0/4106802287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28a0106120 0x7f28a0106540 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f28900098a0 tx=0x7f2890009bb0 comp rx=0 tx=0).stop 2024-03-21T13:21:08.282 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:07.501+0000 7f28a7cc2700 1 -- 172.21.15.97:0/4106802287 shutdown_connections 2024-03-21T13:21:08.282 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:07.501+0000 7f28a7cc2700 1 --2- 172.21.15.97:0/4106802287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28a0106120 0x7f28a0106540 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:08.283 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:07.501+0000 7f28a7cc2700 1 -- 172.21.15.97:0/4106802287 >> 172.21.15.97:0/4106802287 conn(0x7f28a01016a0 msgr2=0x7f28a0103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:21:08.283 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:07.501+0000 7f28a7cc2700 1 -- 172.21.15.97:0/4106802287 shutdown_connections 2024-03-21T13:21:08.283 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:07.501+0000 7f28a7cc2700 1 -- 172.21.15.97:0/4106802287 wait complete. 2024-03-21T13:21:08.283 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:07.501+0000 7f28a7cc2700 1 Processor -- start 2024-03-21T13:21:08.283 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:07.502+0000 7f28a7cc2700 1 -- start start 2024-03-21T13:21:08.283 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:07.502+0000 7f28a7cc2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28a0106120 0x7f28a007b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:08.283 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:07.502+0000 7f28a7cc2700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a010a6d0 con 0x7f28a0106120 2024-03-21T13:21:08.283 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:07.502+0000 7f28a5a5e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28a0106120 0x7f28a007b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:08.283 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:07.502+0000 7f28a5a5e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28a0106120 0x7f28a007b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52618/0 (socket says 172.21.15.97:52618) 2024-03-21T13:21:08.283 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:07.502+0000 7f28a5a5e700 1 -- 172.21.15.97:0/4232039725 learned_addr learned my addr 172.21.15.97:0/4232039725 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:21:08.283 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:07.502+0000 7f28a5a5e700 1 -- 172.21.15.97:0/4232039725 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a007b910 con 0x7f28a0106120 2024-03-21T13:21:08.283 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:07.502+0000 7f28a5a5e700 1 --2- 172.21.15.97:0/4232039725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28a0106120 0x7f28a007b3d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2890011700 tx=0x7f28900046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:08.283 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:07.502+0000 7f2896ffd700 1 -- 172.21.15.97:0/4232039725 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2890003b20 con 0x7f28a0106120 2024-03-21T13:21:08.283 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:07.502+0000 7f2896ffd700 1 -- 172.21.15.97:0/4232039725 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f289001a8c0 con 0x7f28a0106120 2024-03-21T13:21:08.283 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:07.502+0000 7f28a7cc2700 1 -- 172.21.15.97:0/4232039725 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28a0079b00 con 0x7f28a0106120 2024-03-21T13:21:08.283 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:07.502+0000 7f2896ffd700 1 -- 172.21.15.97:0/4232039725 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f28900249a0 con 0x7f28a0106120 2024-03-21T13:21:08.284 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:07.502+0000 7f28a7cc2700 1 -- 172.21.15.97:0/4232039725 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28a007a040 con 0x7f28a0106120 2024-03-21T13:21:08.284 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:07.503+0000 7f2896ffd700 1 -- 172.21.15.97:0/4232039725 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 46259+0+0 (secure 0 0 0) 0x7f2890022020 con 0x7f28a0106120 2024-03-21T13:21:08.284 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:07.503+0000 7f2894ff9700 1 -- 172.21.15.97:0/4232039725 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2888005120 con 0x7f28a0106120 2024-03-21T13:21:08.284 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:07.503+0000 7f2896ffd700 1 --2- 172.21.15.97:0/4232039725 >> v2:172.21.15.97:6800/4140723216 conn(0x7f288c039140 0x7f288c03b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:08.284 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:07.504+0000 7f2896ffd700 1 -- 172.21.15.97:0/4232039725 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (secure 0 0 0) 0x7f289004d670 con 0x7f28a0106120 2024-03-21T13:21:08.284 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:07.504+0000 7f28a525d700 1 --2- 172.21.15.97:0/4232039725 >> v2:172.21.15.97:6800/4140723216 conn(0x7f288c039140 0x7f288c03b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:08.284 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:07.504+0000 7f28a525d700 1 --2- 172.21.15.97:0/4232039725 >> v2:172.21.15.97:6800/4140723216 conn(0x7f288c039140 0x7f288c03b600 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f289c006fd0 tx=0x7f289c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:08.284 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:07.509+0000 7f2896ffd700 1 -- 172.21.15.97:0/4232039725 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f289000b3e0 con 0x7f28a0106120 2024-03-21T13:21:08.284 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:07.712+0000 7f2894ff9700 1 -- 172.21.15.97:0/4232039725 --> v2:172.21.15.97:6800/4140723216 -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f2888000bc0 con 0x7f288c039140 2024-03-21T13:21:08.284 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:07.713+0000 7f2896ffd700 1 -- 172.21.15.97:0/4232039725 <== mgr.14120 v2:172.21.15.97:6800/4140723216 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f2888000bc0 con 0x7f288c039140 2024-03-21T13:21:08.284 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:07.716+0000 7f2894ff9700 1 -- 172.21.15.97:0/4232039725 >> v2:172.21.15.97:6800/4140723216 conn(0x7f288c039140 msgr2=0x7f288c03b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:08.284 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:07.716+0000 7f2894ff9700 1 --2- 172.21.15.97:0/4232039725 >> v2:172.21.15.97:6800/4140723216 conn(0x7f288c039140 0x7f288c03b600 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f289c006fd0 tx=0x7f289c006e40 comp rx=0 tx=0).stop 2024-03-21T13:21:08.284 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:07.716+0000 7f2894ff9700 1 -- 172.21.15.97:0/4232039725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28a0106120 msgr2=0x7f28a007b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:08.284 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:07.716+0000 7f2894ff9700 1 --2- 172.21.15.97:0/4232039725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28a0106120 0x7f28a007b3d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2890011700 tx=0x7f28900046a0 comp rx=0 tx=0).stop 2024-03-21T13:21:08.284 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:07.717+0000 7f2894ff9700 1 -- 172.21.15.97:0/4232039725 shutdown_connections 2024-03-21T13:21:08.285 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:07.717+0000 7f2894ff9700 1 --2- 172.21.15.97:0/4232039725 >> v2:172.21.15.97:6800/4140723216 conn(0x7f288c039140 0x7f288c03b600 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:08.285 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:07.717+0000 7f2894ff9700 1 --2- 172.21.15.97:0/4232039725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28a0106120 0x7f28a007b3d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:08.285 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:07.717+0000 7f2894ff9700 1 -- 172.21.15.97:0/4232039725 >> 172.21.15.97:0/4232039725 conn(0x7f28a01016a0 msgr2=0x7f28a0102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:21:08.285 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:07.717+0000 7f2894ff9700 1 -- 172.21.15.97:0/4232039725 shutdown_connections 2024-03-21T13:21:08.285 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:07.717+0000 7f2894ff9700 1 -- 172.21.15.97:0/4232039725 wait complete. 2024-03-21T13:21:08.285 INFO:teuthology.orchestra.run.smithi097.stdout:Generating ssh key... 2024-03-21T13:21:09.231 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:09 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: Generating public/private rsa key pair. 2024-03-21T13:21:09.231 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:09 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: Your identification has been saved in /tmp/tmpn_cm6mmx/key. 2024-03-21T13:21:09.231 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:09 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: Your public key has been saved in /tmp/tmpn_cm6mmx/key.pub. 2024-03-21T13:21:09.231 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:09 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: The key fingerprint is: 2024-03-21T13:21:09.231 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:09 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: SHA256:LgLchhieztbnwnCx8qgMl2L4ZiEGlJvZgElSjfY8meo ceph-a5dd1f32-e785-11ee-95cd-87774f69a715 2024-03-21T13:21:09.231 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:09 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: The key's randomart image is: 2024-03-21T13:21:09.231 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:09 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: +---[RSA 3072]----+ 2024-03-21T13:21:09.231 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:09 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: |++oo | 2024-03-21T13:21:09.232 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:09 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: |++o . | 2024-03-21T13:21:09.232 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:09 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: |o.*o o | 2024-03-21T13:21:09.232 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:09 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: |oB.=* | 2024-03-21T13:21:09.232 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:09 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: |oo+.=. S | 2024-03-21T13:21:09.232 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:09 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: |=++* . | 2024-03-21T13:21:09.232 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:09 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: |=BXo... . | 2024-03-21T13:21:09.232 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:09 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: |==E+o. . | 2024-03-21T13:21:09.232 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:09 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: |o+. .. | 2024-03-21T13:21:09.232 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:09 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: +----[SHA256]-----+ 2024-03-21T13:21:09.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:09 smithi097 ceph-mon[30040]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:21:09.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:09 smithi097 ceph-mon[30040]: from='mgr.14120 172.21.15.97:0/366686223' entity='mgr.y' 2024-03-21T13:21:09.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:09 smithi097 ceph-mon[30040]: from='mgr.14120 172.21.15.97:0/366686223' entity='mgr.y' 2024-03-21T13:21:09.706 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:08.761+0000 7f482fff7700 1 Processor -- start 2024-03-21T13:21:09.706 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:08.761+0000 7f482fff7700 1 -- start start 2024-03-21T13:21:09.706 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:08.761+0000 7f482fff7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4828106c30 0x7f4828109020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:09.706 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:08.761+0000 7f482fff7700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4828079f60 con 0x7f4828106c30 2024-03-21T13:21:09.706 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:08.761+0000 7f482dd93700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4828106c30 0x7f4828109020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:09.706 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:08.761+0000 7f482dd93700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4828106c30 0x7f4828109020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52626/0 (socket says 172.21.15.97:52626) 2024-03-21T13:21:09.706 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:08.762+0000 7f482dd93700 1 -- 172.21.15.97:0/927548468 learned_addr learned my addr 172.21.15.97:0/927548468 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:21:09.706 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:08.762+0000 7f482dd93700 1 -- 172.21.15.97:0/927548468 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4828109560 con 0x7f4828106c30 2024-03-21T13:21:09.706 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:08.762+0000 7f482dd93700 1 --2- 172.21.15.97:0/927548468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4828106c30 0x7f4828109020 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f48240098a0 tx=0x7f4824009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e646122c6b91c4b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:09.707 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:08.762+0000 7f482cd91700 1 -- 172.21.15.97:0/927548468 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4824004160 con 0x7f4828106c30 2024-03-21T13:21:09.707 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:08.762+0000 7f482cd91700 1 -- 172.21.15.97:0/927548468 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f4824004300 con 0x7f4828106c30 2024-03-21T13:21:09.707 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:08.762+0000 7f482cd91700 1 -- 172.21.15.97:0/927548468 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4824011a60 con 0x7f4828106c30 2024-03-21T13:21:09.707 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:08.763+0000 7f482fff7700 1 -- 172.21.15.97:0/927548468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4828106c30 msgr2=0x7f4828109020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:09.707 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:08.763+0000 7f482fff7700 1 --2- 172.21.15.97:0/927548468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4828106c30 0x7f4828109020 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f48240098a0 tx=0x7f4824009bb0 comp rx=0 tx=0).stop 2024-03-21T13:21:09.707 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:08.763+0000 7f482fff7700 1 -- 172.21.15.97:0/927548468 shutdown_connections 2024-03-21T13:21:09.707 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:08.763+0000 7f482fff7700 1 --2- 172.21.15.97:0/927548468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4828106c30 0x7f4828109020 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:09.707 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:08.763+0000 7f482fff7700 1 -- 172.21.15.97:0/927548468 >> 172.21.15.97:0/927548468 conn(0x7f48281005e0 msgr2=0x7f4828102a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:21:09.707 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:08.763+0000 7f482fff7700 1 -- 172.21.15.97:0/927548468 shutdown_connections 2024-03-21T13:21:09.707 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:08.763+0000 7f482fff7700 1 -- 172.21.15.97:0/927548468 wait complete. 2024-03-21T13:21:09.707 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:08.763+0000 7f482fff7700 1 Processor -- start 2024-03-21T13:21:09.707 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:08.763+0000 7f482fff7700 1 -- start start 2024-03-21T13:21:09.707 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:08.764+0000 7f482fff7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4828106c30 0x7f482819a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:09.707 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:08.764+0000 7f482fff7700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482810a520 con 0x7f4828106c30 2024-03-21T13:21:09.708 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:08.764+0000 7f482dd93700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4828106c30 0x7f482819a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:09.708 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:08.764+0000 7f482dd93700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4828106c30 0x7f482819a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52628/0 (socket says 172.21.15.97:52628) 2024-03-21T13:21:09.708 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:08.764+0000 7f482dd93700 1 -- 172.21.15.97:0/2703776726 learned_addr learned my addr 172.21.15.97:0/2703776726 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:21:09.708 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:08.764+0000 7f482dd93700 1 -- 172.21.15.97:0/2703776726 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f482819a570 con 0x7f4828106c30 2024-03-21T13:21:09.708 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:08.764+0000 7f482dd93700 1 --2- 172.21.15.97:0/2703776726 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4828106c30 0x7f482819a030 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4824011700 tx=0x7f4824011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:09.708 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:08.764+0000 7f481effd700 1 -- 172.21.15.97:0/2703776726 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f482401a430 con 0x7f4828106c30 2024-03-21T13:21:09.708 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:08.764+0000 7f482fff7700 1 -- 172.21.15.97:0/2703776726 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f482819a800 con 0x7f4828106c30 2024-03-21T13:21:09.708 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:08.764+0000 7f481effd700 1 -- 172.21.15.97:0/2703776726 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f482401aa60 con 0x7f4828106c30 2024-03-21T13:21:09.708 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:08.764+0000 7f481effd700 1 -- 172.21.15.97:0/2703776726 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4824024970 con 0x7f4828106c30 2024-03-21T13:21:09.708 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:08.764+0000 7f482fff7700 1 -- 172.21.15.97:0/2703776726 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f482819d4f0 con 0x7f4828106c30 2024-03-21T13:21:09.708 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:08.765+0000 7f481effd700 1 -- 172.21.15.97:0/2703776726 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 46259+0+0 (secure 0 0 0) 0x7f4824022020 con 0x7f4828106c30 2024-03-21T13:21:09.708 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:08.765+0000 7f481effd700 1 --2- 172.21.15.97:0/2703776726 >> v2:172.21.15.97:6800/4140723216 conn(0x7f4814039140 0x7f481403b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:09.708 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:08.765+0000 7f481effd700 1 -- 172.21.15.97:0/2703776726 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (secure 0 0 0) 0x7f482404d680 con 0x7f4828106c30 2024-03-21T13:21:09.708 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:08.765+0000 7f482fff7700 1 -- 172.21.15.97:0/2703776726 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48280637c0 con 0x7f4828106c30 2024-03-21T13:21:09.709 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:08.765+0000 7f482d592700 1 --2- 172.21.15.97:0/2703776726 >> v2:172.21.15.97:6800/4140723216 conn(0x7f4814039140 0x7f481403b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:09.709 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:08.766+0000 7f482d592700 1 --2- 172.21.15.97:0/2703776726 >> v2:172.21.15.97:6800/4140723216 conn(0x7f4814039140 0x7f481403b600 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4818006fd0 tx=0x7f4818006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:09.709 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:08.770+0000 7f481effd700 1 -- 172.21.15.97:0/2703776726 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f482400b3e0 con 0x7f4828106c30 2024-03-21T13:21:09.709 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:08.967+0000 7f482fff7700 1 -- 172.21.15.97:0/2703776726 --> v2:172.21.15.97:6800/4140723216 -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f4828101780 con 0x7f4814039140 2024-03-21T13:21:09.709 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:09.150+0000 7f481effd700 1 -- 172.21.15.97:0/2703776726 <== mgr.14120 v2:172.21.15.97:6800/4140723216 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f4828101780 con 0x7f4814039140 2024-03-21T13:21:09.709 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:09.152+0000 7f482fff7700 1 -- 172.21.15.97:0/2703776726 >> v2:172.21.15.97:6800/4140723216 conn(0x7f4814039140 msgr2=0x7f481403b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:09.709 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:09.152+0000 7f482fff7700 1 --2- 172.21.15.97:0/2703776726 >> v2:172.21.15.97:6800/4140723216 conn(0x7f4814039140 0x7f481403b600 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4818006fd0 tx=0x7f4818006e40 comp rx=0 tx=0).stop 2024-03-21T13:21:09.709 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:09.152+0000 7f482fff7700 1 -- 172.21.15.97:0/2703776726 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4828106c30 msgr2=0x7f482819a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:09.709 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:09.152+0000 7f482fff7700 1 --2- 172.21.15.97:0/2703776726 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4828106c30 0x7f482819a030 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4824011700 tx=0x7f4824011b50 comp rx=0 tx=0).stop 2024-03-21T13:21:09.709 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:09.152+0000 7f482fff7700 1 -- 172.21.15.97:0/2703776726 shutdown_connections 2024-03-21T13:21:09.709 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:09.152+0000 7f482fff7700 1 --2- 172.21.15.97:0/2703776726 >> v2:172.21.15.97:6800/4140723216 conn(0x7f4814039140 0x7f481403b600 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:09.709 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:09.152+0000 7f482fff7700 1 --2- 172.21.15.97:0/2703776726 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4828106c30 0x7f482819a030 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:09.709 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:09.152+0000 7f482fff7700 1 -- 172.21.15.97:0/2703776726 >> 172.21.15.97:0/2703776726 conn(0x7f48281005e0 msgr2=0x7f4828101070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:21:09.709 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:09.152+0000 7f482fff7700 1 -- 172.21.15.97:0/2703776726 shutdown_connections 2024-03-21T13:21:09.710 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:09.152+0000 7f482fff7700 1 -- 172.21.15.97:0/2703776726 wait complete. 2024-03-21T13:21:10.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:10 smithi097 ceph-mon[30040]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:21:10.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:10 smithi097 ceph-mon[30040]: Generating ssh key... 2024-03-21T13:21:10.947 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpdJ82pPY5BRSv0rYleMp18kXUI/Sr76Yajzf6vPUr7BzRIKoeYmkco4Y5BSJz7Lt2JRir17ByRRVoeIEHva39hCqhGQm6EQs3lXctDdw0DtVnO6YWXV7nf6PaHYJwmvySjrgmvTiNP2m9hIRtS62IU5k1mKXZ+dMn7NHxsLQunlI4609oI6AI3gs64ZYroFFQDxdqU4h3fauRBOG8TClZ+gFbG3J8BhVZpsNWtJPJsq5O+0BwHP0cFHq1R64fiZYVstH8U70/8RHupPqCnl7ELdHQkh2ahbNEQx0Rx/kbxqPAz3XRvsSLmQAwqmphS6L7ql7FDwDFrtfneDkDtIQE2hH3UO5IKEKHKyc/F8vxi6wYUJtj8QuwYF22pADvv+3clRz9Lutcav2mzk4J+i9WUFIRrBZO8YxcqP9KqkmXHpO14IYSnHTpnovjZb/epGVziUj2r0FPPIEsgqBjVlVH+sFfia9GlF5pr04GZRi2qLJFcm5f2oF7PIRYsfVxa3E= ceph-a5dd1f32-e785-11ee-95cd-87774f69a715 2024-03-21T13:21:10.948 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:10.116+0000 7f557cbed700 1 Processor -- start 2024-03-21T13:21:10.948 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:10.116+0000 7f557cbed700 1 -- start start 2024-03-21T13:21:10.948 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:10.116+0000 7f557cbed700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5578106bb0 0x7f5578108fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:10.948 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:10.116+0000 7f557cbed700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5578079fa0 con 0x7f5578106bb0 2024-03-21T13:21:10.948 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:10.116+0000 7f557659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5578106bb0 0x7f5578108fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:10.948 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:10.116+0000 7f557659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5578106bb0 0x7f5578108fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52632/0 (socket says 172.21.15.97:52632) 2024-03-21T13:21:10.948 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:10.116+0000 7f557659c700 1 -- 172.21.15.97:0/457076633 learned_addr learned my addr 172.21.15.97:0/457076633 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:21:10.948 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:10.117+0000 7f557659c700 1 -- 172.21.15.97:0/457076633 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55781094e0 con 0x7f5578106bb0 2024-03-21T13:21:10.948 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:10.117+0000 7f557659c700 1 --2- 172.21.15.97:0/457076633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5578106bb0 0x7f5578108fa0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f55680098a0 tx=0x7f5568009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81935a13dbb2d7f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:10.948 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:10.117+0000 7f557559a700 1 -- 172.21.15.97:0/457076633 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5568004160 con 0x7f5578106bb0 2024-03-21T13:21:10.948 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:10.117+0000 7f557559a700 1 -- 172.21.15.97:0/457076633 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f5568004300 con 0x7f5578106bb0 2024-03-21T13:21:10.949 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:10.117+0000 7f557559a700 1 -- 172.21.15.97:0/457076633 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5568011a60 con 0x7f5578106bb0 2024-03-21T13:21:10.949 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:10.117+0000 7f557cbed700 1 -- 172.21.15.97:0/457076633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5578106bb0 msgr2=0x7f5578108fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:10.949 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:10.117+0000 7f557cbed700 1 --2- 172.21.15.97:0/457076633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5578106bb0 0x7f5578108fa0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f55680098a0 tx=0x7f5568009bb0 comp rx=0 tx=0).stop 2024-03-21T13:21:10.949 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:10.118+0000 7f557cbed700 1 -- 172.21.15.97:0/457076633 shutdown_connections 2024-03-21T13:21:10.949 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:10.118+0000 7f557cbed700 1 --2- 172.21.15.97:0/457076633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5578106bb0 0x7f5578108fa0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:10.949 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:10.118+0000 7f557cbed700 1 -- 172.21.15.97:0/457076633 >> 172.21.15.97:0/457076633 conn(0x7f55781005a0 msgr2=0x7f5578102a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:21:10.949 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:10.118+0000 7f557cbed700 1 -- 172.21.15.97:0/457076633 shutdown_connections 2024-03-21T13:21:10.949 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:10.118+0000 7f557cbed700 1 -- 172.21.15.97:0/457076633 wait complete. 2024-03-21T13:21:10.949 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:10.118+0000 7f557cbed700 1 Processor -- start 2024-03-21T13:21:10.949 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:10.118+0000 7f557cbed700 1 -- start start 2024-03-21T13:21:10.949 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:10.118+0000 7f557cbed700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5578106bb0 0x7f5578199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:10.949 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:10.119+0000 7f557cbed700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f557810a4b0 con 0x7f5578106bb0 2024-03-21T13:21:10.950 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:10.119+0000 7f557659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5578106bb0 0x7f5578199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:10.950 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:10.119+0000 7f557659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5578106bb0 0x7f5578199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52634/0 (socket says 172.21.15.97:52634) 2024-03-21T13:21:10.950 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:10.119+0000 7f557659c700 1 -- 172.21.15.97:0/74249928 learned_addr learned my addr 172.21.15.97:0/74249928 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:21:10.950 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:10.119+0000 7f557659c700 1 -- 172.21.15.97:0/74249928 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f557819a510 con 0x7f5578106bb0 2024-03-21T13:21:10.950 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:10.119+0000 7f557659c700 1 --2- 172.21.15.97:0/74249928 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5578106bb0 0x7f5578199fd0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5568009870 tx=0x7f5568011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:10.950 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:10.119+0000 7f55677fe700 1 -- 172.21.15.97:0/74249928 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f556801a430 con 0x7f5578106bb0 2024-03-21T13:21:10.950 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:10.119+0000 7f55677fe700 1 -- 172.21.15.97:0/74249928 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f556801aa60 con 0x7f5578106bb0 2024-03-21T13:21:10.950 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:10.119+0000 7f557cbed700 1 -- 172.21.15.97:0/74249928 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f557819a7a0 con 0x7f5578106bb0 2024-03-21T13:21:10.950 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:10.119+0000 7f55677fe700 1 -- 172.21.15.97:0/74249928 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f55680199a0 con 0x7f5578106bb0 2024-03-21T13:21:10.950 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:10.119+0000 7f557cbed700 1 -- 172.21.15.97:0/74249928 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f557819d490 con 0x7f5578106bb0 2024-03-21T13:21:10.950 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:10.120+0000 7f55677fe700 1 -- 172.21.15.97:0/74249928 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 46259+0+0 (secure 0 0 0) 0x7f5568022020 con 0x7f5578106bb0 2024-03-21T13:21:10.950 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:10.120+0000 7f55677fe700 1 --2- 172.21.15.97:0/74249928 >> v2:172.21.15.97:6800/4140723216 conn(0x7f55600390f0 0x7f556003b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:10.951 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:10.120+0000 7f557cbed700 1 -- 172.21.15.97:0/74249928 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55780637c0 con 0x7f5578106bb0 2024-03-21T13:21:10.951 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:10.120+0000 7f55677fe700 1 -- 172.21.15.97:0/74249928 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (secure 0 0 0) 0x7f556804d3c0 con 0x7f5578106bb0 2024-03-21T13:21:10.951 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:10.120+0000 7f5575d9b700 1 --2- 172.21.15.97:0/74249928 >> v2:172.21.15.97:6800/4140723216 conn(0x7f55600390f0 0x7f556003b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:10.951 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:10.121+0000 7f5575d9b700 1 --2- 172.21.15.97:0/74249928 >> v2:172.21.15.97:6800/4140723216 conn(0x7f55600390f0 0x7f556003b5b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f556c006fd0 tx=0x7f556c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:10.951 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:10.125+0000 7f55677fe700 1 -- 172.21.15.97:0/74249928 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5568029250 con 0x7f5578106bb0 2024-03-21T13:21:10.951 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:10.323+0000 7f557cbed700 1 -- 172.21.15.97:0/74249928 --> v2:172.21.15.97:6800/4140723216 -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f5578101770 con 0x7f55600390f0 2024-03-21T13:21:10.951 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:10.324+0000 7f55677fe700 1 -- 172.21.15.97:0/74249928 <== mgr.14120 v2:172.21.15.97:6800/4140723216 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f5578101770 con 0x7f55600390f0 2024-03-21T13:21:10.951 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:10.326+0000 7f557cbed700 1 -- 172.21.15.97:0/74249928 >> v2:172.21.15.97:6800/4140723216 conn(0x7f55600390f0 msgr2=0x7f556003b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:10.951 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:10.326+0000 7f557cbed700 1 --2- 172.21.15.97:0/74249928 >> v2:172.21.15.97:6800/4140723216 conn(0x7f55600390f0 0x7f556003b5b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f556c006fd0 tx=0x7f556c006e40 comp rx=0 tx=0).stop 2024-03-21T13:21:10.951 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:10.326+0000 7f557cbed700 1 -- 172.21.15.97:0/74249928 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5578106bb0 msgr2=0x7f5578199fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:10.951 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:10.326+0000 7f557cbed700 1 --2- 172.21.15.97:0/74249928 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5578106bb0 0x7f5578199fd0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5568009870 tx=0x7f5568011ae0 comp rx=0 tx=0).stop 2024-03-21T13:21:10.951 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:10.326+0000 7f557cbed700 1 -- 172.21.15.97:0/74249928 shutdown_connections 2024-03-21T13:21:10.952 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:10.326+0000 7f557cbed700 1 --2- 172.21.15.97:0/74249928 >> v2:172.21.15.97:6800/4140723216 conn(0x7f55600390f0 0x7f556003b5b0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:10.952 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:10.326+0000 7f557cbed700 1 --2- 172.21.15.97:0/74249928 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5578106bb0 0x7f5578199fd0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:10.952 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:10.326+0000 7f557cbed700 1 -- 172.21.15.97:0/74249928 >> 172.21.15.97:0/74249928 conn(0x7f55781005a0 msgr2=0x7f5578101060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:21:10.952 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:10.326+0000 7f557cbed700 1 -- 172.21.15.97:0/74249928 shutdown_connections 2024-03-21T13:21:10.952 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:10.326+0000 7f557cbed700 1 -- 172.21.15.97:0/74249928 wait complete. 2024-03-21T13:21:10.952 INFO:teuthology.orchestra.run.smithi097.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-21T13:21:10.952 INFO:teuthology.orchestra.run.smithi097.stdout:Adding key to root@localhost authorized_keys... 2024-03-21T13:21:10.952 INFO:teuthology.orchestra.run.smithi097.stdout:Adding host smithi097... 2024-03-21T13:21:11.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:11 smithi097 ceph-mon[30040]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:21:12.770 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:12 smithi097 ceph-mon[30040]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi097", "addr": "172.21.15.97", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:21:12.770 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:12 smithi097 ceph-mon[30040]: Deploying cephadm binary to smithi097 2024-03-21T13:21:16.461 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout Added host 'smithi097' with addr '172.21.15.97' 2024-03-21T13:21:16.461 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:11.386+0000 7f2cef9bd700 1 Processor -- start 2024-03-21T13:21:16.461 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:11.387+0000 7f2cef9bd700 1 -- start start 2024-03-21T13:21:16.462 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:11.387+0000 7f2cef9bd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ce8105550 0x7f2ce8105970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:16.462 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:11.387+0000 7f2cef9bd700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce8105f40 con 0x7f2ce8105550 2024-03-21T13:21:16.462 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:11.387+0000 7f2ced759700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ce8105550 0x7f2ce8105970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:16.462 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:11.387+0000 7f2ced759700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ce8105550 0x7f2ce8105970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43224/0 (socket says 172.21.15.97:43224) 2024-03-21T13:21:16.462 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:11.387+0000 7f2ced759700 1 -- 172.21.15.97:0/919790063 learned_addr learned my addr 172.21.15.97:0/919790063 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:21:16.462 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:11.387+0000 7f2ced759700 1 -- 172.21.15.97:0/919790063 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce8106270 con 0x7f2ce8105550 2024-03-21T13:21:16.462 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:11.387+0000 7f2ced759700 1 --2- 172.21.15.97:0/919790063 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ce8105550 0x7f2ce8105970 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2ce40098a0 tx=0x7f2ce4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a99d09dbe37328c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:16.462 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:11.388+0000 7f2cdffff700 1 -- 172.21.15.97:0/919790063 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ce4004160 con 0x7f2ce8105550 2024-03-21T13:21:16.462 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:11.388+0000 7f2cdffff700 1 -- 172.21.15.97:0/919790063 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f2ce4004300 con 0x7f2ce8105550 2024-03-21T13:21:16.462 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:11.388+0000 7f2cdffff700 1 -- 172.21.15.97:0/919790063 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ce4011a60 con 0x7f2ce8105550 2024-03-21T13:21:16.462 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:11.388+0000 7f2cef9bd700 1 -- 172.21.15.97:0/919790063 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ce8105550 msgr2=0x7f2ce8105970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:16.462 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:11.388+0000 7f2cef9bd700 1 --2- 172.21.15.97:0/919790063 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ce8105550 0x7f2ce8105970 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2ce40098a0 tx=0x7f2ce4009bb0 comp rx=0 tx=0).stop 2024-03-21T13:21:16.462 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:11.388+0000 7f2cef9bd700 1 -- 172.21.15.97:0/919790063 shutdown_connections 2024-03-21T13:21:16.463 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:11.388+0000 7f2cef9bd700 1 --2- 172.21.15.97:0/919790063 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ce8105550 0x7f2ce8105970 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:16.463 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:11.388+0000 7f2cef9bd700 1 -- 172.21.15.97:0/919790063 >> 172.21.15.97:0/919790063 conn(0x7f2ce80ffd10 msgr2=0x7f2ce8102190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:21:16.463 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:11.388+0000 7f2cef9bd700 1 -- 172.21.15.97:0/919790063 shutdown_connections 2024-03-21T13:21:16.463 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:11.388+0000 7f2cef9bd700 1 -- 172.21.15.97:0/919790063 wait complete. 2024-03-21T13:21:16.463 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:11.389+0000 7f2cef9bd700 1 Processor -- start 2024-03-21T13:21:16.463 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:11.389+0000 7f2cef9bd700 1 -- start start 2024-03-21T13:21:16.463 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:11.389+0000 7f2cef9bd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ce8105550 0x7f2ce819a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:16.463 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:11.389+0000 7f2cef9bd700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce8109620 con 0x7f2ce8105550 2024-03-21T13:21:16.463 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:11.389+0000 7f2ced759700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ce8105550 0x7f2ce819a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:16.463 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:11.389+0000 7f2ced759700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ce8105550 0x7f2ce819a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43226/0 (socket says 172.21.15.97:43226) 2024-03-21T13:21:16.463 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:11.389+0000 7f2ced759700 1 -- 172.21.15.97:0/272469083 learned_addr learned my addr 172.21.15.97:0/272469083 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:21:16.463 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:11.389+0000 7f2ced759700 1 -- 172.21.15.97:0/272469083 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce819a5a0 con 0x7f2ce8105550 2024-03-21T13:21:16.463 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:11.390+0000 7f2ced759700 1 --2- 172.21.15.97:0/272469083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ce8105550 0x7f2ce819a060 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4009fb0 tx=0x7f2ce40046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:16.463 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:11.390+0000 7f2cde7fc700 1 -- 172.21.15.97:0/272469083 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ce401a430 con 0x7f2ce8105550 2024-03-21T13:21:16.463 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:11.390+0000 7f2cde7fc700 1 -- 172.21.15.97:0/272469083 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f2ce401aa60 con 0x7f2ce8105550 2024-03-21T13:21:16.464 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:11.390+0000 7f2cef9bd700 1 -- 172.21.15.97:0/272469083 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ce819a830 con 0x7f2ce8105550 2024-03-21T13:21:16.464 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:11.390+0000 7f2cde7fc700 1 -- 172.21.15.97:0/272469083 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ce40249b0 con 0x7f2ce8105550 2024-03-21T13:21:16.464 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:11.390+0000 7f2cef9bd700 1 -- 172.21.15.97:0/272469083 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ce819d520 con 0x7f2ce8105550 2024-03-21T13:21:16.464 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:11.391+0000 7f2cde7fc700 1 -- 172.21.15.97:0/272469083 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 46259+0+0 (secure 0 0 0) 0x7f2ce4022020 con 0x7f2ce8105550 2024-03-21T13:21:16.464 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:11.391+0000 7f2cef9bd700 1 -- 172.21.15.97:0/272469083 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ce80637c0 con 0x7f2ce8105550 2024-03-21T13:21:16.464 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:11.391+0000 7f2cde7fc700 1 --2- 172.21.15.97:0/272469083 >> v2:172.21.15.97:6800/4140723216 conn(0x7f2cd4039140 0x7f2cd403b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:16.464 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:11.391+0000 7f2cde7fc700 1 -- 172.21.15.97:0/272469083 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (secure 0 0 0) 0x7f2ce404d800 con 0x7f2ce8105550 2024-03-21T13:21:16.464 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:11.391+0000 7f2cecf58700 1 --2- 172.21.15.97:0/272469083 >> v2:172.21.15.97:6800/4140723216 conn(0x7f2cd4039140 0x7f2cd403b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:16.464 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:11.391+0000 7f2cecf58700 1 --2- 172.21.15.97:0/272469083 >> v2:172.21.15.97:6800/4140723216 conn(0x7f2cd4039140 0x7f2cd403b600 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8006fd0 tx=0x7f2cd8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:16.464 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:11.396+0000 7f2cde7fc700 1 -- 172.21.15.97:0/272469083 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2ce4019e10 con 0x7f2ce8105550 2024-03-21T13:21:16.464 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:11.595+0000 7f2cef9bd700 1 -- 172.21.15.97:0/272469083 --> v2:172.21.15.97:6800/4140723216 -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi097", "addr": "172.21.15.97", "target": ["mon-mgr", ""]}) v1 -- 0x7f2ce802db70 con 0x7f2cd4039140 2024-03-21T13:21:16.464 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:15.856+0000 7f2cde7fc700 1 -- 172.21.15.97:0/272469083 <== mgr.14120 v2:172.21.15.97:6800/4140723216 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f2ce802db70 con 0x7f2cd4039140 2024-03-21T13:21:16.465 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:15.858+0000 7f2cef9bd700 1 -- 172.21.15.97:0/272469083 >> v2:172.21.15.97:6800/4140723216 conn(0x7f2cd4039140 msgr2=0x7f2cd403b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:16.465 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:15.858+0000 7f2cef9bd700 1 --2- 172.21.15.97:0/272469083 >> v2:172.21.15.97:6800/4140723216 conn(0x7f2cd4039140 0x7f2cd403b600 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8006fd0 tx=0x7f2cd8006e40 comp rx=0 tx=0).stop 2024-03-21T13:21:16.465 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:15.858+0000 7f2cef9bd700 1 -- 172.21.15.97:0/272469083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ce8105550 msgr2=0x7f2ce819a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:16.465 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:15.858+0000 7f2cef9bd700 1 --2- 172.21.15.97:0/272469083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ce8105550 0x7f2ce819a060 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4009fb0 tx=0x7f2ce40046a0 comp rx=0 tx=0).stop 2024-03-21T13:21:16.465 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:15.859+0000 7f2cef9bd700 1 -- 172.21.15.97:0/272469083 shutdown_connections 2024-03-21T13:21:16.465 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:15.859+0000 7f2cef9bd700 1 --2- 172.21.15.97:0/272469083 >> v2:172.21.15.97:6800/4140723216 conn(0x7f2cd4039140 0x7f2cd403b600 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:16.465 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:15.859+0000 7f2cef9bd700 1 --2- 172.21.15.97:0/272469083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ce8105550 0x7f2ce819a060 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:16.465 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:15.859+0000 7f2cef9bd700 1 -- 172.21.15.97:0/272469083 >> 172.21.15.97:0/272469083 conn(0x7f2ce80ffd10 msgr2=0x7f2ce8100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:21:16.465 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:15.859+0000 7f2cef9bd700 1 -- 172.21.15.97:0/272469083 shutdown_connections 2024-03-21T13:21:16.465 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:15.859+0000 7f2cef9bd700 1 -- 172.21.15.97:0/272469083 wait complete. 2024-03-21T13:21:16.465 INFO:teuthology.orchestra.run.smithi097.stdout:Deploying unmanaged mon service... 2024-03-21T13:21:17.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:16 smithi097 ceph-mon[30040]: from='mgr.14120 172.21.15.97:0/366686223' entity='mgr.y' 2024-03-21T13:21:17.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:16 smithi097 ceph-mon[30040]: Added host smithi097 2024-03-21T13:21:17.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:16 smithi097 ceph-mon[30040]: from='mgr.14120 172.21.15.97:0/366686223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:21:17.703 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-21T13:21:17.703 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:16.945+0000 7fef83deb700 1 Processor -- start 2024-03-21T13:21:17.703 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:16.946+0000 7fef83deb700 1 -- start start 2024-03-21T13:21:17.703 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:16.946+0000 7fef83deb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef7c105090 0x7fef7c1054b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:17.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:16.946+0000 7fef83deb700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c105a80 con 0x7fef7c105090 2024-03-21T13:21:17.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:16.946+0000 7fef81b87700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef7c105090 0x7fef7c1054b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:17.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:16.946+0000 7fef81b87700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef7c105090 0x7fef7c1054b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43240/0 (socket says 172.21.15.97:43240) 2024-03-21T13:21:17.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:16.946+0000 7fef81b87700 1 -- 172.21.15.97:0/347928064 learned_addr learned my addr 172.21.15.97:0/347928064 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:21:17.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:16.946+0000 7fef81b87700 1 -- 172.21.15.97:0/347928064 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef7c106290 con 0x7fef7c105090 2024-03-21T13:21:17.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:16.946+0000 7fef81b87700 1 --2- 172.21.15.97:0/347928064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef7c105090 0x7fef7c1054b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fef780098a0 tx=0x7fef78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b8b34e4f250402a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:17.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:16.947+0000 7fef80b85700 1 -- 172.21.15.97:0/347928064 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef78004160 con 0x7fef7c105090 2024-03-21T13:21:17.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:16.947+0000 7fef80b85700 1 -- 172.21.15.97:0/347928064 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7fef78004300 con 0x7fef7c105090 2024-03-21T13:21:17.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:16.947+0000 7fef80b85700 1 -- 172.21.15.97:0/347928064 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef78011a60 con 0x7fef7c105090 2024-03-21T13:21:17.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:16.947+0000 7fef83deb700 1 -- 172.21.15.97:0/347928064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef7c105090 msgr2=0x7fef7c1054b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:17.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:16.947+0000 7fef83deb700 1 --2- 172.21.15.97:0/347928064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef7c105090 0x7fef7c1054b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fef780098a0 tx=0x7fef78009bb0 comp rx=0 tx=0).stop 2024-03-21T13:21:17.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:16.947+0000 7fef83deb700 1 -- 172.21.15.97:0/347928064 shutdown_connections 2024-03-21T13:21:17.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:16.947+0000 7fef83deb700 1 --2- 172.21.15.97:0/347928064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef7c105090 0x7fef7c1054b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:17.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:16.947+0000 7fef83deb700 1 -- 172.21.15.97:0/347928064 >> 172.21.15.97:0/347928064 conn(0x7fef7c100630 msgr2=0x7fef7c102a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:21:17.705 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:16.947+0000 7fef83deb700 1 -- 172.21.15.97:0/347928064 shutdown_connections 2024-03-21T13:21:17.705 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:16.947+0000 7fef83deb700 1 -- 172.21.15.97:0/347928064 wait complete. 2024-03-21T13:21:17.705 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:16.948+0000 7fef83deb700 1 Processor -- start 2024-03-21T13:21:17.705 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:16.948+0000 7fef83deb700 1 -- start start 2024-03-21T13:21:17.705 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:16.948+0000 7fef83deb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef7c105090 0x7fef7c079b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:17.705 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:16.948+0000 7fef83deb700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c109640 con 0x7fef7c105090 2024-03-21T13:21:17.705 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:16.948+0000 7fef81b87700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef7c105090 0x7fef7c079b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:17.705 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:16.948+0000 7fef81b87700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef7c105090 0x7fef7c079b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43244/0 (socket says 172.21.15.97:43244) 2024-03-21T13:21:17.705 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:16.948+0000 7fef81b87700 1 -- 172.21.15.97:0/2932994960 learned_addr learned my addr 172.21.15.97:0/2932994960 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:21:17.705 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:16.948+0000 7fef81b87700 1 -- 172.21.15.97:0/2932994960 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef7c19cd60 con 0x7fef7c105090 2024-03-21T13:21:17.705 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:16.948+0000 7fef81b87700 1 --2- 172.21.15.97:0/2932994960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef7c105090 0x7fef7c079b60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fef78011700 tx=0x7fef780046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:17.705 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:16.949+0000 7fef72ffd700 1 -- 172.21.15.97:0/2932994960 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef78003b20 con 0x7fef7c105090 2024-03-21T13:21:17.705 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:16.949+0000 7fef72ffd700 1 -- 172.21.15.97:0/2932994960 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7fef7801a8c0 con 0x7fef7c105090 2024-03-21T13:21:17.705 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:16.949+0000 7fef72ffd700 1 -- 172.21.15.97:0/2932994960 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef780249a0 con 0x7fef7c105090 2024-03-21T13:21:17.705 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:16.949+0000 7fef83deb700 1 -- 172.21.15.97:0/2932994960 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef7c19cf30 con 0x7fef7c105090 2024-03-21T13:21:17.705 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:16.949+0000 7fef83deb700 1 -- 172.21.15.97:0/2932994960 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef7c19d350 con 0x7fef7c105090 2024-03-21T13:21:17.706 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:16.950+0000 7fef72ffd700 1 -- 172.21.15.97:0/2932994960 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 46259+0+0 (secure 0 0 0) 0x7fef78022020 con 0x7fef7c105090 2024-03-21T13:21:17.706 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:16.950+0000 7fef83deb700 1 -- 172.21.15.97:0/2932994960 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef7c0637c0 con 0x7fef7c105090 2024-03-21T13:21:17.706 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:16.950+0000 7fef72ffd700 1 --2- 172.21.15.97:0/2932994960 >> v2:172.21.15.97:6800/4140723216 conn(0x7fef68039140 0x7fef6803b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:17.706 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:16.950+0000 7fef72ffd700 1 -- 172.21.15.97:0/2932994960 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (secure 0 0 0) 0x7fef7804d160 con 0x7fef7c105090 2024-03-21T13:21:17.706 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:16.950+0000 7fef81386700 1 --2- 172.21.15.97:0/2932994960 >> v2:172.21.15.97:6800/4140723216 conn(0x7fef68039140 0x7fef6803b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:17.706 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:16.950+0000 7fef81386700 1 --2- 172.21.15.97:0/2932994960 >> v2:172.21.15.97:6800/4140723216 conn(0x7fef68039140 0x7fef6803b600 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fef6c006fd0 tx=0x7fef6c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:17.706 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:16.955+0000 7fef72ffd700 1 -- 172.21.15.97:0/2932994960 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fef7802c3a0 con 0x7fef7c105090 2024-03-21T13:21:17.706 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:17.165+0000 7fef83deb700 1 -- 172.21.15.97:0/2932994960 --> v2:172.21.15.97:6800/4140723216 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fef7c101790 con 0x7fef68039140 2024-03-21T13:21:17.706 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:17.172+0000 7fef72ffd700 1 -- 172.21.15.97:0/2932994960 <== mgr.14120 v2:172.21.15.97:6800/4140723216 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fef7c101790 con 0x7fef68039140 2024-03-21T13:21:17.706 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:17.174+0000 7fef83deb700 1 -- 172.21.15.97:0/2932994960 >> v2:172.21.15.97:6800/4140723216 conn(0x7fef68039140 msgr2=0x7fef6803b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:17.706 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:17.174+0000 7fef83deb700 1 --2- 172.21.15.97:0/2932994960 >> v2:172.21.15.97:6800/4140723216 conn(0x7fef68039140 0x7fef6803b600 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fef6c006fd0 tx=0x7fef6c006e40 comp rx=0 tx=0).stop 2024-03-21T13:21:17.706 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:17.174+0000 7fef83deb700 1 -- 172.21.15.97:0/2932994960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef7c105090 msgr2=0x7fef7c079b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:17.706 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:17.174+0000 7fef83deb700 1 --2- 172.21.15.97:0/2932994960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef7c105090 0x7fef7c079b60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fef78011700 tx=0x7fef780046a0 comp rx=0 tx=0).stop 2024-03-21T13:21:17.706 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:17.174+0000 7fef83deb700 1 -- 172.21.15.97:0/2932994960 shutdown_connections 2024-03-21T13:21:17.707 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:17.174+0000 7fef83deb700 1 --2- 172.21.15.97:0/2932994960 >> v2:172.21.15.97:6800/4140723216 conn(0x7fef68039140 0x7fef6803b600 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:17.707 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:17.174+0000 7fef83deb700 1 --2- 172.21.15.97:0/2932994960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef7c105090 0x7fef7c079b60 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:17.707 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:17.174+0000 7fef83deb700 1 -- 172.21.15.97:0/2932994960 >> 172.21.15.97:0/2932994960 conn(0x7fef7c100630 msgr2=0x7fef7c101080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:21:17.707 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:17.175+0000 7fef83deb700 1 -- 172.21.15.97:0/2932994960 shutdown_connections 2024-03-21T13:21:17.707 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:17.175+0000 7fef83deb700 1 -- 172.21.15.97:0/2932994960 wait complete. 2024-03-21T13:21:17.707 INFO:teuthology.orchestra.run.smithi097.stdout:Deploying unmanaged mgr service... 2024-03-21T13:21:18.396 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:18 smithi097 ceph-mon[30040]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:21:18.396 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:18 smithi097 ceph-mon[30040]: Saving service mon spec with placement count:5 2024-03-21T13:21:18.396 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:18 smithi097 ceph-mon[30040]: from='mgr.14120 172.21.15.97:0/366686223' entity='mgr.y' 2024-03-21T13:21:19.659 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:19 smithi097 ceph-mon[30040]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:21:19.659 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:19 smithi097 ceph-mon[30040]: Saving service mgr spec with placement count:2 2024-03-21T13:21:19.660 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:19 smithi097 ceph-mon[30040]: from='mgr.14120 172.21.15.97:0/366686223' entity='mgr.y' 2024-03-21T13:21:19.660 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:19 smithi097 ceph-mon[30040]: from='mgr.14120 172.21.15.97:0/366686223' entity='mgr.y' 2024-03-21T13:21:19.666 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-21T13:21:19.666 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:18.303+0000 7f78d4bed700 1 Processor -- start 2024-03-21T13:21:19.667 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:18.303+0000 7f78d4bed700 1 -- start start 2024-03-21T13:21:19.667 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:18.303+0000 7f78d4bed700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78d0106c10 0x7f78d0109000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:19.667 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:18.303+0000 7f78d4bed700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d00796d0 con 0x7f78d0106c10 2024-03-21T13:21:19.667 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:18.304+0000 7f78ce59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78d0106c10 0x7f78d0109000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:19.667 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:18.304+0000 7f78ce59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78d0106c10 0x7f78d0109000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43256/0 (socket says 172.21.15.97:43256) 2024-03-21T13:21:19.667 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:18.304+0000 7f78ce59c700 1 -- 172.21.15.97:0/1939999579 learned_addr learned my addr 172.21.15.97:0/1939999579 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:21:19.667 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:18.304+0000 7f78ce59c700 1 -- 172.21.15.97:0/1939999579 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d0109540 con 0x7f78d0106c10 2024-03-21T13:21:19.667 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:18.304+0000 7f78ce59c700 1 --2- 172.21.15.97:0/1939999579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78d0106c10 0x7f78d0109000 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f78c00098a0 tx=0x7f78c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87c7ae686dfa3a87 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:19.667 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:18.304+0000 7f78cd59a700 1 -- 172.21.15.97:0/1939999579 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f78c0004160 con 0x7f78d0106c10 2024-03-21T13:21:19.667 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:18.304+0000 7f78cd59a700 1 -- 172.21.15.97:0/1939999579 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f78c0004300 con 0x7f78d0106c10 2024-03-21T13:21:19.667 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:18.304+0000 7f78cd59a700 1 -- 172.21.15.97:0/1939999579 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f78c0011a60 con 0x7f78d0106c10 2024-03-21T13:21:19.667 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:18.305+0000 7f78d4bed700 1 -- 172.21.15.97:0/1939999579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78d0106c10 msgr2=0x7f78d0109000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:19.667 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:18.305+0000 7f78d4bed700 1 --2- 172.21.15.97:0/1939999579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78d0106c10 0x7f78d0109000 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f78c00098a0 tx=0x7f78c0009bb0 comp rx=0 tx=0).stop 2024-03-21T13:21:19.667 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:18.305+0000 7f78d4bed700 1 -- 172.21.15.97:0/1939999579 shutdown_connections 2024-03-21T13:21:19.667 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:18.305+0000 7f78d4bed700 1 --2- 172.21.15.97:0/1939999579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78d0106c10 0x7f78d0109000 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:19.668 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:18.305+0000 7f78d4bed700 1 -- 172.21.15.97:0/1939999579 >> 172.21.15.97:0/1939999579 conn(0x7f78d01005e0 msgr2=0x7f78d0102a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:21:19.668 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:18.305+0000 7f78d4bed700 1 -- 172.21.15.97:0/1939999579 shutdown_connections 2024-03-21T13:21:19.668 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:18.305+0000 7f78d4bed700 1 -- 172.21.15.97:0/1939999579 wait complete. 2024-03-21T13:21:19.668 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:18.306+0000 7f78d4bed700 1 Processor -- start 2024-03-21T13:21:19.668 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:18.306+0000 7f78d4bed700 1 -- start start 2024-03-21T13:21:19.668 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:18.306+0000 7f78d4bed700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78d0106c10 0x7f78d019a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:19.668 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:18.306+0000 7f78d4bed700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d010a500 con 0x7f78d0106c10 2024-03-21T13:21:19.668 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:18.306+0000 7f78ce59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78d0106c10 0x7f78d019a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:19.668 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:18.306+0000 7f78ce59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78d0106c10 0x7f78d019a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43258/0 (socket says 172.21.15.97:43258) 2024-03-21T13:21:19.668 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:18.306+0000 7f78ce59c700 1 -- 172.21.15.97:0/890973544 learned_addr learned my addr 172.21.15.97:0/890973544 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:21:19.668 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:18.306+0000 7f78ce59c700 1 -- 172.21.15.97:0/890973544 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d019a580 con 0x7f78d0106c10 2024-03-21T13:21:19.668 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:18.306+0000 7f78ce59c700 1 --2- 172.21.15.97:0/890973544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78d0106c10 0x7f78d019a040 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f78c0011700 tx=0x7f78c0011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:19.668 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:18.307+0000 7f78bf7fe700 1 -- 172.21.15.97:0/890973544 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f78c001a430 con 0x7f78d0106c10 2024-03-21T13:21:19.668 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:18.307+0000 7f78bf7fe700 1 -- 172.21.15.97:0/890973544 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f78c001aa60 con 0x7f78d0106c10 2024-03-21T13:21:19.669 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:18.307+0000 7f78d4bed700 1 -- 172.21.15.97:0/890973544 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78d019a810 con 0x7f78d0106c10 2024-03-21T13:21:19.669 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:18.307+0000 7f78bf7fe700 1 -- 172.21.15.97:0/890973544 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f78c0024970 con 0x7f78d0106c10 2024-03-21T13:21:19.669 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:18.307+0000 7f78d4bed700 1 -- 172.21.15.97:0/890973544 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78d019d500 con 0x7f78d0106c10 2024-03-21T13:21:19.669 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:18.308+0000 7f78bf7fe700 1 -- 172.21.15.97:0/890973544 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 46259+0+0 (secure 0 0 0) 0x7f78c0022020 con 0x7f78d0106c10 2024-03-21T13:21:19.669 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:18.308+0000 7f78bf7fe700 1 --2- 172.21.15.97:0/890973544 >> v2:172.21.15.97:6800/4140723216 conn(0x7f78b8039190 0x7f78b803b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:19.669 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:18.308+0000 7f78bf7fe700 1 -- 172.21.15.97:0/890973544 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (secure 0 0 0) 0x7f78c001a5d0 con 0x7f78d0106c10 2024-03-21T13:21:19.669 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:18.308+0000 7f78d4bed700 1 -- 172.21.15.97:0/890973544 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78d00637c0 con 0x7f78d0106c10 2024-03-21T13:21:19.669 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:18.308+0000 7f78cdd9b700 1 --2- 172.21.15.97:0/890973544 >> v2:172.21.15.97:6800/4140723216 conn(0x7f78b8039190 0x7f78b803b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:19.669 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:18.308+0000 7f78cdd9b700 1 --2- 172.21.15.97:0/890973544 >> v2:172.21.15.97:6800/4140723216 conn(0x7f78b8039190 0x7f78b803b650 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f78c4006fd0 tx=0x7f78c4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:19.669 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:18.313+0000 7f78bf7fe700 1 -- 172.21.15.97:0/890973544 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f78c000b3e0 con 0x7f78d0106c10 2024-03-21T13:21:19.669 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:18.533+0000 7f78d4bed700 1 -- 172.21.15.97:0/890973544 --> v2:172.21.15.97:6800/4140723216 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f78d0101710 con 0x7f78b8039190 2024-03-21T13:21:19.669 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:18.541+0000 7f78bf7fe700 1 -- 172.21.15.97:0/890973544 <== mgr.14120 v2:172.21.15.97:6800/4140723216 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f78d0101710 con 0x7f78b8039190 2024-03-21T13:21:19.669 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:18.543+0000 7f78d4bed700 1 -- 172.21.15.97:0/890973544 >> v2:172.21.15.97:6800/4140723216 conn(0x7f78b8039190 msgr2=0x7f78b803b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:19.669 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:18.543+0000 7f78d4bed700 1 --2- 172.21.15.97:0/890973544 >> v2:172.21.15.97:6800/4140723216 conn(0x7f78b8039190 0x7f78b803b650 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f78c4006fd0 tx=0x7f78c4006e40 comp rx=0 tx=0).stop 2024-03-21T13:21:19.669 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:18.543+0000 7f78d4bed700 1 -- 172.21.15.97:0/890973544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78d0106c10 msgr2=0x7f78d019a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:19.670 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:18.543+0000 7f78d4bed700 1 --2- 172.21.15.97:0/890973544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78d0106c10 0x7f78d019a040 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f78c0011700 tx=0x7f78c0011b50 comp rx=0 tx=0).stop 2024-03-21T13:21:19.670 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:18.543+0000 7f78d4bed700 1 -- 172.21.15.97:0/890973544 shutdown_connections 2024-03-21T13:21:19.670 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:18.543+0000 7f78d4bed700 1 --2- 172.21.15.97:0/890973544 >> v2:172.21.15.97:6800/4140723216 conn(0x7f78b8039190 0x7f78b803b650 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:19.670 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:18.543+0000 7f78d4bed700 1 --2- 172.21.15.97:0/890973544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78d0106c10 0x7f78d019a040 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:19.670 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:18.543+0000 7f78d4bed700 1 -- 172.21.15.97:0/890973544 >> 172.21.15.97:0/890973544 conn(0x7f78d01005e0 msgr2=0x7f78d0101000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:21:19.670 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:18.543+0000 7f78d4bed700 1 -- 172.21.15.97:0/890973544 shutdown_connections 2024-03-21T13:21:19.670 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:18.543+0000 7f78d4bed700 1 -- 172.21.15.97:0/890973544 wait complete. 2024-03-21T13:21:20.948 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:20.166+0000 7f04b626b700 1 Processor -- start 2024-03-21T13:21:20.948 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:20.166+0000 7f04b626b700 1 -- start start 2024-03-21T13:21:20.948 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:20.166+0000 7f04b626b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04b0106bf0 0x7f04b0108fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:20.948 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:20.166+0000 7f04b626b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b00796d0 con 0x7f04b0106bf0 2024-03-21T13:21:20.948 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:20.166+0000 7f04affff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04b0106bf0 0x7f04b0108fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:20.948 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:20.167+0000 7f04affff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04b0106bf0 0x7f04b0108fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43270/0 (socket says 172.21.15.97:43270) 2024-03-21T13:21:20.948 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:20.167+0000 7f04affff700 1 -- 172.21.15.97:0/2820322996 learned_addr learned my addr 172.21.15.97:0/2820322996 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:21:20.948 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:20.167+0000 7f04affff700 1 -- 172.21.15.97:0/2820322996 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b0109520 con 0x7f04b0106bf0 2024-03-21T13:21:20.948 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:20.167+0000 7f04affff700 1 --2- 172.21.15.97:0/2820322996 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04b0106bf0 0x7f04b0108fe0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f04a40098a0 tx=0x7f04a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=156c6f87ae6b2cf1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:20.948 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:20.167+0000 7f04aeffd700 1 -- 172.21.15.97:0/2820322996 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f04a4004160 con 0x7f04b0106bf0 2024-03-21T13:21:20.948 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:20.167+0000 7f04aeffd700 1 -- 172.21.15.97:0/2820322996 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f04a4004300 con 0x7f04b0106bf0 2024-03-21T13:21:20.948 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:20.167+0000 7f04aeffd700 1 -- 172.21.15.97:0/2820322996 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f04a4011a00 con 0x7f04b0106bf0 2024-03-21T13:21:20.948 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:20.168+0000 7f04b626b700 1 -- 172.21.15.97:0/2820322996 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04b0106bf0 msgr2=0x7f04b0108fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:20.948 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:20.168+0000 7f04b626b700 1 --2- 172.21.15.97:0/2820322996 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04b0106bf0 0x7f04b0108fe0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f04a40098a0 tx=0x7f04a4009bb0 comp rx=0 tx=0).stop 2024-03-21T13:21:20.949 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:20.168+0000 7f04b626b700 1 -- 172.21.15.97:0/2820322996 shutdown_connections 2024-03-21T13:21:20.949 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:20.168+0000 7f04b626b700 1 --2- 172.21.15.97:0/2820322996 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04b0106bf0 0x7f04b0108fe0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:20.949 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:20.168+0000 7f04b626b700 1 -- 172.21.15.97:0/2820322996 >> 172.21.15.97:0/2820322996 conn(0x7f04b01005c0 msgr2=0x7f04b0102a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:21:20.949 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:20.168+0000 7f04b626b700 1 -- 172.21.15.97:0/2820322996 shutdown_connections 2024-03-21T13:21:20.949 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:20.168+0000 7f04b626b700 1 -- 172.21.15.97:0/2820322996 wait complete. 2024-03-21T13:21:20.949 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:20.168+0000 7f04b626b700 1 Processor -- start 2024-03-21T13:21:20.949 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:20.168+0000 7f04b626b700 1 -- start start 2024-03-21T13:21:20.949 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:20.169+0000 7f04b626b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04b0106bf0 0x7f04b0199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:20.949 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:20.169+0000 7f04b626b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b010a520 con 0x7f04b0106bf0 2024-03-21T13:21:20.949 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:20.169+0000 7f04affff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04b0106bf0 0x7f04b0199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:20.949 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:20.169+0000 7f04affff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04b0106bf0 0x7f04b0199fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43272/0 (socket says 172.21.15.97:43272) 2024-03-21T13:21:20.949 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:20.169+0000 7f04affff700 1 -- 172.21.15.97:0/2571448079 learned_addr learned my addr 172.21.15.97:0/2571448079 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:21:20.949 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:20.169+0000 7f04affff700 1 -- 172.21.15.97:0/2571448079 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b019a520 con 0x7f04b0106bf0 2024-03-21T13:21:20.949 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:20.169+0000 7f04affff700 1 --2- 172.21.15.97:0/2571448079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04b0106bf0 0x7f04b0199fe0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f04a4011a10 tx=0x7f04a4011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:20.949 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:20.169+0000 7f04ad7fa700 1 -- 172.21.15.97:0/2571448079 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f04a401a430 con 0x7f04b0106bf0 2024-03-21T13:21:20.950 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:20.169+0000 7f04ad7fa700 1 -- 172.21.15.97:0/2571448079 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f04a401aa80 con 0x7f04b0106bf0 2024-03-21T13:21:20.950 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:20.169+0000 7f04ad7fa700 1 -- 172.21.15.97:0/2571448079 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f04a40249d0 con 0x7f04b0106bf0 2024-03-21T13:21:20.950 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:20.169+0000 7f04b626b700 1 -- 172.21.15.97:0/2571448079 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04b019a7b0 con 0x7f04b0106bf0 2024-03-21T13:21:20.950 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:20.169+0000 7f04b626b700 1 -- 172.21.15.97:0/2571448079 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04b019d4a0 con 0x7f04b0106bf0 2024-03-21T13:21:20.950 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:20.170+0000 7f04ad7fa700 1 -- 172.21.15.97:0/2571448079 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 46259+0+0 (secure 0 0 0) 0x7f04a4022020 con 0x7f04b0106bf0 2024-03-21T13:21:20.950 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:20.170+0000 7f04b626b700 1 -- 172.21.15.97:0/2571448079 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04b00637c0 con 0x7f04b0106bf0 2024-03-21T13:21:20.950 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:20.170+0000 7f04ad7fa700 1 --2- 172.21.15.97:0/2571448079 >> v2:172.21.15.97:6800/4140723216 conn(0x7f04980390f0 0x7f049803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:20.950 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:20.171+0000 7f04ad7fa700 1 -- 172.21.15.97:0/2571448079 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (secure 0 0 0) 0x7f04a404d5c0 con 0x7f04b0106bf0 2024-03-21T13:21:20.950 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:20.171+0000 7f04af7fe700 1 --2- 172.21.15.97:0/2571448079 >> v2:172.21.15.97:6800/4140723216 conn(0x7f04980390f0 0x7f049803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:20.950 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:20.171+0000 7f04af7fe700 1 --2- 172.21.15.97:0/2571448079 >> v2:172.21.15.97:6800/4140723216 conn(0x7f04980390f0 0x7f049803b5b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f04a0006fd0 tx=0x7f04a0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:20.950 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:20.175+0000 7f04ad7fa700 1 -- 172.21.15.97:0/2571448079 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f04a400b3e0 con 0x7f04b0106bf0 2024-03-21T13:21:20.950 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:20.385+0000 7f04b626b700 1 -- 172.21.15.97:0/2571448079 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f04b019d880 con 0x7f04b0106bf0 2024-03-21T13:21:20.950 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:20.390+0000 7f04ad7fa700 1 -- 172.21.15.97:0/2571448079 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f04a40172b0 con 0x7f04b0106bf0 2024-03-21T13:21:20.950 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:20.393+0000 7f04b626b700 1 -- 172.21.15.97:0/2571448079 >> v2:172.21.15.97:6800/4140723216 conn(0x7f04980390f0 msgr2=0x7f049803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:20.950 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:20.393+0000 7f04b626b700 1 --2- 172.21.15.97:0/2571448079 >> v2:172.21.15.97:6800/4140723216 conn(0x7f04980390f0 0x7f049803b5b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f04a0006fd0 tx=0x7f04a0006e40 comp rx=0 tx=0).stop 2024-03-21T13:21:20.950 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:20.393+0000 7f04b626b700 1 -- 172.21.15.97:0/2571448079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04b0106bf0 msgr2=0x7f04b0199fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:20.951 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:20.393+0000 7f04b626b700 1 --2- 172.21.15.97:0/2571448079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04b0106bf0 0x7f04b0199fe0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f04a4011a10 tx=0x7f04a4011af0 comp rx=0 tx=0).stop 2024-03-21T13:21:20.951 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:20.393+0000 7f04b626b700 1 -- 172.21.15.97:0/2571448079 shutdown_connections 2024-03-21T13:21:20.951 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:20.393+0000 7f04b626b700 1 --2- 172.21.15.97:0/2571448079 >> v2:172.21.15.97:6800/4140723216 conn(0x7f04980390f0 0x7f049803b5b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:20.951 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:20.393+0000 7f04b626b700 1 --2- 172.21.15.97:0/2571448079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04b0106bf0 0x7f04b0199fe0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:20.951 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:20.393+0000 7f04b626b700 1 -- 172.21.15.97:0/2571448079 >> 172.21.15.97:0/2571448079 conn(0x7f04b01005c0 msgr2=0x7f04b0101060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:21:20.951 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:20.393+0000 7f04b626b700 1 -- 172.21.15.97:0/2571448079 shutdown_connections 2024-03-21T13:21:20.951 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:20.393+0000 7f04b626b700 1 -- 172.21.15.97:0/2571448079 wait complete. 2024-03-21T13:21:21.224 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:20 smithi097 ceph-mon[30040]: from='mgr.14120 172.21.15.97:0/366686223' entity='mgr.y' 2024-03-21T13:21:21.224 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:20 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2571448079' entity='client.admin' 2024-03-21T13:21:22.408 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:21.440+0000 7f438d83b700 1 Processor -- start 2024-03-21T13:21:22.408 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:21.440+0000 7f438d83b700 1 -- start start 2024-03-21T13:21:22.408 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:21.440+0000 7f438d83b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4388104a20 0x7f4388104e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:22.408 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:21.440+0000 7f438d83b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43880796b0 con 0x7f4388104a20 2024-03-21T13:21:22.408 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:21.441+0000 7f4386ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4388104a20 0x7f4388104e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:22.408 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:21.441+0000 7f4386ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4388104a20 0x7f4388104e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55134/0 (socket says 172.21.15.97:55134) 2024-03-21T13:21:22.408 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:21.441+0000 7f4386ffd700 1 -- 172.21.15.97:0/3551713436 learned_addr learned my addr 172.21.15.97:0/3551713436 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:21:22.409 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:21.441+0000 7f4386ffd700 1 -- 172.21.15.97:0/3551713436 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4388105380 con 0x7f4388104a20 2024-03-21T13:21:22.409 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:21.441+0000 7f4386ffd700 1 --2- 172.21.15.97:0/3551713436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4388104a20 0x7f4388104e40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f43780098a0 tx=0x7f4378009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ff0fddecd996bf server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:22.409 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:21.441+0000 7f4385ffb700 1 -- 172.21.15.97:0/3551713436 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4378004160 con 0x7f4388104a20 2024-03-21T13:21:22.409 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:21.441+0000 7f4385ffb700 1 -- 172.21.15.97:0/3551713436 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f4378004300 con 0x7f4388104a20 2024-03-21T13:21:22.409 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:21.441+0000 7f4385ffb700 1 -- 172.21.15.97:0/3551713436 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4378011a00 con 0x7f4388104a20 2024-03-21T13:21:22.409 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:21.442+0000 7f438d83b700 1 -- 172.21.15.97:0/3551713436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4388104a20 msgr2=0x7f4388104e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:22.409 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:21.442+0000 7f438d83b700 1 --2- 172.21.15.97:0/3551713436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4388104a20 0x7f4388104e40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f43780098a0 tx=0x7f4378009bb0 comp rx=0 tx=0).stop 2024-03-21T13:21:22.409 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:21.442+0000 7f438d83b700 1 -- 172.21.15.97:0/3551713436 shutdown_connections 2024-03-21T13:21:22.409 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:21.442+0000 7f438d83b700 1 --2- 172.21.15.97:0/3551713436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4388104a20 0x7f4388104e40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:22.409 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:21.442+0000 7f438d83b700 1 -- 172.21.15.97:0/3551713436 >> 172.21.15.97:0/3551713436 conn(0x7f43881005e0 msgr2=0x7f4388102a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:21:22.409 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:21.442+0000 7f438d83b700 1 -- 172.21.15.97:0/3551713436 shutdown_connections 2024-03-21T13:21:22.409 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:21.442+0000 7f438d83b700 1 -- 172.21.15.97:0/3551713436 wait complete. 2024-03-21T13:21:22.409 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:21.443+0000 7f438d83b700 1 Processor -- start 2024-03-21T13:21:22.409 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:21.443+0000 7f438d83b700 1 -- start start 2024-03-21T13:21:22.410 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:21.443+0000 7f438d83b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4388104a20 0x7f438819c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:22.410 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:21.443+0000 7f438d83b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43881063f0 con 0x7f4388104a20 2024-03-21T13:21:22.410 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:21.443+0000 7f4386ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4388104a20 0x7f438819c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:22.410 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:21.443+0000 7f4386ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4388104a20 0x7f438819c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55146/0 (socket says 172.21.15.97:55146) 2024-03-21T13:21:22.410 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:21.443+0000 7f4386ffd700 1 -- 172.21.15.97:0/3429864911 learned_addr learned my addr 172.21.15.97:0/3429864911 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:21:22.410 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:21.443+0000 7f4386ffd700 1 -- 172.21.15.97:0/3429864911 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f438819c760 con 0x7f4388104a20 2024-03-21T13:21:22.410 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:21.444+0000 7f4386ffd700 1 --2- 172.21.15.97:0/3429864911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4388104a20 0x7f438819c220 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4378011a10 tx=0x7f4378011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:22.410 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:21.444+0000 7f438c839700 1 -- 172.21.15.97:0/3429864911 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f437801a430 con 0x7f4388104a20 2024-03-21T13:21:22.410 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:21.444+0000 7f438d83b700 1 -- 172.21.15.97:0/3429864911 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f438819c9f0 con 0x7f4388104a20 2024-03-21T13:21:22.410 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:21.444+0000 7f438c839700 1 -- 172.21.15.97:0/3429864911 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f437801aa80 con 0x7f4388104a20 2024-03-21T13:21:22.410 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:21.444+0000 7f438d83b700 1 -- 172.21.15.97:0/3429864911 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f438819f6e0 con 0x7f4388104a20 2024-03-21T13:21:22.410 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:21.444+0000 7f438c839700 1 -- 172.21.15.97:0/3429864911 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4378024970 con 0x7f4388104a20 2024-03-21T13:21:22.410 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:21.445+0000 7f438c839700 1 -- 172.21.15.97:0/3429864911 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 46259+0+0 (secure 0 0 0) 0x7f4378022020 con 0x7f4388104a20 2024-03-21T13:21:22.410 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:21.445+0000 7f438c839700 1 --2- 172.21.15.97:0/3429864911 >> v2:172.21.15.97:6800/4140723216 conn(0x7f43700390f0 0x7f437003b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:22.410 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:21.445+0000 7f438c839700 1 -- 172.21.15.97:0/3429864911 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (secure 0 0 0) 0x7f437804e0d0 con 0x7f4388104a20 2024-03-21T13:21:22.411 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:21.445+0000 7f43867fc700 1 --2- 172.21.15.97:0/3429864911 >> v2:172.21.15.97:6800/4140723216 conn(0x7f43700390f0 0x7f437003b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:22.411 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:21.445+0000 7f438d83b700 1 -- 172.21.15.97:0/3429864911 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43880637c0 con 0x7f4388104a20 2024-03-21T13:21:22.411 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:21.445+0000 7f43867fc700 1 --2- 172.21.15.97:0/3429864911 >> v2:172.21.15.97:6800/4140723216 conn(0x7f43700390f0 0x7f437003b5b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f437c006fd0 tx=0x7f437c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:22.411 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:21.450+0000 7f438c839700 1 -- 172.21.15.97:0/3429864911 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4378016dd0 con 0x7f4388104a20 2024-03-21T13:21:22.411 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:21.649+0000 7f438d83b700 1 -- 172.21.15.97:0/3429864911 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f43881956e0 con 0x7f4388104a20 2024-03-21T13:21:22.411 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:21.651+0000 7f438c839700 1 -- 172.21.15.97:0/3429864911 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f437800ba20 con 0x7f4388104a20 2024-03-21T13:21:22.411 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:21.653+0000 7f438d83b700 1 -- 172.21.15.97:0/3429864911 >> v2:172.21.15.97:6800/4140723216 conn(0x7f43700390f0 msgr2=0x7f437003b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:22.411 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:21.653+0000 7f438d83b700 1 --2- 172.21.15.97:0/3429864911 >> v2:172.21.15.97:6800/4140723216 conn(0x7f43700390f0 0x7f437003b5b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f437c006fd0 tx=0x7f437c006e40 comp rx=0 tx=0).stop 2024-03-21T13:21:22.411 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:21.653+0000 7f438d83b700 1 -- 172.21.15.97:0/3429864911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4388104a20 msgr2=0x7f438819c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:22.411 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:21.653+0000 7f438d83b700 1 --2- 172.21.15.97:0/3429864911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4388104a20 0x7f438819c220 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4378011a10 tx=0x7f4378011af0 comp rx=0 tx=0).stop 2024-03-21T13:21:22.411 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:21.653+0000 7f438d83b700 1 -- 172.21.15.97:0/3429864911 shutdown_connections 2024-03-21T13:21:22.411 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:21.653+0000 7f438d83b700 1 --2- 172.21.15.97:0/3429864911 >> v2:172.21.15.97:6800/4140723216 conn(0x7f43700390f0 0x7f437003b5b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:22.411 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:21.653+0000 7f438d83b700 1 --2- 172.21.15.97:0/3429864911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4388104a20 0x7f438819c220 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:22.411 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:21.653+0000 7f438d83b700 1 -- 172.21.15.97:0/3429864911 >> 172.21.15.97:0/3429864911 conn(0x7f43881005e0 msgr2=0x7f4388192e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:21:22.412 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:21.654+0000 7f438d83b700 1 -- 172.21.15.97:0/3429864911 shutdown_connections 2024-03-21T13:21:22.412 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:21.654+0000 7f438d83b700 1 -- 172.21.15.97:0/3429864911 wait complete. 2024-03-21T13:21:22.412 INFO:teuthology.orchestra.run.smithi097.stdout:Enabling the dashboard module... 2024-03-21T13:21:22.662 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:22 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3429864911' entity='client.admin' 2024-03-21T13:21:24.476 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:24 smithi097 ceph-mon[30040]: from='mgr.14120 172.21.15.97:0/366686223' entity='mgr.y' 2024-03-21T13:21:24.476 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:24 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2758621260' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-21T13:21:24.476 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:24 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ignoring --setuser ceph since I am not root 2024-03-21T13:21:24.476 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:24 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ignoring --setgroup ceph since I am not root 2024-03-21T13:21:24.792 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:23.106+0000 7fabddd96700 1 Processor -- start 2024-03-21T13:21:24.793 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:23.106+0000 7fabddd96700 1 -- start start 2024-03-21T13:21:24.793 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:23.106+0000 7fabddd96700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabd8105060 0x7fabd8105480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:24.793 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:23.106+0000 7fabddd96700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd8105a50 con 0x7fabd8105060 2024-03-21T13:21:24.793 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:23.107+0000 7fabd77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabd8105060 0x7fabd8105480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:24.793 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:23.107+0000 7fabd77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabd8105060 0x7fabd8105480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55154/0 (socket says 172.21.15.97:55154) 2024-03-21T13:21:24.793 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:23.107+0000 7fabd77fe700 1 -- 172.21.15.97:0/2741938824 learned_addr learned my addr 172.21.15.97:0/2741938824 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:21:24.793 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:23.107+0000 7fabd77fe700 1 -- 172.21.15.97:0/2741938824 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd8106260 con 0x7fabd8105060 2024-03-21T13:21:24.793 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:23.107+0000 7fabd77fe700 1 --2- 172.21.15.97:0/2741938824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabd8105060 0x7fabd8105480 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fabc80098a0 tx=0x7fabc8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=816c32ffc7645d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:24.793 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:23.107+0000 7fabd67fc700 1 -- 172.21.15.97:0/2741938824 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fabc8004160 con 0x7fabd8105060 2024-03-21T13:21:24.793 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:23.107+0000 7fabd67fc700 1 -- 172.21.15.97:0/2741938824 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fabc8004300 con 0x7fabd8105060 2024-03-21T13:21:24.793 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:23.107+0000 7fabd67fc700 1 -- 172.21.15.97:0/2741938824 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fabc8011a00 con 0x7fabd8105060 2024-03-21T13:21:24.793 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:23.108+0000 7fabddd96700 1 -- 172.21.15.97:0/2741938824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabd8105060 msgr2=0x7fabd8105480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:24.794 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:23.108+0000 7fabddd96700 1 --2- 172.21.15.97:0/2741938824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabd8105060 0x7fabd8105480 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fabc80098a0 tx=0x7fabc8009bb0 comp rx=0 tx=0).stop 2024-03-21T13:21:24.794 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:23.108+0000 7fabddd96700 1 -- 172.21.15.97:0/2741938824 shutdown_connections 2024-03-21T13:21:24.794 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:23.108+0000 7fabddd96700 1 --2- 172.21.15.97:0/2741938824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabd8105060 0x7fabd8105480 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:24.794 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:23.108+0000 7fabddd96700 1 -- 172.21.15.97:0/2741938824 >> 172.21.15.97:0/2741938824 conn(0x7fabd8100600 msgr2=0x7fabd8102a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:21:24.794 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:23.108+0000 7fabddd96700 1 -- 172.21.15.97:0/2741938824 shutdown_connections 2024-03-21T13:21:24.794 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:23.108+0000 7fabddd96700 1 -- 172.21.15.97:0/2741938824 wait complete. 2024-03-21T13:21:24.794 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:23.109+0000 7fabddd96700 1 Processor -- start 2024-03-21T13:21:24.794 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:23.109+0000 7fabddd96700 1 -- start start 2024-03-21T13:21:24.794 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:23.109+0000 7fabddd96700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabd8105060 0x7fabd8079b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:24.794 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:23.109+0000 7fabddd96700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd8109610 con 0x7fabd8105060 2024-03-21T13:21:24.794 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:23.109+0000 7fabd77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabd8105060 0x7fabd8079b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:24.794 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:23.109+0000 7fabd77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabd8105060 0x7fabd8079b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55166/0 (socket says 172.21.15.97:55166) 2024-03-21T13:21:24.794 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:23.109+0000 7fabd77fe700 1 -- 172.21.15.97:0/2758621260 learned_addr learned my addr 172.21.15.97:0/2758621260 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:21:24.795 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:23.110+0000 7fabd77fe700 1 -- 172.21.15.97:0/2758621260 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd819aba0 con 0x7fabd8105060 2024-03-21T13:21:24.795 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:23.110+0000 7fabd77fe700 1 --2- 172.21.15.97:0/2758621260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabd8105060 0x7fabd8079b40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fabc8009fb0 tx=0x7fabc8011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:24.795 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:23.110+0000 7fabd4ff9700 1 -- 172.21.15.97:0/2758621260 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fabc801a430 con 0x7fabd8105060 2024-03-21T13:21:24.795 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:23.110+0000 7fabd4ff9700 1 -- 172.21.15.97:0/2758621260 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fabc801aa80 con 0x7fabd8105060 2024-03-21T13:21:24.795 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:23.110+0000 7fabddd96700 1 -- 172.21.15.97:0/2758621260 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabd819f1b0 con 0x7fabd8105060 2024-03-21T13:21:24.795 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:23.110+0000 7fabd4ff9700 1 -- 172.21.15.97:0/2758621260 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fabc80249b0 con 0x7fabd8105060 2024-03-21T13:21:24.795 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:23.110+0000 7fabddd96700 1 -- 172.21.15.97:0/2758621260 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabd819f550 con 0x7fabd8105060 2024-03-21T13:21:24.795 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:23.111+0000 7fabd4ff9700 1 -- 172.21.15.97:0/2758621260 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 46259+0+0 (secure 0 0 0) 0x7fabc8022020 con 0x7fabd8105060 2024-03-21T13:21:24.795 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:23.111+0000 7fabd4ff9700 1 --2- 172.21.15.97:0/2758621260 >> v2:172.21.15.97:6800/4140723216 conn(0x7fabc0039140 0x7fabc003b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:24.795 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:23.111+0000 7fabd4ff9700 1 -- 172.21.15.97:0/2758621260 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (secure 0 0 0) 0x7fabc804d630 con 0x7fabd8105060 2024-03-21T13:21:24.795 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:23.111+0000 7fabd6ffd700 1 --2- 172.21.15.97:0/2758621260 >> v2:172.21.15.97:6800/4140723216 conn(0x7fabc0039140 0x7fabc003b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:24.795 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:23.111+0000 7fabddd96700 1 -- 172.21.15.97:0/2758621260 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabd80637c0 con 0x7fabd8105060 2024-03-21T13:21:24.795 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:23.112+0000 7fabd6ffd700 1 --2- 172.21.15.97:0/2758621260 >> v2:172.21.15.97:6800/4140723216 conn(0x7fabc0039140 0x7fabc003b600 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fabcc006fd0 tx=0x7fabcc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:24.795 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:23.117+0000 7fabd4ff9700 1 -- 172.21.15.97:0/2758621260 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fabc800b3e0 con 0x7fabd8105060 2024-03-21T13:21:24.796 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:23.375+0000 7fabddd96700 1 -- 172.21.15.97:0/2758621260 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fabd819fa20 con 0x7fabd8105060 2024-03-21T13:21:24.796 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:24.146+0000 7fabd4ff9700 1 -- 172.21.15.97:0/2758621260 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7fabc80172b0 con 0x7fabd8105060 2024-03-21T13:21:24.796 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:24.148+0000 7fabd4ff9700 1 -- 172.21.15.97:0/2758621260 <== mon.0 v2:172.21.15.97:3300/0 8 ==== mgrmap(e 9) v1 ==== 46272+0+0 (secure 0 0 0) 0x7fabc802d3f0 con 0x7fabd8105060 2024-03-21T13:21:24.796 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:24.149+0000 7fabddd96700 1 -- 172.21.15.97:0/2758621260 >> v2:172.21.15.97:6800/4140723216 conn(0x7fabc0039140 msgr2=0x7fabc003b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:24.796 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:24.149+0000 7fabddd96700 1 --2- 172.21.15.97:0/2758621260 >> v2:172.21.15.97:6800/4140723216 conn(0x7fabc0039140 0x7fabc003b600 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fabcc006fd0 tx=0x7fabcc006e40 comp rx=0 tx=0).stop 2024-03-21T13:21:24.796 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:24.149+0000 7fabddd96700 1 -- 172.21.15.97:0/2758621260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabd8105060 msgr2=0x7fabd8079b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:24.796 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:24.149+0000 7fabddd96700 1 --2- 172.21.15.97:0/2758621260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabd8105060 0x7fabd8079b40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fabc8009fb0 tx=0x7fabc8011ac0 comp rx=0 tx=0).stop 2024-03-21T13:21:24.796 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:24.149+0000 7fabddd96700 1 -- 172.21.15.97:0/2758621260 shutdown_connections 2024-03-21T13:21:24.796 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:24.149+0000 7fabddd96700 1 --2- 172.21.15.97:0/2758621260 >> v2:172.21.15.97:6800/4140723216 conn(0x7fabc0039140 0x7fabc003b600 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:24.796 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:24.149+0000 7fabddd96700 1 --2- 172.21.15.97:0/2758621260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabd8105060 0x7fabd8079b40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:24.796 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:24.149+0000 7fabddd96700 1 -- 172.21.15.97:0/2758621260 >> 172.21.15.97:0/2758621260 conn(0x7fabd8100600 msgr2=0x7fabd81010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:21:24.796 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:24.150+0000 7fabddd96700 1 -- 172.21.15.97:0/2758621260 shutdown_connections 2024-03-21T13:21:24.796 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:24.150+0000 7fabddd96700 1 -- 172.21.15.97:0/2758621260 wait complete. 2024-03-21T13:21:25.039 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:24 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:21:24.729+0000 7fcc75a66200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-21T13:21:25.039 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:24 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:21:24.827+0000 7fcc75a66200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-21T13:21:25.295 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:25 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2758621260' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-21T13:21:25.295 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:25 smithi097 ceph-mon[30040]: mgrmap e9: y(active, since 19s) 2024-03-21T13:21:25.553 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:25 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:21:25.506+0000 7fcc75a66200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-21T13:21:25.855 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:25 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:21:25.610+0000 7fcc75a66200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-21T13:21:25.855 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:25 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:21:25.754+0000 7fcc75a66200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-21T13:21:25.855 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:25 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:21:25.854+0000 7fcc75a66200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-21T13:21:26.152 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:25 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:21:25.937+0000 7fcc75a66200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-21T13:21:26.159 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout { 2024-03-21T13:21:26.165 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-21T13:21:26.165 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-21T13:21:26.165 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-03-21T13:21:26.166 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-21T13:21:26.166 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout } 2024-03-21T13:21:26.166 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.294+0000 7f1c9b9c1700 1 Processor -- start 2024-03-21T13:21:26.166 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.294+0000 7f1c9b9c1700 1 -- start start 2024-03-21T13:21:26.166 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.294+0000 7f1c9b9c1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c94106170 0x7f1c94106590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:26.166 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.294+0000 7f1c9b9c1700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c94106b60 con 0x7f1c94106170 2024-03-21T13:21:26.166 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.295+0000 7f1c9975d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c94106170 0x7f1c94106590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:26.166 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.295+0000 7f1c9975d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c94106170 0x7f1c94106590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55184/0 (socket says 172.21.15.97:55184) 2024-03-21T13:21:26.166 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.295+0000 7f1c9975d700 1 -- 172.21.15.97:0/2895327488 learned_addr learned my addr 172.21.15.97:0/2895327488 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:21:26.166 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.295+0000 7f1c9975d700 1 -- 172.21.15.97:0/2895327488 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c94107370 con 0x7f1c94106170 2024-03-21T13:21:26.166 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.295+0000 7f1c9975d700 1 --2- 172.21.15.97:0/2895327488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c94106170 0x7f1c94106590 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1c840098a0 tx=0x7f1c84009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=191846822d19849e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:26.166 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.296+0000 7f1c8bfff700 1 -- 172.21.15.97:0/2895327488 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c84004160 con 0x7f1c94106170 2024-03-21T13:21:26.166 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.296+0000 7f1c8bfff700 1 -- 172.21.15.97:0/2895327488 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f1c84004300 con 0x7f1c94106170 2024-03-21T13:21:26.167 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.296+0000 7f1c8bfff700 1 -- 172.21.15.97:0/2895327488 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c84011a00 con 0x7f1c94106170 2024-03-21T13:21:26.167 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.296+0000 7f1c9b9c1700 1 -- 172.21.15.97:0/2895327488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c94106170 msgr2=0x7f1c94106590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:26.167 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.296+0000 7f1c9b9c1700 1 --2- 172.21.15.97:0/2895327488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c94106170 0x7f1c94106590 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1c840098a0 tx=0x7f1c84009bb0 comp rx=0 tx=0).stop 2024-03-21T13:21:26.167 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.296+0000 7f1c9b9c1700 1 -- 172.21.15.97:0/2895327488 shutdown_connections 2024-03-21T13:21:26.167 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.296+0000 7f1c9b9c1700 1 --2- 172.21.15.97:0/2895327488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c94106170 0x7f1c94106590 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:26.167 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.296+0000 7f1c9b9c1700 1 -- 172.21.15.97:0/2895327488 >> 172.21.15.97:0/2895327488 conn(0x7f1c941016d0 msgr2=0x7f1c94103b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:21:26.167 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.297+0000 7f1c9b9c1700 1 -- 172.21.15.97:0/2895327488 shutdown_connections 2024-03-21T13:21:26.167 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.297+0000 7f1c9b9c1700 1 -- 172.21.15.97:0/2895327488 wait complete. 2024-03-21T13:21:26.167 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.297+0000 7f1c9b9c1700 1 Processor -- start 2024-03-21T13:21:26.167 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.297+0000 7f1c9b9c1700 1 -- start start 2024-03-21T13:21:26.167 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.297+0000 7f1c9b9c1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c94106170 0x7f1c9419a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:26.167 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.297+0000 7f1c9b9c1700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9410a720 con 0x7f1c94106170 2024-03-21T13:21:26.168 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.297+0000 7f1c9975d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c94106170 0x7f1c9419a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:26.168 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.298+0000 7f1c9975d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c94106170 0x7f1c9419a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55186/0 (socket says 172.21.15.97:55186) 2024-03-21T13:21:26.168 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.298+0000 7f1c9975d700 1 -- 172.21.15.97:0/4073227302 learned_addr learned my addr 172.21.15.97:0/4073227302 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:21:26.168 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.298+0000 7f1c9975d700 1 -- 172.21.15.97:0/4073227302 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c9419a5a0 con 0x7f1c94106170 2024-03-21T13:21:26.168 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.298+0000 7f1c9975d700 1 --2- 172.21.15.97:0/4073227302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c94106170 0x7f1c9419a060 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1c84009870 tx=0x7f1c84011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:26.168 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.298+0000 7f1c8a7fc700 1 -- 172.21.15.97:0/4073227302 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c84003af0 con 0x7f1c94106170 2024-03-21T13:21:26.168 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.298+0000 7f1c9b9c1700 1 -- 172.21.15.97:0/4073227302 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c9419a830 con 0x7f1c94106170 2024-03-21T13:21:26.168 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.298+0000 7f1c8a7fc700 1 -- 172.21.15.97:0/4073227302 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f1c8401a8e0 con 0x7f1c94106170 2024-03-21T13:21:26.168 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.298+0000 7f1c8a7fc700 1 -- 172.21.15.97:0/4073227302 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c84024940 con 0x7f1c94106170 2024-03-21T13:21:26.168 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.298+0000 7f1c9b9c1700 1 -- 172.21.15.97:0/4073227302 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c9419d520 con 0x7f1c94106170 2024-03-21T13:21:26.168 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.299+0000 7f1c8a7fc700 1 -- 172.21.15.97:0/4073227302 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 9) v1 ==== 46272+0+0 (secure 0 0 0) 0x7f1c84022020 con 0x7f1c94106170 2024-03-21T13:21:26.168 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.299+0000 7f1c9b9c1700 1 -- 172.21.15.97:0/4073227302 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c940637c0 con 0x7f1c94106170 2024-03-21T13:21:26.168 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.300+0000 7f1c8a7fc700 1 --2- 172.21.15.97:0/4073227302 >> v2:172.21.15.97:6800/4140723216 conn(0x7f1c80039330 0x7f1c8003b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:26.169 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.300+0000 7f1c8a7fc700 1 -- 172.21.15.97:0/4073227302 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (secure 0 0 0) 0x7f1c8404d690 con 0x7f1c94106170 2024-03-21T13:21:26.169 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.300+0000 7f1c98f5c700 1 -- 172.21.15.97:0/4073227302 >> v2:172.21.15.97:6800/4140723216 conn(0x7f1c80039330 msgr2=0x7f1c8003b7f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/4140723216 2024-03-21T13:21:26.169 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.300+0000 7f1c98f5c700 1 --2- 172.21.15.97:0/4073227302 >> v2:172.21.15.97:6800/4140723216 conn(0x7f1c80039330 0x7f1c8003b7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T13:21:26.169 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.309+0000 7f1c8a7fc700 1 -- 172.21.15.97:0/4073227302 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1c8400f2e0 con 0x7f1c94106170 2024-03-21T13:21:26.169 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.501+0000 7f1c98f5c700 1 -- 172.21.15.97:0/4073227302 >> v2:172.21.15.97:6800/4140723216 conn(0x7f1c80039330 msgr2=0x7f1c8003b7f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/4140723216 2024-03-21T13:21:26.169 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.501+0000 7f1c98f5c700 1 --2- 172.21.15.97:0/4073227302 >> v2:172.21.15.97:6800/4140723216 conn(0x7f1c80039330 0x7f1c8003b7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-21T13:21:26.169 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.577+0000 7f1c9b9c1700 1 -- 172.21.15.97:0/4073227302 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f1c9419d900 con 0x7f1c94106170 2024-03-21T13:21:26.169 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.578+0000 7f1c8a7fc700 1 -- 172.21.15.97:0/4073227302 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f1c8404d3f0 con 0x7f1c94106170 2024-03-21T13:21:26.169 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.582+0000 7f1c9b9c1700 1 -- 172.21.15.97:0/4073227302 >> v2:172.21.15.97:6800/4140723216 conn(0x7f1c80039330 msgr2=0x7f1c8003b7f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T13:21:26.169 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.582+0000 7f1c9b9c1700 1 --2- 172.21.15.97:0/4073227302 >> v2:172.21.15.97:6800/4140723216 conn(0x7f1c80039330 0x7f1c8003b7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:26.169 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.582+0000 7f1c9b9c1700 1 -- 172.21.15.97:0/4073227302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c94106170 msgr2=0x7f1c9419a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:26.169 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.582+0000 7f1c9b9c1700 1 --2- 172.21.15.97:0/4073227302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c94106170 0x7f1c9419a060 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1c84009870 tx=0x7f1c84011ac0 comp rx=0 tx=0).stop 2024-03-21T13:21:26.169 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.582+0000 7f1c9b9c1700 1 -- 172.21.15.97:0/4073227302 shutdown_connections 2024-03-21T13:21:26.169 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.582+0000 7f1c9b9c1700 1 --2- 172.21.15.97:0/4073227302 >> v2:172.21.15.97:6800/4140723216 conn(0x7f1c80039330 0x7f1c8003b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:26.170 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.582+0000 7f1c9b9c1700 1 --2- 172.21.15.97:0/4073227302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c94106170 0x7f1c9419a060 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:26.170 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.582+0000 7f1c9b9c1700 1 -- 172.21.15.97:0/4073227302 >> 172.21.15.97:0/4073227302 conn(0x7f1c941016d0 msgr2=0x7f1c94102160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:21:26.170 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.582+0000 7f1c9b9c1700 1 -- 172.21.15.97:0/4073227302 shutdown_connections 2024-03-21T13:21:26.170 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:25.582+0000 7f1c9b9c1700 1 -- 172.21.15.97:0/4073227302 wait complete. 2024-03-21T13:21:26.170 INFO:teuthology.orchestra.run.smithi097.stdout:Waiting for the mgr to restart... 2024-03-21T13:21:26.170 INFO:teuthology.orchestra.run.smithi097.stdout:Waiting for mgr epoch 9... 2024-03-21T13:21:26.406 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:26 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4073227302' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-21T13:21:26.407 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:26 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:21:26.208+0000 7fcc75a66200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-21T13:21:26.407 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:26 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:21:26.319+0000 7fcc75a66200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-21T13:21:26.407 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:26 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:21:26.406+0000 7fcc75a66200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-21T13:21:26.685 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:26 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:21:26.502+0000 7fcc75a66200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-21T13:21:26.946 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:26 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:21:26.685+0000 7fcc75a66200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-21T13:21:28.053 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:27 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:21:27.642+0000 7fcc75a66200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-21T13:21:28.053 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:27 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:21:27.773+0000 7fcc75a66200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-21T13:21:28.392 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:28 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:21:28.108+0000 7fcc75a66200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-21T13:21:28.392 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:28 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:21:28.197+0000 7fcc75a66200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-21T13:21:28.392 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:28 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:21:28.391+0000 7fcc75a66200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-21T13:21:28.803 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:28 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:21:28.577+0000 7fcc75a66200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-21T13:21:29.117 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:28 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:21:28.861+0000 7fcc75a66200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-21T13:21:29.117 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:29 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:21:29.116+0000 7fcc75a66200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-21T13:21:29.553 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:29 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:21:29.353+0000 7fcc75a66200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-21T13:21:30.553 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:30 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:21:30.148+0000 7fcc75a66200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-21T13:21:31.053 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:30 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:21:30.740+0000 7fcc75a66200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-21T13:21:31.053 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:30 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:21:30.839+0000 7fcc75a66200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-21T13:21:31.553 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:21:31 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:21:31.056+0000 7fcc75a66200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-21T13:21:31.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:31 smithi097 ceph-mon[30040]: Active manager daemon y restarted 2024-03-21T13:21:31.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:31 smithi097 ceph-mon[30040]: Activating manager daemon y 2024-03-21T13:21:31.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:31 smithi097 ceph-mon[30040]: osdmap e3: 0 total, 0 up, 0 in 2024-03-21T13:21:31.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:31 smithi097 ceph-mon[30040]: mgrmap e10: y(active, starting, since 0.00334527s) 2024-03-21T13:21:31.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:31 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T13:21:31.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:31 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-21T13:21:31.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:31 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-21T13:21:31.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:31 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-21T13:21:31.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:31 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-21T13:21:31.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:31 smithi097 ceph-mon[30040]: Manager daemon y is now available 2024-03-21T13:21:31.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:31 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:21:32.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:32 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-21T13:21:32.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:32 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-21T13:21:32.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:32 smithi097 ceph-mon[30040]: mgrmap e11: y(active, since 1.00721s) 2024-03-21T13:21:32.641 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout { 2024-03-21T13:21:32.642 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-21T13:21:32.642 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-21T13:21:32.642 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout } 2024-03-21T13:21:32.642 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:26.739+0000 7fd90e01a700 1 Processor -- start 2024-03-21T13:21:32.642 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:26.739+0000 7fd90e01a700 1 -- start start 2024-03-21T13:21:32.642 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:26.740+0000 7fd90e01a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd908106c10 0x7fd908109000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:32.642 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:26.740+0000 7fd90e01a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9080796b0 con 0x7fd908106c10 2024-03-21T13:21:32.642 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:26.740+0000 7fd9077fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd908106c10 0x7fd908109000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:32.642 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:26.740+0000 7fd9077fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd908106c10 0x7fd908109000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55200/0 (socket says 172.21.15.97:55200) 2024-03-21T13:21:32.642 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:26.740+0000 7fd9077fe700 1 -- 172.21.15.97:0/2602897224 learned_addr learned my addr 172.21.15.97:0/2602897224 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:21:32.642 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:26.740+0000 7fd9077fe700 1 -- 172.21.15.97:0/2602897224 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd908109540 con 0x7fd908106c10 2024-03-21T13:21:32.642 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:26.740+0000 7fd9077fe700 1 --2- 172.21.15.97:0/2602897224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd908106c10 0x7fd908109000 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd8f80098a0 tx=0x7fd8f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=521b4926b2533372 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:32.643 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:26.740+0000 7fd9067fc700 1 -- 172.21.15.97:0/2602897224 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd8f8004160 con 0x7fd908106c10 2024-03-21T13:21:32.643 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:26.740+0000 7fd9067fc700 1 -- 172.21.15.97:0/2602897224 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fd8f8004300 con 0x7fd908106c10 2024-03-21T13:21:32.643 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:26.740+0000 7fd9067fc700 1 -- 172.21.15.97:0/2602897224 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd8f8011a00 con 0x7fd908106c10 2024-03-21T13:21:32.643 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:26.741+0000 7fd90e01a700 1 -- 172.21.15.97:0/2602897224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd908106c10 msgr2=0x7fd908109000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:32.643 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:26.741+0000 7fd90e01a700 1 --2- 172.21.15.97:0/2602897224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd908106c10 0x7fd908109000 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd8f80098a0 tx=0x7fd8f8009bb0 comp rx=0 tx=0).stop 2024-03-21T13:21:32.643 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:26.741+0000 7fd90e01a700 1 -- 172.21.15.97:0/2602897224 shutdown_connections 2024-03-21T13:21:32.643 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:26.741+0000 7fd90e01a700 1 --2- 172.21.15.97:0/2602897224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd908106c10 0x7fd908109000 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:32.643 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:26.741+0000 7fd90e01a700 1 -- 172.21.15.97:0/2602897224 >> 172.21.15.97:0/2602897224 conn(0x7fd9081005c0 msgr2=0x7fd908102a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:21:32.643 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:26.741+0000 7fd90e01a700 1 -- 172.21.15.97:0/2602897224 shutdown_connections 2024-03-21T13:21:32.643 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:26.741+0000 7fd90e01a700 1 -- 172.21.15.97:0/2602897224 wait complete. 2024-03-21T13:21:32.643 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:26.742+0000 7fd90e01a700 1 Processor -- start 2024-03-21T13:21:32.643 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:26.742+0000 7fd90e01a700 1 -- start start 2024-03-21T13:21:32.643 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:26.742+0000 7fd90e01a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd908106c10 0x7fd90819a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:32.643 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:26.742+0000 7fd90e01a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd90810a4f0 con 0x7fd908106c10 2024-03-21T13:21:32.643 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:26.742+0000 7fd9077fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd908106c10 0x7fd90819a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:32.644 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:26.742+0000 7fd9077fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd908106c10 0x7fd90819a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55202/0 (socket says 172.21.15.97:55202) 2024-03-21T13:21:32.644 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:26.742+0000 7fd9077fe700 1 -- 172.21.15.97:0/2297521742 learned_addr learned my addr 172.21.15.97:0/2297521742 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:21:32.644 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:26.742+0000 7fd9077fe700 1 -- 172.21.15.97:0/2297521742 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd90819a550 con 0x7fd908106c10 2024-03-21T13:21:32.644 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:26.743+0000 7fd9077fe700 1 --2- 172.21.15.97:0/2297521742 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd908106c10 0x7fd90819a010 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd8f8011a10 tx=0x7fd8f8011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:32.644 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:26.743+0000 7fd904ff9700 1 -- 172.21.15.97:0/2297521742 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd8f801a430 con 0x7fd908106c10 2024-03-21T13:21:32.644 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:26.743+0000 7fd90e01a700 1 -- 172.21.15.97:0/2297521742 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd90819a7e0 con 0x7fd908106c10 2024-03-21T13:21:32.644 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:26.743+0000 7fd904ff9700 1 -- 172.21.15.97:0/2297521742 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fd8f801aa80 con 0x7fd908106c10 2024-03-21T13:21:32.644 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:26.743+0000 7fd90e01a700 1 -- 172.21.15.97:0/2297521742 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd90819d4d0 con 0x7fd908106c10 2024-03-21T13:21:32.644 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:26.743+0000 7fd904ff9700 1 -- 172.21.15.97:0/2297521742 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd8f80249d0 con 0x7fd908106c10 2024-03-21T13:21:32.644 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:26.744+0000 7fd904ff9700 1 -- 172.21.15.97:0/2297521742 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 9) v1 ==== 46272+0+0 (secure 0 0 0) 0x7fd8f8022020 con 0x7fd908106c10 2024-03-21T13:21:32.644 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:26.744+0000 7fd904ff9700 1 --2- 172.21.15.97:0/2297521742 >> v2:172.21.15.97:6800/4140723216 conn(0x7fd8f0039140 0x7fd8f003b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:32.644 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:26.744+0000 7fd904ff9700 1 -- 172.21.15.97:0/2297521742 --> v2:172.21.15.97:6800/4140723216 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd8f003bcd0 con 0x7fd8f0039140 2024-03-21T13:21:32.644 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:26.744+0000 7fd904ff9700 1 -- 172.21.15.97:0/2297521742 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (secure 0 0 0) 0x7fd8f804d5e0 con 0x7fd908106c10 2024-03-21T13:21:32.644 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:26.744+0000 7fd906ffd700 1 -- 172.21.15.97:0/2297521742 >> v2:172.21.15.97:6800/4140723216 conn(0x7fd8f0039140 msgr2=0x7fd8f003b600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/4140723216 2024-03-21T13:21:32.645 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:26.744+0000 7fd906ffd700 1 --2- 172.21.15.97:0/2297521742 >> v2:172.21.15.97:6800/4140723216 conn(0x7fd8f0039140 0x7fd8f003b600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T13:21:32.645 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:26.945+0000 7fd906ffd700 1 -- 172.21.15.97:0/2297521742 >> v2:172.21.15.97:6800/4140723216 conn(0x7fd8f0039140 msgr2=0x7fd8f003b600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/4140723216 2024-03-21T13:21:32.645 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:26.945+0000 7fd906ffd700 1 --2- 172.21.15.97:0/2297521742 >> v2:172.21.15.97:6800/4140723216 conn(0x7fd8f0039140 0x7fd8f003b600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-21T13:21:32.645 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:27.346+0000 7fd906ffd700 1 -- 172.21.15.97:0/2297521742 >> v2:172.21.15.97:6800/4140723216 conn(0x7fd8f0039140 msgr2=0x7fd8f003b600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/4140723216 2024-03-21T13:21:32.645 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:27.346+0000 7fd906ffd700 1 --2- 172.21.15.97:0/2297521742 >> v2:172.21.15.97:6800/4140723216 conn(0x7fd8f0039140 0x7fd8f003b600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-21T13:21:32.645 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:28.147+0000 7fd906ffd700 1 -- 172.21.15.97:0/2297521742 >> v2:172.21.15.97:6800/4140723216 conn(0x7fd8f0039140 msgr2=0x7fd8f003b600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/4140723216 2024-03-21T13:21:32.645 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:28.147+0000 7fd906ffd700 1 --2- 172.21.15.97:0/2297521742 >> v2:172.21.15.97:6800/4140723216 conn(0x7fd8f0039140 0x7fd8f003b600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-21T13:21:32.645 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:29.748+0000 7fd906ffd700 1 -- 172.21.15.97:0/2297521742 >> v2:172.21.15.97:6800/4140723216 conn(0x7fd8f0039140 msgr2=0x7fd8f003b600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/4140723216 2024-03-21T13:21:32.645 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:29.748+0000 7fd906ffd700 1 --2- 172.21.15.97:0/2297521742 >> v2:172.21.15.97:6800/4140723216 conn(0x7fd8f0039140 0x7fd8f003b600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-21T13:21:32.645 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:31.071+0000 7fd904ff9700 1 -- 172.21.15.97:0/2297521742 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mgrmap(e 10) v1 ==== 46074+0+0 (secure 0 0 0) 0x7fd8f802d3f0 con 0x7fd908106c10 2024-03-21T13:21:32.645 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:31.071+0000 7fd904ff9700 1 -- 172.21.15.97:0/2297521742 >> v2:172.21.15.97:6800/4140723216 conn(0x7fd8f0039140 msgr2=0x7fd8f003b600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T13:21:32.645 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:31.071+0000 7fd904ff9700 1 --2- 172.21.15.97:0/2297521742 >> v2:172.21.15.97:6800/4140723216 conn(0x7fd8f0039140 0x7fd8f003b600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:32.645 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:32.073+0000 7fd904ff9700 1 -- 172.21.15.97:0/2297521742 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mgrmap(e 11) v1 ==== 46166+0+0 (secure 0 0 0) 0x7fd8f804f3a0 con 0x7fd908106c10 2024-03-21T13:21:32.645 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:32.073+0000 7fd904ff9700 1 --2- 172.21.15.97:0/2297521742 >> v2:172.21.15.97:6800/1943212787 conn(0x7fd8f003cb50 0x7fd8f003ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:32.645 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:32.073+0000 7fd904ff9700 1 -- 172.21.15.97:0/2297521742 --> v2:172.21.15.97:6800/1943212787 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd8f8009530 con 0x7fd8f003cb50 2024-03-21T13:21:32.646 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:32.074+0000 7fd906ffd700 1 --2- 172.21.15.97:0/2297521742 >> v2:172.21.15.97:6800/1943212787 conn(0x7fd8f003cb50 0x7fd8f003ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:32.646 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:32.074+0000 7fd906ffd700 1 --2- 172.21.15.97:0/2297521742 >> v2:172.21.15.97:6800/1943212787 conn(0x7fd8f003cb50 0x7fd8f003ef40 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc003a10 tx=0x7fd8fc0092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:32.646 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:32.075+0000 7fd904ff9700 1 -- 172.21.15.97:0/2297521742 <== mgr.14152 v2:172.21.15.97:6800/1943212787 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7fd8f8009530 con 0x7fd8f003cb50 2024-03-21T13:21:32.646 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:32.082+0000 7fd90e01a700 1 -- 172.21.15.97:0/2297521742 --> v2:172.21.15.97:6800/1943212787 -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fd9080637c0 con 0x7fd8f003cb50 2024-03-21T13:21:32.646 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:32.082+0000 7fd904ff9700 1 -- 172.21.15.97:0/2297521742 <== mgr.14152 v2:172.21.15.97:6800/1943212787 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fd9080637c0 con 0x7fd8f003cb50 2024-03-21T13:21:32.646 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:32.082+0000 7fd90e01a700 1 -- 172.21.15.97:0/2297521742 >> v2:172.21.15.97:6800/1943212787 conn(0x7fd8f003cb50 msgr2=0x7fd8f003ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:32.646 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:32.082+0000 7fd90e01a700 1 --2- 172.21.15.97:0/2297521742 >> v2:172.21.15.97:6800/1943212787 conn(0x7fd8f003cb50 0x7fd8f003ef40 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc003a10 tx=0x7fd8fc0092b0 comp rx=0 tx=0).stop 2024-03-21T13:21:32.646 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:32.082+0000 7fd90e01a700 1 -- 172.21.15.97:0/2297521742 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd908106c10 msgr2=0x7fd90819a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:32.646 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:32.082+0000 7fd90e01a700 1 --2- 172.21.15.97:0/2297521742 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd908106c10 0x7fd90819a010 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd8f8011a10 tx=0x7fd8f8011af0 comp rx=0 tx=0).stop 2024-03-21T13:21:32.646 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:32.082+0000 7fd90e01a700 1 -- 172.21.15.97:0/2297521742 shutdown_connections 2024-03-21T13:21:32.646 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:32.082+0000 7fd90e01a700 1 --2- 172.21.15.97:0/2297521742 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd908106c10 0x7fd90819a010 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:32.646 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:32.082+0000 7fd90e01a700 1 --2- 172.21.15.97:0/2297521742 >> v2:172.21.15.97:6800/1943212787 conn(0x7fd8f003cb50 0x7fd8f003ef40 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:32.646 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:32.082+0000 7fd90e01a700 1 --2- 172.21.15.97:0/2297521742 >> v2:172.21.15.97:6800/4140723216 conn(0x7fd8f0039140 0x7fd8f003b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:32.646 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:32.082+0000 7fd90e01a700 1 -- 172.21.15.97:0/2297521742 >> 172.21.15.97:0/2297521742 conn(0x7fd9081005c0 msgr2=0x7fd908100fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:21:32.647 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:32.082+0000 7fd90e01a700 1 -- 172.21.15.97:0/2297521742 shutdown_connections 2024-03-21T13:21:32.647 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:32.082+0000 7fd90e01a700 1 -- 172.21.15.97:0/2297521742 wait complete. 2024-03-21T13:21:32.647 INFO:teuthology.orchestra.run.smithi097.stdout:mgr epoch 9 is available 2024-03-21T13:21:32.647 INFO:teuthology.orchestra.run.smithi097.stdout:Generating a dashboard self-signed certificate... 2024-03-21T13:21:33.429 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:33 smithi097 ceph-mon[30040]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-21T13:21:33.429 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:33 smithi097 ceph-mon[30040]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-21T13:21:34.006 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-21T13:21:34.006 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.076+0000 7fd107311700 1 Processor -- start 2024-03-21T13:21:34.006 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.076+0000 7fd107311700 1 -- start start 2024-03-21T13:21:34.006 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.076+0000 7fd107311700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1001050a0 0x7fd1001054c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:34.006 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.076+0000 7fd107311700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd100105a90 con 0x7fd1001050a0 2024-03-21T13:21:34.006 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.076+0000 7fd1050ad700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1001050a0 0x7fd1001054c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:34.006 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.076+0000 7fd1050ad700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1001050a0 0x7fd1001054c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50132/0 (socket says 172.21.15.97:50132) 2024-03-21T13:21:34.006 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.076+0000 7fd1050ad700 1 -- 172.21.15.97:0/1387325664 learned_addr learned my addr 172.21.15.97:0/1387325664 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:21:34.006 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.077+0000 7fd1050ad700 1 -- 172.21.15.97:0/1387325664 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1001062a0 con 0x7fd1001050a0 2024-03-21T13:21:34.007 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.077+0000 7fd1050ad700 1 --2- 172.21.15.97:0/1387325664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1001050a0 0x7fd1001054c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc0098a0 tx=0x7fd0fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a86a9fc9be9906b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:34.007 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.077+0000 7fd0f7fff700 1 -- 172.21.15.97:0/1387325664 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd0fc004160 con 0x7fd1001050a0 2024-03-21T13:21:34.007 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.077+0000 7fd0f7fff700 1 -- 172.21.15.97:0/1387325664 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fd0fc004300 con 0x7fd1001050a0 2024-03-21T13:21:34.007 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.077+0000 7fd0f7fff700 1 -- 172.21.15.97:0/1387325664 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd0fc011a00 con 0x7fd1001050a0 2024-03-21T13:21:34.007 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.077+0000 7fd107311700 1 -- 172.21.15.97:0/1387325664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1001050a0 msgr2=0x7fd1001054c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:34.007 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.077+0000 7fd107311700 1 --2- 172.21.15.97:0/1387325664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1001050a0 0x7fd1001054c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc0098a0 tx=0x7fd0fc009bb0 comp rx=0 tx=0).stop 2024-03-21T13:21:34.007 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.078+0000 7fd107311700 1 -- 172.21.15.97:0/1387325664 shutdown_connections 2024-03-21T13:21:34.007 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.078+0000 7fd107311700 1 --2- 172.21.15.97:0/1387325664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1001050a0 0x7fd1001054c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:34.007 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.078+0000 7fd107311700 1 -- 172.21.15.97:0/1387325664 >> 172.21.15.97:0/1387325664 conn(0x7fd100100660 msgr2=0x7fd100102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:21:34.007 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.078+0000 7fd107311700 1 -- 172.21.15.97:0/1387325664 shutdown_connections 2024-03-21T13:21:34.007 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.078+0000 7fd107311700 1 -- 172.21.15.97:0/1387325664 wait complete. 2024-03-21T13:21:34.007 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.078+0000 7fd107311700 1 Processor -- start 2024-03-21T13:21:34.007 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.078+0000 7fd107311700 1 -- start start 2024-03-21T13:21:34.007 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.078+0000 7fd107311700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1001050a0 0x7fd100198f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:34.008 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.078+0000 7fd107311700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd100109650 con 0x7fd1001050a0 2024-03-21T13:21:34.008 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.079+0000 7fd1050ad700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1001050a0 0x7fd100198f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:34.008 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.079+0000 7fd1050ad700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1001050a0 0x7fd100198f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50148/0 (socket says 172.21.15.97:50148) 2024-03-21T13:21:34.008 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.079+0000 7fd1050ad700 1 -- 172.21.15.97:0/1159305508 learned_addr learned my addr 172.21.15.97:0/1159305508 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:21:34.008 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.079+0000 7fd1050ad700 1 -- 172.21.15.97:0/1159305508 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1001994a0 con 0x7fd1001050a0 2024-03-21T13:21:34.008 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.079+0000 7fd1050ad700 1 --2- 172.21.15.97:0/1159305508 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1001050a0 0x7fd100198f60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc009fb0 tx=0x7fd0fc011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:34.008 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.079+0000 7fd0f67fc700 1 -- 172.21.15.97:0/1159305508 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd0fc01a430 con 0x7fd1001050a0 2024-03-21T13:21:34.008 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.079+0000 7fd107311700 1 -- 172.21.15.97:0/1159305508 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd100199730 con 0x7fd1001050a0 2024-03-21T13:21:34.008 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.079+0000 7fd0f67fc700 1 -- 172.21.15.97:0/1159305508 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fd0fc01aa80 con 0x7fd1001050a0 2024-03-21T13:21:34.008 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.079+0000 7fd0f67fc700 1 -- 172.21.15.97:0/1159305508 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd0fc0249b0 con 0x7fd1001050a0 2024-03-21T13:21:34.008 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.079+0000 7fd107311700 1 -- 172.21.15.97:0/1159305508 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1000790c0 con 0x7fd1001050a0 2024-03-21T13:21:34.008 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.080+0000 7fd0f67fc700 1 -- 172.21.15.97:0/1159305508 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 11) v1 ==== 46166+0+0 (secure 0 0 0) 0x7fd0fc022020 con 0x7fd1001050a0 2024-03-21T13:21:34.008 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.080+0000 7fd107311700 1 -- 172.21.15.97:0/1159305508 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1000637c0 con 0x7fd1001050a0 2024-03-21T13:21:34.008 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.080+0000 7fd0f67fc700 1 --2- 172.21.15.97:0/1159305508 >> v2:172.21.15.97:6800/1943212787 conn(0x7fd0ec03b240 0x7fd0ec03d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:34.008 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.080+0000 7fd0f67fc700 1 -- 172.21.15.97:0/1159305508 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (secure 0 0 0) 0x7fd0fc04d830 con 0x7fd1001050a0 2024-03-21T13:21:34.009 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.080+0000 7fd1048ac700 1 --2- 172.21.15.97:0/1159305508 >> v2:172.21.15.97:6800/1943212787 conn(0x7fd0ec03b240 0x7fd0ec03d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:34.009 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.081+0000 7fd1048ac700 1 --2- 172.21.15.97:0/1159305508 >> v2:172.21.15.97:6800/1943212787 conn(0x7fd0ec03b240 0x7fd0ec03d700 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0006fd0 tx=0x7fd0f0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:34.009 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.085+0000 7fd0f67fc700 1 -- 172.21.15.97:0/1159305508 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd0fc02ad60 con 0x7fd1001050a0 2024-03-21T13:21:34.009 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.123+0000 7fd0f67fc700 1 -- 172.21.15.97:0/1159305508 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mgrmap(e 12) v1 ==== 46272+0+0 (secure 0 0 0) 0x7fd0fc02d3f0 con 0x7fd1001050a0 2024-03-21T13:21:34.009 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.290+0000 7fd107311700 1 -- 172.21.15.97:0/1159305508 --> v2:172.21.15.97:6800/1943212787 -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fd1001017c0 con 0x7fd0ec03b240 2024-03-21T13:21:34.009 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.427+0000 7fd0f67fc700 1 -- 172.21.15.97:0/1159305508 <== mgr.14152 v2:172.21.15.97:6800/1943212787 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fd1001017c0 con 0x7fd0ec03b240 2024-03-21T13:21:34.009 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.431+0000 7fd107311700 1 -- 172.21.15.97:0/1159305508 >> v2:172.21.15.97:6800/1943212787 conn(0x7fd0ec03b240 msgr2=0x7fd0ec03d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:34.009 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.431+0000 7fd107311700 1 --2- 172.21.15.97:0/1159305508 >> v2:172.21.15.97:6800/1943212787 conn(0x7fd0ec03b240 0x7fd0ec03d700 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0006fd0 tx=0x7fd0f0006e40 comp rx=0 tx=0).stop 2024-03-21T13:21:34.009 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.431+0000 7fd107311700 1 -- 172.21.15.97:0/1159305508 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1001050a0 msgr2=0x7fd100198f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:34.009 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.431+0000 7fd107311700 1 --2- 172.21.15.97:0/1159305508 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1001050a0 0x7fd100198f60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc009fb0 tx=0x7fd0fc011ac0 comp rx=0 tx=0).stop 2024-03-21T13:21:34.009 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.431+0000 7fd107311700 1 -- 172.21.15.97:0/1159305508 shutdown_connections 2024-03-21T13:21:34.009 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.431+0000 7fd107311700 1 --2- 172.21.15.97:0/1159305508 >> v2:172.21.15.97:6800/1943212787 conn(0x7fd0ec03b240 0x7fd0ec03d700 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:34.009 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.431+0000 7fd107311700 1 --2- 172.21.15.97:0/1159305508 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1001050a0 0x7fd100198f60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:34.009 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.431+0000 7fd107311700 1 -- 172.21.15.97:0/1159305508 >> 172.21.15.97:0/1159305508 conn(0x7fd100100660 msgr2=0x7fd1001010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:21:34.010 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.431+0000 7fd107311700 1 -- 172.21.15.97:0/1159305508 shutdown_connections 2024-03-21T13:21:34.010 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:33.431+0000 7fd107311700 1 -- 172.21.15.97:0/1159305508 wait complete. 2024-03-21T13:21:34.010 INFO:teuthology.orchestra.run.smithi097.stdout:Creating initial admin user... 2024-03-21T13:21:34.272 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:34 smithi097 ceph-mon[30040]: [21/Mar/2024:13:21:32] ENGINE Bus STARTING 2024-03-21T13:21:34.272 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:34 smithi097 ceph-mon[30040]: [21/Mar/2024:13:21:32] ENGINE Serving on https://172.21.15.97:7150 2024-03-21T13:21:34.272 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:34 smithi097 ceph-mon[30040]: [21/Mar/2024:13:21:32] ENGINE Serving on http://172.21.15.97:8765 2024-03-21T13:21:34.272 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:34 smithi097 ceph-mon[30040]: [21/Mar/2024:13:21:32] ENGINE Bus STARTED 2024-03-21T13:21:34.273 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:34 smithi097 ceph-mon[30040]: mgrmap e12: y(active, since 2s) 2024-03-21T13:21:34.273 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:34 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:21:34.273 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:34 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:21:34.273 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:34 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:21:35.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:35 smithi097 ceph-mon[30040]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:21:35.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:35 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:21:35.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:35 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:21:35.563 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$8PmtuKPurKnYkSFFLSVWmuC9fddvTZENP6FhQw5O6rWNbQpOINDV.", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1711027295, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-21T13:21:35.563 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:34.548+0000 7fdd16f2e700 1 Processor -- start 2024-03-21T13:21:35.563 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:34.548+0000 7fdd16f2e700 1 -- start start 2024-03-21T13:21:35.563 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:34.548+0000 7fdd16f2e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd10106170 0x7fdd10106590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:35.563 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:34.548+0000 7fdd16f2e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd10106b60 con 0x7fdd10106170 2024-03-21T13:21:35.563 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:34.549+0000 7fdd14cca700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd10106170 0x7fdd10106590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:35.563 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:34.549+0000 7fdd14cca700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd10106170 0x7fdd10106590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50162/0 (socket says 172.21.15.97:50162) 2024-03-21T13:21:35.564 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:34.549+0000 7fdd14cca700 1 -- 172.21.15.97:0/259144181 learned_addr learned my addr 172.21.15.97:0/259144181 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:21:35.564 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:34.549+0000 7fdd14cca700 1 -- 172.21.15.97:0/259144181 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd10107370 con 0x7fdd10106170 2024-03-21T13:21:35.564 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:34.549+0000 7fdd14cca700 1 --2- 172.21.15.97:0/259144181 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd10106170 0x7fdd10106590 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdd040098a0 tx=0x7fdd04009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3296943ead638cc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:35.564 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:34.549+0000 7fdd0f7fe700 1 -- 172.21.15.97:0/259144181 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd04004160 con 0x7fdd10106170 2024-03-21T13:21:35.564 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:34.549+0000 7fdd0f7fe700 1 -- 172.21.15.97:0/259144181 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fdd04004300 con 0x7fdd10106170 2024-03-21T13:21:35.564 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:34.549+0000 7fdd0f7fe700 1 -- 172.21.15.97:0/259144181 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd04011a00 con 0x7fdd10106170 2024-03-21T13:21:35.564 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:34.550+0000 7fdd16f2e700 1 -- 172.21.15.97:0/259144181 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd10106170 msgr2=0x7fdd10106590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:35.564 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:34.550+0000 7fdd16f2e700 1 --2- 172.21.15.97:0/259144181 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd10106170 0x7fdd10106590 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdd040098a0 tx=0x7fdd04009bb0 comp rx=0 tx=0).stop 2024-03-21T13:21:35.564 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:34.550+0000 7fdd16f2e700 1 -- 172.21.15.97:0/259144181 shutdown_connections 2024-03-21T13:21:35.564 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:34.550+0000 7fdd16f2e700 1 --2- 172.21.15.97:0/259144181 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd10106170 0x7fdd10106590 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:35.564 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:34.550+0000 7fdd16f2e700 1 -- 172.21.15.97:0/259144181 >> 172.21.15.97:0/259144181 conn(0x7fdd101016d0 msgr2=0x7fdd10103b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:21:35.564 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:34.550+0000 7fdd16f2e700 1 -- 172.21.15.97:0/259144181 shutdown_connections 2024-03-21T13:21:35.564 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:34.550+0000 7fdd16f2e700 1 -- 172.21.15.97:0/259144181 wait complete. 2024-03-21T13:21:35.564 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:34.550+0000 7fdd16f2e700 1 Processor -- start 2024-03-21T13:21:35.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:34.551+0000 7fdd16f2e700 1 -- start start 2024-03-21T13:21:35.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:34.551+0000 7fdd16f2e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd10106170 0x7fdd1007bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:35.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:34.551+0000 7fdd16f2e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1010a720 con 0x7fdd10106170 2024-03-21T13:21:35.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:34.551+0000 7fdd14cca700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd10106170 0x7fdd1007bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:35.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:34.551+0000 7fdd14cca700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd10106170 0x7fdd1007bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50166/0 (socket says 172.21.15.97:50166) 2024-03-21T13:21:35.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:34.551+0000 7fdd14cca700 1 -- 172.21.15.97:0/3963913527 learned_addr learned my addr 172.21.15.97:0/3963913527 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:21:35.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:34.551+0000 7fdd14cca700 1 -- 172.21.15.97:0/3963913527 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd1007c1a0 con 0x7fdd10106170 2024-03-21T13:21:35.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:34.551+0000 7fdd14cca700 1 --2- 172.21.15.97:0/3963913527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd10106170 0x7fdd1007bc60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdd04009fb0 tx=0x7fdd04011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:35.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:34.551+0000 7fdd0dffb700 1 -- 172.21.15.97:0/3963913527 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd0401a430 con 0x7fdd10106170 2024-03-21T13:21:35.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:34.551+0000 7fdd0dffb700 1 -- 172.21.15.97:0/3963913527 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fdd0401aa80 con 0x7fdd10106170 2024-03-21T13:21:35.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:34.551+0000 7fdd16f2e700 1 -- 172.21.15.97:0/3963913527 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd1007a390 con 0x7fdd10106170 2024-03-21T13:21:35.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:34.551+0000 7fdd0dffb700 1 -- 172.21.15.97:0/3963913527 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd040249b0 con 0x7fdd10106170 2024-03-21T13:21:35.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:34.552+0000 7fdd16f2e700 1 -- 172.21.15.97:0/3963913527 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd1007a8d0 con 0x7fdd10106170 2024-03-21T13:21:35.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:34.553+0000 7fdd0dffb700 1 -- 172.21.15.97:0/3963913527 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 46272+0+0 (secure 0 0 0) 0x7fdd04022020 con 0x7fdd10106170 2024-03-21T13:21:35.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:34.553+0000 7fdd0dffb700 1 --2- 172.21.15.97:0/3963913527 >> v2:172.21.15.97:6800/1943212787 conn(0x7fdd00039140 0x7fdd0003b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:35.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:34.553+0000 7fdd16f2e700 1 -- 172.21.15.97:0/3963913527 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd100637c0 con 0x7fdd10106170 2024-03-21T13:21:35.566 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:34.553+0000 7fdd0dffb700 1 -- 172.21.15.97:0/3963913527 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (secure 0 0 0) 0x7fdd0404d960 con 0x7fdd10106170 2024-03-21T13:21:35.566 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:34.553+0000 7fdd0ffff700 1 --2- 172.21.15.97:0/3963913527 >> v2:172.21.15.97:6800/1943212787 conn(0x7fdd00039140 0x7fdd0003b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:35.566 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:34.553+0000 7fdd0ffff700 1 --2- 172.21.15.97:0/3963913527 >> v2:172.21.15.97:6800/1943212787 conn(0x7fdd00039140 0x7fdd0003b600 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc006fd0 tx=0x7fdcfc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:35.566 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:34.558+0000 7fdd0dffb700 1 -- 172.21.15.97:0/3963913527 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdd040172b0 con 0x7fdd10106170 2024-03-21T13:21:35.566 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:34.789+0000 7fdd16f2e700 1 -- 172.21.15.97:0/3963913527 --> v2:172.21.15.97:6800/1943212787 -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fdd1007afc0 con 0x7fdd00039140 2024-03-21T13:21:35.566 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:35.017+0000 7fdd0dffb700 1 -- 172.21.15.97:0/3963913527 <== mgr.14152 v2:172.21.15.97:6800/1943212787 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7fdd1007afc0 con 0x7fdd00039140 2024-03-21T13:21:35.566 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:35.020+0000 7fdd16f2e700 1 -- 172.21.15.97:0/3963913527 >> v2:172.21.15.97:6800/1943212787 conn(0x7fdd00039140 msgr2=0x7fdd0003b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:35.566 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:35.020+0000 7fdd16f2e700 1 --2- 172.21.15.97:0/3963913527 >> v2:172.21.15.97:6800/1943212787 conn(0x7fdd00039140 0x7fdd0003b600 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc006fd0 tx=0x7fdcfc006e40 comp rx=0 tx=0).stop 2024-03-21T13:21:35.566 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:35.020+0000 7fdd16f2e700 1 -- 172.21.15.97:0/3963913527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd10106170 msgr2=0x7fdd1007bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:35.566 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:35.020+0000 7fdd16f2e700 1 --2- 172.21.15.97:0/3963913527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd10106170 0x7fdd1007bc60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdd04009fb0 tx=0x7fdd04011ac0 comp rx=0 tx=0).stop 2024-03-21T13:21:35.566 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:35.020+0000 7fdd16f2e700 1 -- 172.21.15.97:0/3963913527 shutdown_connections 2024-03-21T13:21:35.566 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:35.020+0000 7fdd16f2e700 1 --2- 172.21.15.97:0/3963913527 >> v2:172.21.15.97:6800/1943212787 conn(0x7fdd00039140 0x7fdd0003b600 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:35.566 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:35.020+0000 7fdd16f2e700 1 --2- 172.21.15.97:0/3963913527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd10106170 0x7fdd1007bc60 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:35.567 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:35.020+0000 7fdd16f2e700 1 -- 172.21.15.97:0/3963913527 >> 172.21.15.97:0/3963913527 conn(0x7fdd101016d0 msgr2=0x7fdd10102150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:21:35.567 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:35.020+0000 7fdd16f2e700 1 -- 172.21.15.97:0/3963913527 shutdown_connections 2024-03-21T13:21:35.567 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:35.020+0000 7fdd16f2e700 1 -- 172.21.15.97:0/3963913527 wait complete. 2024-03-21T13:21:35.567 INFO:teuthology.orchestra.run.smithi097.stdout:Fetching dashboard port number... 2024-03-21T13:21:36.240 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:36 smithi097 ceph-mon[30040]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:21:36.779 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 8443 2024-03-21T13:21:36.780 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:36.021+0000 7f1901623700 1 Processor -- start 2024-03-21T13:21:36.780 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:36.021+0000 7f1901623700 1 -- start start 2024-03-21T13:21:36.780 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:36.021+0000 7f1901623700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18fc106640 0x7f18fc106a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:36.780 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:36.021+0000 7f1901623700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18fc107030 con 0x7f18fc106640 2024-03-21T13:21:36.780 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:36.021+0000 7f18faffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18fc106640 0x7f18fc106a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:36.780 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:36.021+0000 7f18faffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18fc106640 0x7f18fc106a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50182/0 (socket says 172.21.15.97:50182) 2024-03-21T13:21:36.780 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:36.021+0000 7f18faffd700 1 -- 172.21.15.97:0/1037269050 learned_addr learned my addr 172.21.15.97:0/1037269050 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:21:36.780 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:36.022+0000 7f18faffd700 1 -- 172.21.15.97:0/1037269050 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18fc105250 con 0x7f18fc106640 2024-03-21T13:21:36.780 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:36.022+0000 7f18faffd700 1 --2- 172.21.15.97:0/1037269050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18fc106640 0x7f18fc106a60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f18ec0098a0 tx=0x7f18ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53b3d47db7b942e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:36.780 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:36.022+0000 7f18f9ffb700 1 -- 172.21.15.97:0/1037269050 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f18ec004160 con 0x7f18fc106640 2024-03-21T13:21:36.780 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:36.022+0000 7f18f9ffb700 1 -- 172.21.15.97:0/1037269050 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f18ec004300 con 0x7f18fc106640 2024-03-21T13:21:36.781 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:36.022+0000 7f18f9ffb700 1 -- 172.21.15.97:0/1037269050 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f18ec011a00 con 0x7f18fc106640 2024-03-21T13:21:36.781 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:36.023+0000 7f1901623700 1 -- 172.21.15.97:0/1037269050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18fc106640 msgr2=0x7f18fc106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:36.781 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:36.023+0000 7f1901623700 1 --2- 172.21.15.97:0/1037269050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18fc106640 0x7f18fc106a60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f18ec0098a0 tx=0x7f18ec009bb0 comp rx=0 tx=0).stop 2024-03-21T13:21:36.781 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:36.023+0000 7f1901623700 1 -- 172.21.15.97:0/1037269050 shutdown_connections 2024-03-21T13:21:36.781 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:36.023+0000 7f1901623700 1 --2- 172.21.15.97:0/1037269050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18fc106640 0x7f18fc106a60 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:36.781 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:36.023+0000 7f1901623700 1 -- 172.21.15.97:0/1037269050 >> 172.21.15.97:0/1037269050 conn(0x7f18fc0ffd60 msgr2=0x7f18fc1021c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:21:36.781 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:36.023+0000 7f1901623700 1 -- 172.21.15.97:0/1037269050 shutdown_connections 2024-03-21T13:21:36.781 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:36.023+0000 7f1901623700 1 -- 172.21.15.97:0/1037269050 wait complete. 2024-03-21T13:21:36.781 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:36.023+0000 7f1901623700 1 Processor -- start 2024-03-21T13:21:36.781 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:36.024+0000 7f1901623700 1 -- start start 2024-03-21T13:21:36.781 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:36.024+0000 7f1901623700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18fc106640 0x7f18fc19a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:36.781 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:36.024+0000 7f1901623700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18fc109610 con 0x7f18fc106640 2024-03-21T13:21:36.781 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:36.024+0000 7f18faffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18fc106640 0x7f18fc19a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:36.781 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:36.024+0000 7f18faffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18fc106640 0x7f18fc19a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50198/0 (socket says 172.21.15.97:50198) 2024-03-21T13:21:36.782 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:36.024+0000 7f18faffd700 1 -- 172.21.15.97:0/4139786118 learned_addr learned my addr 172.21.15.97:0/4139786118 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:21:36.782 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:36.024+0000 7f18faffd700 1 -- 172.21.15.97:0/4139786118 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18fc19a5d0 con 0x7f18fc106640 2024-03-21T13:21:36.782 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:36.025+0000 7f18faffd700 1 --2- 172.21.15.97:0/4139786118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18fc106640 0x7f18fc19a090 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f18ec009870 tx=0x7f18ec011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:36.782 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:36.025+0000 7f18e3fff700 1 -- 172.21.15.97:0/4139786118 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f18ec003af0 con 0x7f18fc106640 2024-03-21T13:21:36.782 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:36.025+0000 7f18e3fff700 1 -- 172.21.15.97:0/4139786118 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f18ec01a8e0 con 0x7f18fc106640 2024-03-21T13:21:36.782 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:36.025+0000 7f1901623700 1 -- 172.21.15.97:0/4139786118 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18fc19a860 con 0x7f18fc106640 2024-03-21T13:21:36.782 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:36.025+0000 7f18e3fff700 1 -- 172.21.15.97:0/4139786118 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f18ec024940 con 0x7f18fc106640 2024-03-21T13:21:36.782 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:36.025+0000 7f1901623700 1 -- 172.21.15.97:0/4139786118 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18fc19d550 con 0x7f18fc106640 2024-03-21T13:21:36.782 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:36.026+0000 7f18e3fff700 1 -- 172.21.15.97:0/4139786118 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 46272+0+0 (secure 0 0 0) 0x7f18ec022020 con 0x7f18fc106640 2024-03-21T13:21:36.782 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:36.026+0000 7f18e3fff700 1 --2- 172.21.15.97:0/4139786118 >> v2:172.21.15.97:6800/1943212787 conn(0x7f18e4039330 0x7f18e403b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:36.782 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:36.026+0000 7f1901623700 1 -- 172.21.15.97:0/4139786118 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18fc0637c0 con 0x7f18fc106640 2024-03-21T13:21:36.782 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:36.026+0000 7f18e3fff700 1 -- 172.21.15.97:0/4139786118 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f18ec04dae0 con 0x7f18fc106640 2024-03-21T13:21:36.782 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:36.026+0000 7f18fa7fc700 1 --2- 172.21.15.97:0/4139786118 >> v2:172.21.15.97:6800/1943212787 conn(0x7f18e4039330 0x7f18e403b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:36.783 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:36.027+0000 7f18fa7fc700 1 --2- 172.21.15.97:0/4139786118 >> v2:172.21.15.97:6800/1943212787 conn(0x7f18e4039330 0x7f18e403b7f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f18f0006fd0 tx=0x7f18f0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:36.783 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:36.031+0000 7f18e3fff700 1 -- 172.21.15.97:0/4139786118 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f18ec0172b0 con 0x7f18fc106640 2024-03-21T13:21:36.783 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:36.239+0000 7f1901623700 1 -- 172.21.15.97:0/4139786118 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f18fc02dad0 con 0x7f18fc106640 2024-03-21T13:21:36.783 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:36.240+0000 7f18e3fff700 1 -- 172.21.15.97:0/4139786118 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f18ec01ad90 con 0x7f18fc106640 2024-03-21T13:21:36.783 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:36.242+0000 7f1901623700 1 -- 172.21.15.97:0/4139786118 >> v2:172.21.15.97:6800/1943212787 conn(0x7f18e4039330 msgr2=0x7f18e403b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:36.783 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:36.242+0000 7f1901623700 1 --2- 172.21.15.97:0/4139786118 >> v2:172.21.15.97:6800/1943212787 conn(0x7f18e4039330 0x7f18e403b7f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f18f0006fd0 tx=0x7f18f0006e40 comp rx=0 tx=0).stop 2024-03-21T13:21:36.783 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:36.242+0000 7f1901623700 1 -- 172.21.15.97:0/4139786118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18fc106640 msgr2=0x7f18fc19a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:36.783 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:36.242+0000 7f1901623700 1 --2- 172.21.15.97:0/4139786118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18fc106640 0x7f18fc19a090 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f18ec009870 tx=0x7f18ec011ac0 comp rx=0 tx=0).stop 2024-03-21T13:21:36.783 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:36.242+0000 7f1901623700 1 -- 172.21.15.97:0/4139786118 shutdown_connections 2024-03-21T13:21:36.783 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:36.242+0000 7f1901623700 1 --2- 172.21.15.97:0/4139786118 >> v2:172.21.15.97:6800/1943212787 conn(0x7f18e4039330 0x7f18e403b7f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:36.783 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:36.242+0000 7f1901623700 1 --2- 172.21.15.97:0/4139786118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18fc106640 0x7f18fc19a090 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:36.783 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:36.242+0000 7f1901623700 1 -- 172.21.15.97:0/4139786118 >> 172.21.15.97:0/4139786118 conn(0x7f18fc0ffd60 msgr2=0x7f18fc100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:21:36.783 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:36.242+0000 7f1901623700 1 -- 172.21.15.97:0/4139786118 shutdown_connections 2024-03-21T13:21:36.783 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:36.242+0000 7f1901623700 1 -- 172.21.15.97:0/4139786118 wait complete. 2024-03-21T13:21:36.785 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-21T13:21:36.786 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout disabled 2024-03-21T13:21:36.792 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-21T13:21:36.792 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout inactive 2024-03-21T13:21:36.792 INFO:teuthology.orchestra.run.smithi097.stdout:firewalld.service is not enabled 2024-03-21T13:21:36.792 INFO:teuthology.orchestra.run.smithi097.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-21T13:21:36.794 INFO:teuthology.orchestra.run.smithi097.stdout:Ceph Dashboard is now available at: 2024-03-21T13:21:36.794 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:21:36.794 INFO:teuthology.orchestra.run.smithi097.stdout: URL: https://smithi097.front.sepia.ceph.com:8443/ 2024-03-21T13:21:36.794 INFO:teuthology.orchestra.run.smithi097.stdout: User: admin 2024-03-21T13:21:36.794 INFO:teuthology.orchestra.run.smithi097.stdout: Password: pmtdvgs18f 2024-03-21T13:21:36.794 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:21:36.794 INFO:teuthology.orchestra.run.smithi097.stdout:Saving cluster configuration to /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/config directory 2024-03-21T13:21:36.795 INFO:teuthology.orchestra.run.smithi097.stdout:Enabling autotune for osd_memory_target 2024-03-21T13:21:37.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:37 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4139786118' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-21T13:21:38.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:38 smithi097 ceph-mon[30040]: mgrmap e13: y(active, since 6s) 2024-03-21T13:21:38.882 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:38.112+0000 7f1511811700 1 Processor -- start 2024-03-21T13:21:38.882 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:38.112+0000 7f1511811700 1 -- start start 2024-03-21T13:21:38.882 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:38.112+0000 7f1511811700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f150c1061a0 0x7f150c1065c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:38.882 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:38.112+0000 7f1511811700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150c106b90 con 0x7f150c1061a0 2024-03-21T13:21:38.883 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:38.112+0000 7f150affd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f150c1061a0 0x7f150c1065c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:38.883 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:38.112+0000 7f150affd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f150c1061a0 0x7f150c1065c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50206/0 (socket says 172.21.15.97:50206) 2024-03-21T13:21:38.883 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:38.112+0000 7f150affd700 1 -- 172.21.15.97:0/3920006317 learned_addr learned my addr 172.21.15.97:0/3920006317 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:21:38.883 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:38.113+0000 7f150affd700 1 -- 172.21.15.97:0/3920006317 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f150c1073a0 con 0x7f150c1061a0 2024-03-21T13:21:38.883 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:38.113+0000 7f150affd700 1 --2- 172.21.15.97:0/3920006317 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f150c1061a0 0x7f150c1065c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f14fc0098a0 tx=0x7f14fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55e96447c3c93389 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:38.883 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:38.113+0000 7f1509ffb700 1 -- 172.21.15.97:0/3920006317 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f14fc004160 con 0x7f150c1061a0 2024-03-21T13:21:38.883 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:38.113+0000 7f1509ffb700 1 -- 172.21.15.97:0/3920006317 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f14fc004300 con 0x7f150c1061a0 2024-03-21T13:21:38.883 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:38.113+0000 7f1509ffb700 1 -- 172.21.15.97:0/3920006317 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f14fc011a00 con 0x7f150c1061a0 2024-03-21T13:21:38.883 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:38.114+0000 7f1511811700 1 -- 172.21.15.97:0/3920006317 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f150c1061a0 msgr2=0x7f150c1065c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:38.883 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:38.114+0000 7f1511811700 1 --2- 172.21.15.97:0/3920006317 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f150c1061a0 0x7f150c1065c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f14fc0098a0 tx=0x7f14fc009bb0 comp rx=0 tx=0).stop 2024-03-21T13:21:38.883 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:38.114+0000 7f1511811700 1 -- 172.21.15.97:0/3920006317 shutdown_connections 2024-03-21T13:21:38.883 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:38.114+0000 7f1511811700 1 --2- 172.21.15.97:0/3920006317 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f150c1061a0 0x7f150c1065c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:38.883 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:38.114+0000 7f1511811700 1 -- 172.21.15.97:0/3920006317 >> 172.21.15.97:0/3920006317 conn(0x7f150c101780 msgr2=0x7f150c103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:21:38.883 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:38.114+0000 7f1511811700 1 -- 172.21.15.97:0/3920006317 shutdown_connections 2024-03-21T13:21:38.884 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:38.114+0000 7f1511811700 1 -- 172.21.15.97:0/3920006317 wait complete. 2024-03-21T13:21:38.884 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:38.114+0000 7f1511811700 1 Processor -- start 2024-03-21T13:21:38.884 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:38.114+0000 7f1511811700 1 -- start start 2024-03-21T13:21:38.884 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:38.115+0000 7f1511811700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f150c1061a0 0x7f150c19a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:38.884 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:38.115+0000 7f1511811700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150c10a750 con 0x7f150c1061a0 2024-03-21T13:21:38.884 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:38.115+0000 7f150affd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f150c1061a0 0x7f150c19a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:38.884 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:38.115+0000 7f150affd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f150c1061a0 0x7f150c19a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50216/0 (socket says 172.21.15.97:50216) 2024-03-21T13:21:38.884 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:38.115+0000 7f150affd700 1 -- 172.21.15.97:0/2180282450 learned_addr learned my addr 172.21.15.97:0/2180282450 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:21:38.884 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:38.115+0000 7f150affd700 1 -- 172.21.15.97:0/2180282450 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f150c19a5d0 con 0x7f150c1061a0 2024-03-21T13:21:38.884 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:38.115+0000 7f150affd700 1 --2- 172.21.15.97:0/2180282450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f150c1061a0 0x7f150c19a090 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f14fc009fb0 tx=0x7f14fc011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:38.884 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:38.115+0000 7f151080f700 1 -- 172.21.15.97:0/2180282450 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f14fc01a430 con 0x7f150c1061a0 2024-03-21T13:21:38.884 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:38.115+0000 7f1511811700 1 -- 172.21.15.97:0/2180282450 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f150c19a860 con 0x7f150c1061a0 2024-03-21T13:21:38.884 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:38.115+0000 7f151080f700 1 -- 172.21.15.97:0/2180282450 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f14fc01aa80 con 0x7f150c1061a0 2024-03-21T13:21:38.884 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:38.115+0000 7f151080f700 1 -- 172.21.15.97:0/2180282450 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f14fc0249b0 con 0x7f150c1061a0 2024-03-21T13:21:38.884 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:38.115+0000 7f1511811700 1 -- 172.21.15.97:0/2180282450 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f150c19d550 con 0x7f150c1061a0 2024-03-21T13:21:38.885 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:38.116+0000 7f151080f700 1 -- 172.21.15.97:0/2180282450 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 46315+0+0 (secure 0 0 0) 0x7f14fc022020 con 0x7f150c1061a0 2024-03-21T13:21:38.885 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:38.116+0000 7f1511811700 1 -- 172.21.15.97:0/2180282450 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f150c0637c0 con 0x7f150c1061a0 2024-03-21T13:21:38.885 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:38.117+0000 7f151080f700 1 --2- 172.21.15.97:0/2180282450 >> v2:172.21.15.97:6800/1943212787 conn(0x7f14f40391e0 0x7f14f403b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:38.885 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:38.117+0000 7f151080f700 1 -- 172.21.15.97:0/2180282450 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f14fc04daa0 con 0x7f150c1061a0 2024-03-21T13:21:38.885 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:38.117+0000 7f150a7fc700 1 --2- 172.21.15.97:0/2180282450 >> v2:172.21.15.97:6800/1943212787 conn(0x7f14f40391e0 0x7f14f403b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:38.885 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:38.117+0000 7f150a7fc700 1 --2- 172.21.15.97:0/2180282450 >> v2:172.21.15.97:6800/1943212787 conn(0x7f14f40391e0 0x7f14f403b6a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1500006fd0 tx=0x7f1500006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:38.885 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:38.121+0000 7f151080f700 1 -- 172.21.15.97:0/2180282450 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f14fc0172b0 con 0x7f150c1061a0 2024-03-21T13:21:38.885 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:38.323+0000 7f1511811700 1 -- 172.21.15.97:0/2180282450 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f150c19d9a0 con 0x7f150c1061a0 2024-03-21T13:21:38.885 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:38.324+0000 7f151080f700 1 -- 172.21.15.97:0/2180282450 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f14fc00fdd0 con 0x7f150c1061a0 2024-03-21T13:21:38.885 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:38.326+0000 7f1511811700 1 -- 172.21.15.97:0/2180282450 >> v2:172.21.15.97:6800/1943212787 conn(0x7f14f40391e0 msgr2=0x7f14f403b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:38.885 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:38.326+0000 7f1511811700 1 --2- 172.21.15.97:0/2180282450 >> v2:172.21.15.97:6800/1943212787 conn(0x7f14f40391e0 0x7f14f403b6a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1500006fd0 tx=0x7f1500006e40 comp rx=0 tx=0).stop 2024-03-21T13:21:38.885 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:38.326+0000 7f1511811700 1 -- 172.21.15.97:0/2180282450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f150c1061a0 msgr2=0x7f150c19a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:38.885 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:38.326+0000 7f1511811700 1 --2- 172.21.15.97:0/2180282450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f150c1061a0 0x7f150c19a090 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f14fc009fb0 tx=0x7f14fc011ac0 comp rx=0 tx=0).stop 2024-03-21T13:21:38.885 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:38.326+0000 7f1511811700 1 -- 172.21.15.97:0/2180282450 shutdown_connections 2024-03-21T13:21:38.886 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:38.326+0000 7f1511811700 1 --2- 172.21.15.97:0/2180282450 >> v2:172.21.15.97:6800/1943212787 conn(0x7f14f40391e0 0x7f14f403b6a0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:38.886 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:38.326+0000 7f1511811700 1 --2- 172.21.15.97:0/2180282450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f150c1061a0 0x7f150c19a090 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:38.886 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:38.326+0000 7f1511811700 1 -- 172.21.15.97:0/2180282450 >> 172.21.15.97:0/2180282450 conn(0x7f150c101780 msgr2=0x7f150c102270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:21:38.886 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:38.326+0000 7f1511811700 1 -- 172.21.15.97:0/2180282450 shutdown_connections 2024-03-21T13:21:38.886 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:38.326+0000 7f1511811700 1 -- 172.21.15.97:0/2180282450 wait complete. 2024-03-21T13:21:40.147 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:39.389+0000 7f2c39936700 1 Processor -- start 2024-03-21T13:21:40.147 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:39.389+0000 7f2c39936700 1 -- start start 2024-03-21T13:21:40.147 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:39.389+0000 7f2c39936700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c34105060 0x7f2c34105480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:40.147 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:39.389+0000 7f2c39936700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c34105a50 con 0x7f2c34105060 2024-03-21T13:21:40.147 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:39.389+0000 7f2c32ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c34105060 0x7f2c34105480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:40.147 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:39.389+0000 7f2c32ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c34105060 0x7f2c34105480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50222/0 (socket says 172.21.15.97:50222) 2024-03-21T13:21:40.147 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:39.389+0000 7f2c32ffd700 1 -- 172.21.15.97:0/3275680068 learned_addr learned my addr 172.21.15.97:0/3275680068 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:21:40.147 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:39.390+0000 7f2c32ffd700 1 -- 172.21.15.97:0/3275680068 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c34106260 con 0x7f2c34105060 2024-03-21T13:21:40.147 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:39.390+0000 7f2c32ffd700 1 --2- 172.21.15.97:0/3275680068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c34105060 0x7f2c34105480 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2c240098a0 tx=0x7f2c24009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df3303af1fa554ab server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:40.147 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:39.390+0000 7f2c31ffb700 1 -- 172.21.15.97:0/3275680068 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2c24004160 con 0x7f2c34105060 2024-03-21T13:21:40.147 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:39.390+0000 7f2c31ffb700 1 -- 172.21.15.97:0/3275680068 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f2c24004300 con 0x7f2c34105060 2024-03-21T13:21:40.147 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:39.390+0000 7f2c31ffb700 1 -- 172.21.15.97:0/3275680068 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2c24011a00 con 0x7f2c34105060 2024-03-21T13:21:40.148 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:39.390+0000 7f2c39936700 1 -- 172.21.15.97:0/3275680068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c34105060 msgr2=0x7f2c34105480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:40.148 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:39.390+0000 7f2c39936700 1 --2- 172.21.15.97:0/3275680068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c34105060 0x7f2c34105480 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2c240098a0 tx=0x7f2c24009bb0 comp rx=0 tx=0).stop 2024-03-21T13:21:40.148 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:39.390+0000 7f2c39936700 1 -- 172.21.15.97:0/3275680068 shutdown_connections 2024-03-21T13:21:40.148 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:39.390+0000 7f2c39936700 1 --2- 172.21.15.97:0/3275680068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c34105060 0x7f2c34105480 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:40.148 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:39.390+0000 7f2c39936700 1 -- 172.21.15.97:0/3275680068 >> 172.21.15.97:0/3275680068 conn(0x7f2c341005e0 msgr2=0x7f2c34102a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:21:40.148 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:39.390+0000 7f2c39936700 1 -- 172.21.15.97:0/3275680068 shutdown_connections 2024-03-21T13:21:40.148 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:39.391+0000 7f2c39936700 1 -- 172.21.15.97:0/3275680068 wait complete. 2024-03-21T13:21:40.148 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:39.391+0000 7f2c39936700 1 Processor -- start 2024-03-21T13:21:40.148 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:39.391+0000 7f2c39936700 1 -- start start 2024-03-21T13:21:40.148 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:39.391+0000 7f2c39936700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c34105060 0x7f2c3419a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:40.148 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:39.391+0000 7f2c39936700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c34109610 con 0x7f2c34105060 2024-03-21T13:21:40.148 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:39.391+0000 7f2c32ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c34105060 0x7f2c3419a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:40.148 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:39.392+0000 7f2c32ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c34105060 0x7f2c3419a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50228/0 (socket says 172.21.15.97:50228) 2024-03-21T13:21:40.148 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:39.392+0000 7f2c32ffd700 1 -- 172.21.15.97:0/1835902064 learned_addr learned my addr 172.21.15.97:0/1835902064 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:21:40.149 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:39.392+0000 7f2c32ffd700 1 -- 172.21.15.97:0/1835902064 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c3419a580 con 0x7f2c34105060 2024-03-21T13:21:40.149 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:39.392+0000 7f2c32ffd700 1 --2- 172.21.15.97:0/1835902064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c34105060 0x7f2c3419a040 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2c24009fb0 tx=0x7f2c24011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:40.149 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:39.392+0000 7f2c38934700 1 -- 172.21.15.97:0/1835902064 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2c2401a430 con 0x7f2c34105060 2024-03-21T13:21:40.149 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:39.392+0000 7f2c38934700 1 -- 172.21.15.97:0/1835902064 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f2c2401aa80 con 0x7f2c34105060 2024-03-21T13:21:40.149 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:39.392+0000 7f2c38934700 1 -- 172.21.15.97:0/1835902064 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2c240249b0 con 0x7f2c34105060 2024-03-21T13:21:40.149 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:39.392+0000 7f2c39936700 1 -- 172.21.15.97:0/1835902064 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c3419a810 con 0x7f2c34105060 2024-03-21T13:21:40.149 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:39.392+0000 7f2c39936700 1 -- 172.21.15.97:0/1835902064 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c3419d500 con 0x7f2c34105060 2024-03-21T13:21:40.149 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:39.393+0000 7f2c38934700 1 -- 172.21.15.97:0/1835902064 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 46315+0+0 (secure 0 0 0) 0x7f2c24022020 con 0x7f2c34105060 2024-03-21T13:21:40.149 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:39.393+0000 7f2c38934700 1 --2- 172.21.15.97:0/1835902064 >> v2:172.21.15.97:6800/1943212787 conn(0x7f2c1c039230 0x7f2c1c03b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:40.149 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:39.393+0000 7f2c38934700 1 -- 172.21.15.97:0/1835902064 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f2c2404da60 con 0x7f2c34105060 2024-03-21T13:21:40.149 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:39.393+0000 7f2c39936700 1 -- 172.21.15.97:0/1835902064 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c340637c0 con 0x7f2c34105060 2024-03-21T13:21:40.149 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:39.393+0000 7f2c327fc700 1 --2- 172.21.15.97:0/1835902064 >> v2:172.21.15.97:6800/1943212787 conn(0x7f2c1c039230 0x7f2c1c03b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:40.149 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:39.393+0000 7f2c327fc700 1 --2- 172.21.15.97:0/1835902064 >> v2:172.21.15.97:6800/1943212787 conn(0x7f2c1c039230 0x7f2c1c03b6f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2c28006fd0 tx=0x7f2c28006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:40.149 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:39.398+0000 7f2c38934700 1 -- 172.21.15.97:0/1835902064 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2c240172b0 con 0x7f2c34105060 2024-03-21T13:21:40.149 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:39.697+0000 7f2c39936700 1 -- 172.21.15.97:0/1835902064 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f2c3419d810 con 0x7f2c34105060 2024-03-21T13:21:40.150 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:39.700+0000 7f2c38934700 1 -- 172.21.15.97:0/1835902064 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) v1 ==== 153+0+0 (secure 0 0 0) 0x7f2c2400fdd0 con 0x7f2c34105060 2024-03-21T13:21:40.150 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:39.702+0000 7f2c39936700 1 -- 172.21.15.97:0/1835902064 >> v2:172.21.15.97:6800/1943212787 conn(0x7f2c1c039230 msgr2=0x7f2c1c03b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:40.150 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:39.702+0000 7f2c39936700 1 --2- 172.21.15.97:0/1835902064 >> v2:172.21.15.97:6800/1943212787 conn(0x7f2c1c039230 0x7f2c1c03b6f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2c28006fd0 tx=0x7f2c28006e40 comp rx=0 tx=0).stop 2024-03-21T13:21:40.150 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:39.702+0000 7f2c39936700 1 -- 172.21.15.97:0/1835902064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c34105060 msgr2=0x7f2c3419a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:40.150 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:39.702+0000 7f2c39936700 1 --2- 172.21.15.97:0/1835902064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c34105060 0x7f2c3419a040 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2c24009fb0 tx=0x7f2c24011ac0 comp rx=0 tx=0).stop 2024-03-21T13:21:40.150 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:39.702+0000 7f2c39936700 1 -- 172.21.15.97:0/1835902064 shutdown_connections 2024-03-21T13:21:40.150 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:39.702+0000 7f2c39936700 1 --2- 172.21.15.97:0/1835902064 >> v2:172.21.15.97:6800/1943212787 conn(0x7f2c1c039230 0x7f2c1c03b6f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:40.150 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:39.702+0000 7f2c39936700 1 --2- 172.21.15.97:0/1835902064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c34105060 0x7f2c3419a040 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:40.150 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:39.702+0000 7f2c39936700 1 -- 172.21.15.97:0/1835902064 >> 172.21.15.97:0/1835902064 conn(0x7f2c341005e0 msgr2=0x7f2c34101080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:21:40.150 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:39.702+0000 7f2c39936700 1 -- 172.21.15.97:0/1835902064 shutdown_connections 2024-03-21T13:21:40.150 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-21T13:21:39.702+0000 7f2c39936700 1 -- 172.21.15.97:0/1835902064 wait complete. 2024-03-21T13:21:40.150 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-21T13:21:40.150 INFO:teuthology.orchestra.run.smithi097.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-21T13:21:40.151 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:21:40.151 INFO:teuthology.orchestra.run.smithi097.stdout: sudo /sbin/cephadm shell --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-21T13:21:40.151 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:21:40.151 INFO:teuthology.orchestra.run.smithi097.stdout:Or, if you are only running a single cluster on this host: 2024-03-21T13:21:40.151 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:21:40.151 INFO:teuthology.orchestra.run.smithi097.stdout: sudo /sbin/cephadm shell 2024-03-21T13:21:40.151 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:21:40.151 INFO:teuthology.orchestra.run.smithi097.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-21T13:21:40.151 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:21:40.151 INFO:teuthology.orchestra.run.smithi097.stdout: ceph telemetry on 2024-03-21T13:21:40.151 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:21:40.151 INFO:teuthology.orchestra.run.smithi097.stdout:For more information see: 2024-03-21T13:21:40.151 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:21:40.151 INFO:teuthology.orchestra.run.smithi097.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-21T13:21:40.151 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:21:40.152 INFO:teuthology.orchestra.run.smithi097.stdout:Bootstrap complete. 2024-03-21T13:21:40.185 INFO:tasks.cephadm:Fetching config... 2024-03-21T13:21:40.185 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-21T13:21:40.186 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-21T13:21:40.202 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-21T13:21:40.202 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-21T13:21:40.202 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-21T13:21:40.256 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-21T13:21:40.257 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-21T13:21:40.257 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.a/keyring of=/dev/stdout 2024-03-21T13:21:40.324 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-21T13:21:40.324 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-21T13:21:40.324 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-21T13:21:40.379 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-21T13:21:40.379 DEBUG:teuthology.orchestra.run.smithi097:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpdJ82pPY5BRSv0rYleMp18kXUI/Sr76Yajzf6vPUr7BzRIKoeYmkco4Y5BSJz7Lt2JRir17ByRRVoeIEHva39hCqhGQm6EQs3lXctDdw0DtVnO6YWXV7nf6PaHYJwmvySjrgmvTiNP2m9hIRtS62IU5k1mKXZ+dMn7NHxsLQunlI4609oI6AI3gs64ZYroFFQDxdqU4h3fauRBOG8TClZ+gFbG3J8BhVZpsNWtJPJsq5O+0BwHP0cFHq1R64fiZYVstH8U70/8RHupPqCnl7ELdHQkh2ahbNEQx0Rx/kbxqPAz3XRvsSLmQAwqmphS6L7ql7FDwDFrtfneDkDtIQE2hH3UO5IKEKHKyc/F8vxi6wYUJtj8QuwYF22pADvv+3clRz9Lutcav2mzk4J+i9WUFIRrBZO8YxcqP9KqkmXHpO14IYSnHTpnovjZb/epGVziUj2r0FPPIEsgqBjVlVH+sFfia9GlF5pr04GZRi2qLJFcm5f2oF7PIRYsfVxa3E= ceph-a5dd1f32-e785-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-21T13:21:40.463 INFO:teuthology.orchestra.run.smithi097.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpdJ82pPY5BRSv0rYleMp18kXUI/Sr76Yajzf6vPUr7BzRIKoeYmkco4Y5BSJz7Lt2JRir17ByRRVoeIEHva39hCqhGQm6EQs3lXctDdw0DtVnO6YWXV7nf6PaHYJwmvySjrgmvTiNP2m9hIRtS62IU5k1mKXZ+dMn7NHxsLQunlI4609oI6AI3gs64ZYroFFQDxdqU4h3fauRBOG8TClZ+gFbG3J8BhVZpsNWtJPJsq5O+0BwHP0cFHq1R64fiZYVstH8U70/8RHupPqCnl7ELdHQkh2ahbNEQx0Rx/kbxqPAz3XRvsSLmQAwqmphS6L7ql7FDwDFrtfneDkDtIQE2hH3UO5IKEKHKyc/F8vxi6wYUJtj8QuwYF22pADvv+3clRz9Lutcav2mzk4J+i9WUFIRrBZO8YxcqP9KqkmXHpO14IYSnHTpnovjZb/epGVziUj2r0FPPIEsgqBjVlVH+sFfia9GlF5pr04GZRi2qLJFcm5f2oF7PIRYsfVxa3E= ceph-a5dd1f32-e785-11ee-95cd-87774f69a715 2024-03-21T13:21:40.478 DEBUG:teuthology.orchestra.run.smithi144:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpdJ82pPY5BRSv0rYleMp18kXUI/Sr76Yajzf6vPUr7BzRIKoeYmkco4Y5BSJz7Lt2JRir17ByRRVoeIEHva39hCqhGQm6EQs3lXctDdw0DtVnO6YWXV7nf6PaHYJwmvySjrgmvTiNP2m9hIRtS62IU5k1mKXZ+dMn7NHxsLQunlI4609oI6AI3gs64ZYroFFQDxdqU4h3fauRBOG8TClZ+gFbG3J8BhVZpsNWtJPJsq5O+0BwHP0cFHq1R64fiZYVstH8U70/8RHupPqCnl7ELdHQkh2ahbNEQx0Rx/kbxqPAz3XRvsSLmQAwqmphS6L7ql7FDwDFrtfneDkDtIQE2hH3UO5IKEKHKyc/F8vxi6wYUJtj8QuwYF22pADvv+3clRz9Lutcav2mzk4J+i9WUFIRrBZO8YxcqP9KqkmXHpO14IYSnHTpnovjZb/epGVziUj2r0FPPIEsgqBjVlVH+sFfia9GlF5pr04GZRi2qLJFcm5f2oF7PIRYsfVxa3E= ceph-a5dd1f32-e785-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-21T13:21:40.523 INFO:teuthology.orchestra.run.smithi144.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpdJ82pPY5BRSv0rYleMp18kXUI/Sr76Yajzf6vPUr7BzRIKoeYmkco4Y5BSJz7Lt2JRir17ByRRVoeIEHva39hCqhGQm6EQs3lXctDdw0DtVnO6YWXV7nf6PaHYJwmvySjrgmvTiNP2m9hIRtS62IU5k1mKXZ+dMn7NHxsLQunlI4609oI6AI3gs64ZYroFFQDxdqU4h3fauRBOG8TClZ+gFbG3J8BhVZpsNWtJPJsq5O+0BwHP0cFHq1R64fiZYVstH8U70/8RHupPqCnl7ELdHQkh2ahbNEQx0Rx/kbxqPAz3XRvsSLmQAwqmphS6L7ql7FDwDFrtfneDkDtIQE2hH3UO5IKEKHKyc/F8vxi6wYUJtj8QuwYF22pADvv+3clRz9Lutcav2mzk4J+i9WUFIRrBZO8YxcqP9KqkmXHpO14IYSnHTpnovjZb/epGVziUj2r0FPPIEsgqBjVlVH+sFfia9GlF5pr04GZRi2qLJFcm5f2oF7PIRYsfVxa3E= ceph-a5dd1f32-e785-11ee-95cd-87774f69a715 2024-03-21T13:21:40.538 DEBUG:teuthology.orchestra.run.smithi097:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-21T13:21:40.778 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T13:21:41.039 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:40 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1835902064' entity='client.admin' 2024-03-21T13:21:42.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:42.912+0000 7efce3545700 1 -- 172.21.15.97:0/2748399849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcdc100ac0 msgr2=0x7efcdc100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:42.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:42.912+0000 7efce3545700 1 --2- 172.21.15.97:0/2748399849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcdc100ac0 0x7efcdc100ee0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efcd8009910 tx=0x7efcd8009c20 comp rx=0 tx=0).stop 2024-03-21T13:21:42.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:42.912+0000 7efce3545700 1 -- 172.21.15.97:0/2748399849 shutdown_connections 2024-03-21T13:21:42.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:42.912+0000 7efce3545700 1 --2- 172.21.15.97:0/2748399849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcdc100ac0 0x7efcdc100ee0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:42.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:42.912+0000 7efce3545700 1 -- 172.21.15.97:0/2748399849 >> 172.21.15.97:0/2748399849 conn(0x7efcdc0fc080 msgr2=0x7efcdc0fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:21:42.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:42.912+0000 7efce3545700 1 -- 172.21.15.97:0/2748399849 shutdown_connections 2024-03-21T13:21:42.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:42.912+0000 7efce3545700 1 -- 172.21.15.97:0/2748399849 wait complete. 2024-03-21T13:21:42.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:42.913+0000 7efce3545700 1 Processor -- start 2024-03-21T13:21:42.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:42.913+0000 7efce3545700 1 -- start start 2024-03-21T13:21:42.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:42.913+0000 7efce3545700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcdc100ac0 0x7efcdc196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:42.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:42.913+0000 7efce3545700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcdc106200 con 0x7efcdc100ac0 2024-03-21T13:21:42.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:42.913+0000 7efce12e1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcdc100ac0 0x7efcdc196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:42.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:42.913+0000 7efce12e1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcdc100ac0 0x7efcdc196a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44998/0 (socket says 172.21.15.97:44998) 2024-03-21T13:21:42.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:42.913+0000 7efce12e1700 1 -- 172.21.15.97:0/766999135 learned_addr learned my addr 172.21.15.97:0/766999135 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:21:42.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:42.913+0000 7efce12e1700 1 -- 172.21.15.97:0/766999135 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcdc196f50 con 0x7efcdc100ac0 2024-03-21T13:21:42.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:42.914+0000 7efce12e1700 1 --2- 172.21.15.97:0/766999135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcdc100ac0 0x7efcdc196a10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efcd800b5c0 tx=0x7efcd80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:42.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:42.914+0000 7efcd27fc700 1 -- 172.21.15.97:0/766999135 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efcd8027070 con 0x7efcdc100ac0 2024-03-21T13:21:42.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:42.914+0000 7efcd27fc700 1 -- 172.21.15.97:0/766999135 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7efcd800be10 con 0x7efcdc100ac0 2024-03-21T13:21:42.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:42.914+0000 7efce3545700 1 -- 172.21.15.97:0/766999135 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcdc1971e0 con 0x7efcdc100ac0 2024-03-21T13:21:42.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:42.914+0000 7efcd27fc700 1 -- 172.21.15.97:0/766999135 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efcd8022e40 con 0x7efcdc100ac0 2024-03-21T13:21:42.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:42.914+0000 7efce3545700 1 -- 172.21.15.97:0/766999135 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcdc199ed0 con 0x7efcdc100ac0 2024-03-21T13:21:42.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:42.915+0000 7efcd27fc700 1 -- 172.21.15.97:0/766999135 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 46315+0+0 (secure 0 0 0) 0x7efcd802a070 con 0x7efcdc100ac0 2024-03-21T13:21:42.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:42.915+0000 7efce3545700 1 -- 172.21.15.97:0/766999135 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcdc106980 con 0x7efcdc100ac0 2024-03-21T13:21:42.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:42.915+0000 7efcd27fc700 1 --2- 172.21.15.97:0/766999135 >> v2:172.21.15.97:6800/1943212787 conn(0x7efcc8039230 0x7efcc803b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:42.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:42.915+0000 7efcd27fc700 1 -- 172.21.15.97:0/766999135 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (secure 0 0 0) 0x7efcd801e030 con 0x7efcdc100ac0 2024-03-21T13:21:42.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:42.915+0000 7efce0ae0700 1 --2- 172.21.15.97:0/766999135 >> v2:172.21.15.97:6800/1943212787 conn(0x7efcc8039230 0x7efcc803b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:42.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:42.915+0000 7efce0ae0700 1 --2- 172.21.15.97:0/766999135 >> v2:172.21.15.97:6800/1943212787 conn(0x7efcc8039230 0x7efcc803b6f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7efccc006fd0 tx=0x7efccc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:42.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:42.920+0000 7efcd27fc700 1 -- 172.21.15.97:0/766999135 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7efcd8013070 con 0x7efcdc100ac0 2024-03-21T13:21:43.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:43.118+0000 7efce3545700 1 -- 172.21.15.97:0/766999135 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7efcdc02da30 con 0x7efcdc100ac0 2024-03-21T13:21:43.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:43.119+0000 7efcd27fc700 1 -- 172.21.15.97:0/766999135 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7efcd80047d0 con 0x7efcdc100ac0 2024-03-21T13:21:43.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:43.122+0000 7efce3545700 1 -- 172.21.15.97:0/766999135 >> v2:172.21.15.97:6800/1943212787 conn(0x7efcc8039230 msgr2=0x7efcc803b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:43.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:43.122+0000 7efce3545700 1 --2- 172.21.15.97:0/766999135 >> v2:172.21.15.97:6800/1943212787 conn(0x7efcc8039230 0x7efcc803b6f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7efccc006fd0 tx=0x7efccc006e40 comp rx=0 tx=0).stop 2024-03-21T13:21:43.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:43.122+0000 7efce3545700 1 -- 172.21.15.97:0/766999135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcdc100ac0 msgr2=0x7efcdc196a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:43.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:43.122+0000 7efce3545700 1 --2- 172.21.15.97:0/766999135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcdc100ac0 0x7efcdc196a10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efcd800b5c0 tx=0x7efcd80046b0 comp rx=0 tx=0).stop 2024-03-21T13:21:43.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:43.122+0000 7efce3545700 1 -- 172.21.15.97:0/766999135 shutdown_connections 2024-03-21T13:21:43.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:43.122+0000 7efce3545700 1 --2- 172.21.15.97:0/766999135 >> v2:172.21.15.97:6800/1943212787 conn(0x7efcc8039230 0x7efcc803b6f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:43.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:43.122+0000 7efce3545700 1 --2- 172.21.15.97:0/766999135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcdc100ac0 0x7efcdc196a10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:43.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:43.122+0000 7efce3545700 1 -- 172.21.15.97:0/766999135 >> 172.21.15.97:0/766999135 conn(0x7efcdc0fc080 msgr2=0x7efcdc103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:21:43.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:43.122+0000 7efce3545700 1 -- 172.21.15.97:0/766999135 shutdown_connections 2024-03-21T13:21:43.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:43.122+0000 7efce3545700 1 -- 172.21.15.97:0/766999135 wait complete. 2024-03-21T13:21:43.434 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:43 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:21:43.434 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:43 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:21:43.434 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:43 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi097", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:21:43.435 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:43 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:21:43.435 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:43 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:21:43.435 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:43 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:21:43.746 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-21T13:21:43.746 DEBUG:teuthology.orchestra.run.smithi097:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-21T13:21:43.980 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T13:21:44.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:44 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/766999135' entity='client.admin' 2024-03-21T13:21:44.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:44 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:21:44.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:44 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:21:45.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:45.510+0000 7f6f0db04700 1 -- 172.21.15.97:0/4110622479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f080ffe90 msgr2=0x7f6f081002b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:45.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:45.510+0000 7f6f0db04700 1 --2- 172.21.15.97:0/4110622479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f080ffe90 0x7f6f081002b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6ef800b0a0 tx=0x7f6ef800b3b0 comp rx=0 tx=0).stop 2024-03-21T13:21:45.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:45.510+0000 7f6f0db04700 1 -- 172.21.15.97:0/4110622479 shutdown_connections 2024-03-21T13:21:45.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:45.510+0000 7f6f0db04700 1 --2- 172.21.15.97:0/4110622479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f080ffe90 0x7f6f081002b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:45.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:45.510+0000 7f6f0db04700 1 -- 172.21.15.97:0/4110622479 >> 172.21.15.97:0/4110622479 conn(0x7f6f080fb7f0 msgr2=0x7f6f080fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:21:45.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:45.510+0000 7f6f0db04700 1 -- 172.21.15.97:0/4110622479 shutdown_connections 2024-03-21T13:21:45.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:45.510+0000 7f6f0db04700 1 -- 172.21.15.97:0/4110622479 wait complete. 2024-03-21T13:21:45.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:45.510+0000 7f6f0db04700 1 Processor -- start 2024-03-21T13:21:45.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:45.511+0000 7f6f0db04700 1 -- start start 2024-03-21T13:21:45.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:45.511+0000 7f6f0db04700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f080ffe90 0x7f6f081026f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:45.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:45.511+0000 7f6f0db04700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f08107310 con 0x7f6f080ffe90 2024-03-21T13:21:45.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:45.511+0000 7f6f077fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f080ffe90 0x7f6f081026f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:45.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:45.511+0000 7f6f077fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f080ffe90 0x7f6f081026f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45018/0 (socket says 172.21.15.97:45018) 2024-03-21T13:21:45.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:45.511+0000 7f6f077fe700 1 -- 172.21.15.97:0/2910447356 learned_addr learned my addr 172.21.15.97:0/2910447356 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:21:45.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:45.511+0000 7f6f077fe700 1 -- 172.21.15.97:0/2910447356 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f08102c30 con 0x7f6f080ffe90 2024-03-21T13:21:45.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:45.511+0000 7f6f077fe700 1 --2- 172.21.15.97:0/2910447356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f080ffe90 0x7f6f081026f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8015040 tx=0x7f6ef800bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:45.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:45.511+0000 7f6f04ff9700 1 -- 172.21.15.97:0/2910447356 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6ef8017040 con 0x7f6f080ffe90 2024-03-21T13:21:45.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:45.511+0000 7f6f04ff9700 1 -- 172.21.15.97:0/2910447356 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f6ef8022070 con 0x7f6f080ffe90 2024-03-21T13:21:45.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:45.512+0000 7f6f0db04700 1 -- 172.21.15.97:0/2910447356 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f08100e20 con 0x7f6f080ffe90 2024-03-21T13:21:45.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:45.512+0000 7f6f04ff9700 1 -- 172.21.15.97:0/2910447356 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6ef801d440 con 0x7f6f080ffe90 2024-03-21T13:21:45.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:45.512+0000 7f6f0db04700 1 -- 172.21.15.97:0/2910447356 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f08101360 con 0x7f6f080ffe90 2024-03-21T13:21:45.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:45.512+0000 7f6f04ff9700 1 -- 172.21.15.97:0/2910447356 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 46315+0+0 (secure 0 0 0) 0x7f6ef801d6b0 con 0x7f6f080ffe90 2024-03-21T13:21:45.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:45.512+0000 7f6f04ff9700 1 --2- 172.21.15.97:0/2910447356 >> v2:172.21.15.97:6800/1943212787 conn(0x7f6ef0039230 0x7f6ef003b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:45.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:45.512+0000 7f6f04ff9700 1 -- 172.21.15.97:0/2910447356 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f6ef800f050 con 0x7f6f080ffe90 2024-03-21T13:21:45.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:45.513+0000 7f6f06ffd700 1 --2- 172.21.15.97:0/2910447356 >> v2:172.21.15.97:6800/1943212787 conn(0x7f6ef0039230 0x7f6ef003b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:45.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:45.513+0000 7f6f0db04700 1 -- 172.21.15.97:0/2910447356 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f08107a90 con 0x7f6f080ffe90 2024-03-21T13:21:45.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:45.513+0000 7f6f06ffd700 1 --2- 172.21.15.97:0/2910447356 >> v2:172.21.15.97:6800/1943212787 conn(0x7f6ef0039230 0x7f6ef003b6f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6efc006fd0 tx=0x7f6efc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:45.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:45.518+0000 7f6f04ff9700 1 -- 172.21.15.97:0/2910447356 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6ef80255c0 con 0x7f6f080ffe90 2024-03-21T13:21:45.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:45.714+0000 7f6f0db04700 1 -- 172.21.15.97:0/2910447356 --> v2:172.21.15.97:6800/1943212787 -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f6f081015e0 con 0x7f6ef0039230 2024-03-21T13:21:45.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:45.718+0000 7f6f04ff9700 1 -- 172.21.15.97:0/2910447356 <== mgr.14152 v2:172.21.15.97:6800/1943212787 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f6f081015e0 con 0x7f6ef0039230 2024-03-21T13:21:45.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:45.720+0000 7f6f0db04700 1 -- 172.21.15.97:0/2910447356 >> v2:172.21.15.97:6800/1943212787 conn(0x7f6ef0039230 msgr2=0x7f6ef003b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:45.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:45.720+0000 7f6f0db04700 1 --2- 172.21.15.97:0/2910447356 >> v2:172.21.15.97:6800/1943212787 conn(0x7f6ef0039230 0x7f6ef003b6f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6efc006fd0 tx=0x7f6efc006e40 comp rx=0 tx=0).stop 2024-03-21T13:21:45.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:45.720+0000 7f6f0db04700 1 -- 172.21.15.97:0/2910447356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f080ffe90 msgr2=0x7f6f081026f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:45.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:45.720+0000 7f6f0db04700 1 --2- 172.21.15.97:0/2910447356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f080ffe90 0x7f6f081026f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8015040 tx=0x7f6ef800bd40 comp rx=0 tx=0).stop 2024-03-21T13:21:45.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:45.720+0000 7f6f0db04700 1 -- 172.21.15.97:0/2910447356 shutdown_connections 2024-03-21T13:21:45.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:45.720+0000 7f6f0db04700 1 --2- 172.21.15.97:0/2910447356 >> v2:172.21.15.97:6800/1943212787 conn(0x7f6ef0039230 0x7f6ef003b6f0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:45.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:45.720+0000 7f6f0db04700 1 --2- 172.21.15.97:0/2910447356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f080ffe90 0x7f6f081026f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:45.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:45.720+0000 7f6f0db04700 1 -- 172.21.15.97:0/2910447356 >> 172.21.15.97:0/2910447356 conn(0x7f6f080fb7f0 msgr2=0x7f6f080fc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:21:45.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:45.720+0000 7f6f0db04700 1 -- 172.21.15.97:0/2910447356 shutdown_connections 2024-03-21T13:21:45.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:45.720+0000 7f6f0db04700 1 -- 172.21.15.97:0/2910447356 wait complete. 2024-03-21T13:21:46.521 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi144 2024-03-21T13:21:46.521 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-21T13:21:46.521 DEBUG:teuthology.orchestra.run.smithi144:> dd of=/etc/ceph/ceph.conf 2024-03-21T13:21:46.541 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-21T13:21:46.541 DEBUG:teuthology.orchestra.run.smithi144:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-21T13:21:46.598 INFO:tasks.cephadm:Adding host smithi144 to orchestrator... 2024-03-21T13:21:46.598 DEBUG:teuthology.orchestra.run.smithi097:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph orch host add smithi144 2024-03-21T13:21:46.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:46 smithi097 ceph-mon[30040]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:21:46.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:46 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:21:46.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:46 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:21:46.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:46 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:21:46.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:46 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:21:46.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:46 smithi097 ceph-mon[30040]: Updating smithi097:/etc/ceph/ceph.conf 2024-03-21T13:21:46.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:46 smithi097 ceph-mon[30040]: Updating smithi097:/var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T13:21:46.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:46 smithi097 ceph-mon[30040]: Updating smithi097:/etc/ceph/ceph.client.admin.keyring 2024-03-21T13:21:46.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:46 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:21:46.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:46 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:21:46.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:46 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:21:46.834 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T13:21:48.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:47 smithi097 ceph-mon[30040]: Updating smithi097:/var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T13:21:48.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:48.467+0000 7fe624030700 1 -- 172.21.15.97:0/1358414372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe61c106aa0 msgr2=0x7fe61c106e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:48.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:48.467+0000 7fe624030700 1 --2- 172.21.15.97:0/1358414372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe61c106aa0 0x7fe61c106e80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe618009910 tx=0x7fe618009c20 comp rx=0 tx=0).stop 2024-03-21T13:21:48.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:48.467+0000 7fe624030700 1 -- 172.21.15.97:0/1358414372 shutdown_connections 2024-03-21T13:21:48.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:48.467+0000 7fe624030700 1 --2- 172.21.15.97:0/1358414372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe61c106aa0 0x7fe61c106e80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:48.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:48.467+0000 7fe624030700 1 -- 172.21.15.97:0/1358414372 >> 172.21.15.97:0/1358414372 conn(0x7fe61c0fc5d0 msgr2=0x7fe61c0fe9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:21:48.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:48.467+0000 7fe624030700 1 -- 172.21.15.97:0/1358414372 shutdown_connections 2024-03-21T13:21:48.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:48.467+0000 7fe624030700 1 -- 172.21.15.97:0/1358414372 wait complete. 2024-03-21T13:21:48.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:48.467+0000 7fe624030700 1 Processor -- start 2024-03-21T13:21:48.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:48.467+0000 7fe624030700 1 -- start start 2024-03-21T13:21:48.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:48.467+0000 7fe624030700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe61c106aa0 0x7fe61c197c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:48.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:48.467+0000 7fe624030700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61c10a7a0 con 0x7fe61c106aa0 2024-03-21T13:21:48.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:48.468+0000 7fe621dcc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe61c106aa0 0x7fe61c197c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:48.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:48.468+0000 7fe621dcc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe61c106aa0 0x7fe61c197c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45034/0 (socket says 172.21.15.97:45034) 2024-03-21T13:21:48.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:48.468+0000 7fe621dcc700 1 -- 172.21.15.97:0/1922736376 learned_addr learned my addr 172.21.15.97:0/1922736376 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:21:48.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:48.468+0000 7fe621dcc700 1 -- 172.21.15.97:0/1922736376 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe61c198150 con 0x7fe61c106aa0 2024-03-21T13:21:48.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:48.468+0000 7fe621dcc700 1 --2- 172.21.15.97:0/1922736376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe61c106aa0 0x7fe61c197c10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe618004f80 tx=0x7fe618005060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:48.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:48.468+0000 7fe612ffd700 1 -- 172.21.15.97:0/1922736376 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe618026070 con 0x7fe61c106aa0 2024-03-21T13:21:48.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:48.468+0000 7fe624030700 1 -- 172.21.15.97:0/1922736376 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe61c198440 con 0x7fe61c106aa0 2024-03-21T13:21:48.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:48.468+0000 7fe612ffd700 1 -- 172.21.15.97:0/1922736376 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fe61800bba0 con 0x7fe61c106aa0 2024-03-21T13:21:48.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:48.468+0000 7fe624030700 1 -- 172.21.15.97:0/1922736376 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe61c198900 con 0x7fe61c106aa0 2024-03-21T13:21:48.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:48.468+0000 7fe612ffd700 1 -- 172.21.15.97:0/1922736376 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe618003930 con 0x7fe61c106aa0 2024-03-21T13:21:48.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:48.469+0000 7fe624030700 1 -- 172.21.15.97:0/1922736376 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe61c1021e0 con 0x7fe61c106aa0 2024-03-21T13:21:48.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:48.469+0000 7fe612ffd700 1 -- 172.21.15.97:0/1922736376 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 46315+0+0 (secure 0 0 0) 0x7fe618029070 con 0x7fe61c106aa0 2024-03-21T13:21:48.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:48.469+0000 7fe612ffd700 1 --2- 172.21.15.97:0/1922736376 >> v2:172.21.15.97:6800/1943212787 conn(0x7fe6080391e0 0x7fe60803b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:48.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:48.469+0000 7fe612ffd700 1 -- 172.21.15.97:0/1922736376 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (secure 0 0 0) 0x7fe61801e070 con 0x7fe61c106aa0 2024-03-21T13:21:48.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:48.469+0000 7fe6215cb700 1 --2- 172.21.15.97:0/1922736376 >> v2:172.21.15.97:6800/1943212787 conn(0x7fe6080391e0 0x7fe60803b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:48.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:48.470+0000 7fe6215cb700 1 --2- 172.21.15.97:0/1922736376 >> v2:172.21.15.97:6800/1943212787 conn(0x7fe6080391e0 0x7fe60803b6a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe60c006fd0 tx=0x7fe60c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:48.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:48.474+0000 7fe612ffd700 1 -- 172.21.15.97:0/1922736376 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe618013070 con 0x7fe61c106aa0 2024-03-21T13:21:48.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:48.672+0000 7fe624030700 1 -- 172.21.15.97:0/1922736376 --> v2:172.21.15.97:6800/1943212787 -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi144", "target": ["mon-mgr", ""]}) v1 -- 0x7fe61c0637c0 con 0x7fe6080391e0 2024-03-21T13:21:49.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:49 smithi097 ceph-mon[30040]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi144", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:21:50.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:50 smithi097 ceph-mon[30040]: Deploying cephadm binary to smithi144 2024-03-21T13:21:52.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:52 smithi097 ceph-mon[30040]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:21:53.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:53.077+0000 7fe612ffd700 1 -- 172.21.15.97:0/1922736376 <== mgr.14152 v2:172.21.15.97:6800/1943212787 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fe61c0637c0 con 0x7fe6080391e0 2024-03-21T13:21:53.078 INFO:teuthology.orchestra.run.smithi097.stdout:Added host 'smithi144' with addr '172.21.15.144' 2024-03-21T13:21:53.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:53.079+0000 7fe624030700 1 -- 172.21.15.97:0/1922736376 >> v2:172.21.15.97:6800/1943212787 conn(0x7fe6080391e0 msgr2=0x7fe60803b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:53.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:53.079+0000 7fe624030700 1 --2- 172.21.15.97:0/1922736376 >> v2:172.21.15.97:6800/1943212787 conn(0x7fe6080391e0 0x7fe60803b6a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe60c006fd0 tx=0x7fe60c006e40 comp rx=0 tx=0).stop 2024-03-21T13:21:53.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:53.079+0000 7fe624030700 1 -- 172.21.15.97:0/1922736376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe61c106aa0 msgr2=0x7fe61c197c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:53.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:53.079+0000 7fe624030700 1 --2- 172.21.15.97:0/1922736376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe61c106aa0 0x7fe61c197c10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe618004f80 tx=0x7fe618005060 comp rx=0 tx=0).stop 2024-03-21T13:21:53.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:53.079+0000 7fe624030700 1 -- 172.21.15.97:0/1922736376 shutdown_connections 2024-03-21T13:21:53.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:53.079+0000 7fe624030700 1 --2- 172.21.15.97:0/1922736376 >> v2:172.21.15.97:6800/1943212787 conn(0x7fe6080391e0 0x7fe60803b6a0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:53.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:53.079+0000 7fe624030700 1 --2- 172.21.15.97:0/1922736376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe61c106aa0 0x7fe61c197c10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:53.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:53.079+0000 7fe624030700 1 -- 172.21.15.97:0/1922736376 >> 172.21.15.97:0/1922736376 conn(0x7fe61c0fc5d0 msgr2=0x7fe61c108860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:21:53.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:53.079+0000 7fe624030700 1 -- 172.21.15.97:0/1922736376 shutdown_connections 2024-03-21T13:21:53.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:53.079+0000 7fe624030700 1 -- 172.21.15.97:0/1922736376 wait complete. 2024-03-21T13:21:53.721 DEBUG:teuthology.orchestra.run.smithi097:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph orch host ls --format=json 2024-03-21T13:21:53.962 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T13:21:54.342 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:54 smithi097 ceph-mon[30040]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:21:54.342 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:54 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:21:54.342 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:54 smithi097 ceph-mon[30040]: Added host smithi144 2024-03-21T13:21:54.342 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:54 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:21:54.342 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:54 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:21:55.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:55 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:21:55.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:55.603+0000 7f8b15d9e700 1 -- 172.21.15.97:0/262975119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b10101dd0 msgr2=0x7f8b101021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:55.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:55.603+0000 7f8b15d9e700 1 --2- 172.21.15.97:0/262975119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b10101dd0 0x7f8b101021b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8b00009910 tx=0x7f8b00009c20 comp rx=0 tx=0).stop 2024-03-21T13:21:55.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:55.603+0000 7f8b15d9e700 1 -- 172.21.15.97:0/262975119 shutdown_connections 2024-03-21T13:21:55.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:55.603+0000 7f8b15d9e700 1 --2- 172.21.15.97:0/262975119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b10101dd0 0x7f8b101021b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:55.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:55.603+0000 7f8b15d9e700 1 -- 172.21.15.97:0/262975119 >> 172.21.15.97:0/262975119 conn(0x7f8b100fd680 msgr2=0x7f8b100ffaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:21:55.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:55.603+0000 7f8b15d9e700 1 -- 172.21.15.97:0/262975119 shutdown_connections 2024-03-21T13:21:55.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:55.603+0000 7f8b15d9e700 1 -- 172.21.15.97:0/262975119 wait complete. 2024-03-21T13:21:55.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:55.603+0000 7f8b15d9e700 1 Processor -- start 2024-03-21T13:21:55.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:55.603+0000 7f8b15d9e700 1 -- start start 2024-03-21T13:21:55.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:55.604+0000 7f8b15d9e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b10101dd0 0x7f8b10196d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:55.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:55.604+0000 7f8b15d9e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1010b850 con 0x7f8b10101dd0 2024-03-21T13:21:55.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:55.604+0000 7f8b0f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b10101dd0 0x7f8b10196d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:55.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:55.604+0000 7f8b0f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b10101dd0 0x7f8b10196d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47812/0 (socket says 172.21.15.97:47812) 2024-03-21T13:21:55.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:55.604+0000 7f8b0f7fe700 1 -- 172.21.15.97:0/4172126283 learned_addr learned my addr 172.21.15.97:0/4172126283 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:21:55.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:55.604+0000 7f8b0f7fe700 1 -- 172.21.15.97:0/4172126283 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b101972b0 con 0x7f8b10101dd0 2024-03-21T13:21:55.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:55.604+0000 7f8b0f7fe700 1 --2- 172.21.15.97:0/4172126283 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b10101dd0 0x7f8b10196d70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8b00006010 tx=0x7f8b00005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:55.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:55.604+0000 7f8b0d7fa700 1 -- 172.21.15.97:0/4172126283 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8b00026070 con 0x7f8b10101dd0 2024-03-21T13:21:55.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:55.604+0000 7f8b0d7fa700 1 -- 172.21.15.97:0/4172126283 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f8b0000bba0 con 0x7f8b10101dd0 2024-03-21T13:21:55.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:55.604+0000 7f8b15d9e700 1 -- 172.21.15.97:0/4172126283 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b101975a0 con 0x7f8b10101dd0 2024-03-21T13:21:55.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:55.605+0000 7f8b0d7fa700 1 -- 172.21.15.97:0/4172126283 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8b000037d0 con 0x7f8b10101dd0 2024-03-21T13:21:55.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:55.605+0000 7f8b15d9e700 1 -- 172.21.15.97:0/4172126283 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b1019b640 con 0x7f8b10101dd0 2024-03-21T13:21:55.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:55.605+0000 7f8b0d7fa700 1 -- 172.21.15.97:0/4172126283 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 46315+0+0 (secure 0 0 0) 0x7f8b00029070 con 0x7f8b10101dd0 2024-03-21T13:21:55.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:55.605+0000 7f8b15d9e700 1 -- 172.21.15.97:0/4172126283 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b101032b0 con 0x7f8b10101dd0 2024-03-21T13:21:55.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:55.605+0000 7f8b0d7fa700 1 --2- 172.21.15.97:0/4172126283 >> v2:172.21.15.97:6800/1943212787 conn(0x7f8afc0391e0 0x7f8afc03b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:55.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:55.606+0000 7f8b0d7fa700 1 -- 172.21.15.97:0/4172126283 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f8b0001e070 con 0x7f8b10101dd0 2024-03-21T13:21:55.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:55.606+0000 7f8b06ffd700 1 --2- 172.21.15.97:0/4172126283 >> v2:172.21.15.97:6800/1943212787 conn(0x7f8afc0391e0 0x7f8afc03b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:55.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:55.606+0000 7f8b06ffd700 1 --2- 172.21.15.97:0/4172126283 >> v2:172.21.15.97:6800/1943212787 conn(0x7f8afc0391e0 0x7f8afc03b6a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8af8006fd0 tx=0x7f8af8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:55.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:55.610+0000 7f8b0d7fa700 1 -- 172.21.15.97:0/4172126283 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8b00013070 con 0x7f8b10101dd0 2024-03-21T13:21:55.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:55.811+0000 7f8b15d9e700 1 -- 172.21.15.97:0/4172126283 --> v2:172.21.15.97:6800/1943212787 -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8b101980d0 con 0x7f8afc0391e0 2024-03-21T13:21:55.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:55.813+0000 7f8b0d7fa700 1 -- 172.21.15.97:0/4172126283 <== mgr.14152 v2:172.21.15.97:6800/1943212787 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7f8b101980d0 con 0x7f8afc0391e0 2024-03-21T13:21:55.813 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:21:55.813 INFO:teuthology.orchestra.run.smithi097.stdout:[{"addr": "172.21.15.97", "hostname": "smithi097", "labels": [], "status": ""}, {"addr": "172.21.15.144", "hostname": "smithi144", "labels": [], "status": ""}] 2024-03-21T13:21:55.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:55.815+0000 7f8b15d9e700 1 -- 172.21.15.97:0/4172126283 >> v2:172.21.15.97:6800/1943212787 conn(0x7f8afc0391e0 msgr2=0x7f8afc03b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:55.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:55.815+0000 7f8b15d9e700 1 --2- 172.21.15.97:0/4172126283 >> v2:172.21.15.97:6800/1943212787 conn(0x7f8afc0391e0 0x7f8afc03b6a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8af8006fd0 tx=0x7f8af8006e40 comp rx=0 tx=0).stop 2024-03-21T13:21:55.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:55.815+0000 7f8b15d9e700 1 -- 172.21.15.97:0/4172126283 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b10101dd0 msgr2=0x7f8b10196d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:55.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:55.815+0000 7f8b15d9e700 1 --2- 172.21.15.97:0/4172126283 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b10101dd0 0x7f8b10196d70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8b00006010 tx=0x7f8b00005030 comp rx=0 tx=0).stop 2024-03-21T13:21:55.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:55.815+0000 7f8b15d9e700 1 -- 172.21.15.97:0/4172126283 shutdown_connections 2024-03-21T13:21:55.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:55.815+0000 7f8b15d9e700 1 --2- 172.21.15.97:0/4172126283 >> v2:172.21.15.97:6800/1943212787 conn(0x7f8afc0391e0 0x7f8afc03b6a0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:55.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:55.815+0000 7f8b15d9e700 1 --2- 172.21.15.97:0/4172126283 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b10101dd0 0x7f8b10196d70 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:55.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:55.815+0000 7f8b15d9e700 1 -- 172.21.15.97:0/4172126283 >> 172.21.15.97:0/4172126283 conn(0x7f8b100fd680 msgr2=0x7f8b10109930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:21:55.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:55.815+0000 7f8b15d9e700 1 -- 172.21.15.97:0/4172126283 shutdown_connections 2024-03-21T13:21:55.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:55.815+0000 7f8b15d9e700 1 -- 172.21.15.97:0/4172126283 wait complete. 2024-03-21T13:21:56.384 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:56 smithi097 ceph-mon[30040]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:21:56.384 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:56 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:21:56.409 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-21T13:21:56.409 DEBUG:teuthology.orchestra.run.smithi097:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph osd crush tunables default 2024-03-21T13:21:56.641 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T13:21:57.440 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:57 smithi097 ceph-mon[30040]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T13:21:58.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:58.238+0000 7f24f359e700 1 -- 172.21.15.97:0/463035419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24f41058a0 msgr2=0x7f24f4105c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:58.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:58.238+0000 7f24f359e700 1 --2- 172.21.15.97:0/463035419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24f41058a0 0x7f24f4105c80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f24ec009910 tx=0x7f24ec009c20 comp rx=0 tx=0).stop 2024-03-21T13:21:58.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:58.238+0000 7f24f359e700 1 -- 172.21.15.97:0/463035419 shutdown_connections 2024-03-21T13:21:58.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:58.238+0000 7f24f359e700 1 --2- 172.21.15.97:0/463035419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24f41058a0 0x7f24f4105c80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:58.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:58.238+0000 7f24f359e700 1 -- 172.21.15.97:0/463035419 >> 172.21.15.97:0/463035419 conn(0x7f24f40fbb20 msgr2=0x7f24f40fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:21:58.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:58.238+0000 7f24f359e700 1 -- 172.21.15.97:0/463035419 shutdown_connections 2024-03-21T13:21:58.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:58.238+0000 7f24f359e700 1 -- 172.21.15.97:0/463035419 wait complete. 2024-03-21T13:21:58.239 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:58 smithi097 ceph-mon[30040]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:21:58.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:58.239+0000 7f24f359e700 1 Processor -- start 2024-03-21T13:21:58.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:58.239+0000 7f24f359e700 1 -- start start 2024-03-21T13:21:58.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:58.239+0000 7f24f359e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24f41058a0 0x7f24f4195860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:58.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:58.239+0000 7f24f359e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f410a6e0 con 0x7f24f41058a0 2024-03-21T13:21:58.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:58.239+0000 7f24f259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24f41058a0 0x7f24f4195860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:58.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:58.239+0000 7f24f259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24f41058a0 0x7f24f4195860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47826/0 (socket says 172.21.15.97:47826) 2024-03-21T13:21:58.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:58.239+0000 7f24f259c700 1 -- 172.21.15.97:0/3601209568 learned_addr learned my addr 172.21.15.97:0/3601209568 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:21:58.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:58.239+0000 7f24f259c700 1 -- 172.21.15.97:0/3601209568 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f4100d40 con 0x7f24f41058a0 2024-03-21T13:21:58.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:58.240+0000 7f24f259c700 1 --2- 172.21.15.97:0/3601209568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24f41058a0 0x7f24f4195860 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f24ec004f80 tx=0x7f24ec005060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:58.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:58.240+0000 7f24eb7fe700 1 -- 172.21.15.97:0/3601209568 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24ec026070 con 0x7f24f41058a0 2024-03-21T13:21:58.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:58.240+0000 7f24eb7fe700 1 -- 172.21.15.97:0/3601209568 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f24ec00bc00 con 0x7f24f41058a0 2024-03-21T13:21:58.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:58.240+0000 7f24f359e700 1 -- 172.21.15.97:0/3601209568 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24f4195da0 con 0x7f24f41058a0 2024-03-21T13:21:58.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:58.240+0000 7f24eb7fe700 1 -- 172.21.15.97:0/3601209568 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24ec003990 con 0x7f24f41058a0 2024-03-21T13:21:58.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:58.240+0000 7f24f359e700 1 -- 172.21.15.97:0/3601209568 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24f4196200 con 0x7f24f41058a0 2024-03-21T13:21:58.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:58.241+0000 7f24eb7fe700 1 -- 172.21.15.97:0/3601209568 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 46315+0+0 (secure 0 0 0) 0x7f24ec029070 con 0x7f24f41058a0 2024-03-21T13:21:58.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:58.241+0000 7f24eb7fe700 1 --2- 172.21.15.97:0/3601209568 >> v2:172.21.15.97:6800/1943212787 conn(0x7f24e00391e0 0x7f24e003b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:21:58.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:58.241+0000 7f24f359e700 1 -- 172.21.15.97:0/3601209568 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24f4102060 con 0x7f24f41058a0 2024-03-21T13:21:58.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:58.241+0000 7f24eb7fe700 1 -- 172.21.15.97:0/3601209568 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f24ec01e070 con 0x7f24f41058a0 2024-03-21T13:21:58.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:58.241+0000 7f24f1d9b700 1 --2- 172.21.15.97:0/3601209568 >> v2:172.21.15.97:6800/1943212787 conn(0x7f24e00391e0 0x7f24e003b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:21:58.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:58.241+0000 7f24f1d9b700 1 --2- 172.21.15.97:0/3601209568 >> v2:172.21.15.97:6800/1943212787 conn(0x7f24e00391e0 0x7f24e003b6a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f24dc006fd0 tx=0x7f24dc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:21:58.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:58.246+0000 7f24eb7fe700 1 -- 172.21.15.97:0/3601209568 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f24ec013070 con 0x7f24f41058a0 2024-03-21T13:21:58.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:58.445+0000 7f24f359e700 1 -- 172.21.15.97:0/3601209568 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f24f40637c0 con 0x7f24f41058a0 2024-03-21T13:21:59.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:59.119+0000 7f24eb7fe700 1 -- 172.21.15.97:0/3601209568 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f24ec00e5a0 con 0x7f24f41058a0 2024-03-21T13:21:59.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:59.123+0000 7f24f359e700 1 -- 172.21.15.97:0/3601209568 >> v2:172.21.15.97:6800/1943212787 conn(0x7f24e00391e0 msgr2=0x7f24e003b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:59.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:59.123+0000 7f24f359e700 1 --2- 172.21.15.97:0/3601209568 >> v2:172.21.15.97:6800/1943212787 conn(0x7f24e00391e0 0x7f24e003b6a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f24dc006fd0 tx=0x7f24dc006e40 comp rx=0 tx=0).stop 2024-03-21T13:21:59.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:59.123+0000 7f24f359e700 1 -- 172.21.15.97:0/3601209568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24f41058a0 msgr2=0x7f24f4195860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:21:59.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:59.123+0000 7f24f359e700 1 --2- 172.21.15.97:0/3601209568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24f41058a0 0x7f24f4195860 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f24ec004f80 tx=0x7f24ec005060 comp rx=0 tx=0).stop 2024-03-21T13:21:59.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:59.123+0000 7f24f359e700 1 -- 172.21.15.97:0/3601209568 shutdown_connections 2024-03-21T13:21:59.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:59.123+0000 7f24f359e700 1 --2- 172.21.15.97:0/3601209568 >> v2:172.21.15.97:6800/1943212787 conn(0x7f24e00391e0 0x7f24e003b6a0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:59.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:59.123+0000 7f24f359e700 1 --2- 172.21.15.97:0/3601209568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24f41058a0 0x7f24f4195860 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:21:59.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:59.123+0000 7f24f359e700 1 -- 172.21.15.97:0/3601209568 >> 172.21.15.97:0/3601209568 conn(0x7f24f40fbb20 msgr2=0x7f24f40fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:21:59.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:59.123+0000 7f24f359e700 1 -- 172.21.15.97:0/3601209568 shutdown_connections 2024-03-21T13:21:59.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:21:59.123+0000 7f24f359e700 1 -- 172.21.15.97:0/3601209568 wait complete. 2024-03-21T13:21:59.124 INFO:teuthology.orchestra.run.smithi097.stderr:adjusted tunables profile to default 2024-03-21T13:21:59.463 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:21:59 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3601209568' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-21T13:21:59.791 INFO:tasks.cephadm:Adding mon.a on smithi097 2024-03-21T13:21:59.791 INFO:tasks.cephadm:Adding mon.c on smithi097 2024-03-21T13:21:59.791 INFO:tasks.cephadm:Adding mon.b on smithi144 2024-03-21T13:21:59.792 DEBUG:teuthology.orchestra.run.smithi144:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph orch apply mon '3;smithi097:172.21.15.97=a;smithi097:[v2:172.21.15.97:3301,v1:172.21.15.97:6790]=c;smithi144:172.21.15.144=b' 2024-03-21T13:22:00.027 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T13:22:00.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:00 smithi097 ceph-mon[30040]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:00.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:00 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3601209568' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-21T13:22:00.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:00 smithi097 ceph-mon[30040]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T13:22:00.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:00 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:00.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:00 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:00.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:00 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:00.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:00 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:00.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:00 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi144", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:22:00.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:00 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:00.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:00 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:22:00.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:00 smithi097 ceph-mon[30040]: Updating smithi144:/etc/ceph/ceph.conf 2024-03-21T13:22:00.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:00 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:00.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:00 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:00.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:00 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:01.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:01 smithi097 ceph-mon[30040]: Updating smithi144:/var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T13:22:01.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:01 smithi097 ceph-mon[30040]: Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-03-21T13:22:01.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:01 smithi097 ceph-mon[30040]: Updating smithi144:/var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T13:22:01.577 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:01.577+0000 7f5161fff700 1 -- 172.21.15.144:0/1927555394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f515c101dd0 msgr2=0x7f515c1021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:22:01.577 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:01.577+0000 7f5161fff700 1 --2- 172.21.15.144:0/1927555394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f515c101dd0 0x7f515c1021b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f514c009910 tx=0x7f514c009c20 comp rx=0 tx=0).stop 2024-03-21T13:22:01.577 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:01.577+0000 7f5161fff700 1 -- 172.21.15.144:0/1927555394 shutdown_connections 2024-03-21T13:22:01.578 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:01.577+0000 7f5161fff700 1 --2- 172.21.15.144:0/1927555394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f515c101dd0 0x7f515c1021b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:22:01.578 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:01.577+0000 7f5161fff700 1 -- 172.21.15.144:0/1927555394 >> 172.21.15.144:0/1927555394 conn(0x7f515c0fd6a0 msgr2=0x7f515c0ffac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:22:01.578 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:01.577+0000 7f5161fff700 1 -- 172.21.15.144:0/1927555394 shutdown_connections 2024-03-21T13:22:01.578 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:01.577+0000 7f5161fff700 1 -- 172.21.15.144:0/1927555394 wait complete. 2024-03-21T13:22:01.579 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:01.577+0000 7f5161fff700 1 Processor -- start 2024-03-21T13:22:01.579 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:01.578+0000 7f5161fff700 1 -- start start 2024-03-21T13:22:01.579 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:01.578+0000 7f5161fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f515c101dd0 0x7f515c196d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:22:01.579 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:01.578+0000 7f5161fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f515c10b850 con 0x7f515c101dd0 2024-03-21T13:22:01.580 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:01.578+0000 7f515b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f515c101dd0 0x7f515c196d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:22:01.580 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:01.578+0000 7f515b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f515c101dd0 0x7f515c196d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.144:49112/0 (socket says 172.21.15.144:49112) 2024-03-21T13:22:01.580 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:01.578+0000 7f515b7fe700 1 -- 172.21.15.144:0/484046713 learned_addr learned my addr 172.21.15.144:0/484046713 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-21T13:22:01.580 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:01.579+0000 7f515b7fe700 1 -- 172.21.15.144:0/484046713 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f515c1972d0 con 0x7f515c101dd0 2024-03-21T13:22:01.580 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:01.579+0000 7f515b7fe700 1 --2- 172.21.15.144:0/484046713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f515c101dd0 0x7f515c196d90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f514c004f80 tx=0x7f514c005060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:22:01.580 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:01.579+0000 7f5158ff9700 1 -- 172.21.15.144:0/484046713 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f514c026070 con 0x7f515c101dd0 2024-03-21T13:22:01.580 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:01.579+0000 7f5161fff700 1 -- 172.21.15.144:0/484046713 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f515c1975c0 con 0x7f515c101dd0 2024-03-21T13:22:01.580 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:01.579+0000 7f5158ff9700 1 -- 172.21.15.144:0/484046713 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f514c00bba0 con 0x7f515c101dd0 2024-03-21T13:22:01.580 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:01.579+0000 7f5158ff9700 1 -- 172.21.15.144:0/484046713 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f514c003930 con 0x7f515c101dd0 2024-03-21T13:22:01.580 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:01.579+0000 7f5161fff700 1 -- 172.21.15.144:0/484046713 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f515c19b660 con 0x7f515c101dd0 2024-03-21T13:22:01.581 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:01.580+0000 7f5158ff9700 1 -- 172.21.15.144:0/484046713 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 46315+0+0 (secure 0 0 0) 0x7f514c029070 con 0x7f515c101dd0 2024-03-21T13:22:01.581 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:01.580+0000 7f5161fff700 1 -- 172.21.15.144:0/484046713 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f515c1032b0 con 0x7f515c101dd0 2024-03-21T13:22:01.581 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:01.580+0000 7f5158ff9700 1 --2- 172.21.15.144:0/484046713 >> v2:172.21.15.97:6800/1943212787 conn(0x7f51440391e0 0x7f514403b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:22:01.581 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:01.580+0000 7f5158ff9700 1 -- 172.21.15.144:0/484046713 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f514c01e070 con 0x7f515c101dd0 2024-03-21T13:22:01.581 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:01.580+0000 7f515affd700 1 --2- 172.21.15.144:0/484046713 >> v2:172.21.15.97:6800/1943212787 conn(0x7f51440391e0 0x7f514403b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:22:01.581 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:01.581+0000 7f515affd700 1 --2- 172.21.15.144:0/484046713 >> v2:172.21.15.97:6800/1943212787 conn(0x7f51440391e0 0x7f514403b6a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5150006fd0 tx=0x7f5150006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:22:01.586 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:01.586+0000 7f5158ff9700 1 -- 172.21.15.144:0/484046713 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f514c013070 con 0x7f515c101dd0 2024-03-21T13:22:01.794 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:01.794+0000 7f5161fff700 1 -- 172.21.15.144:0/484046713 --> v2:172.21.15.97:6800/1943212787 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi097:172.21.15.97=a;smithi097:[v2:172.21.15.97:3301,v1:172.21.15.97:6790]=c;smithi144:172.21.15.144=b", "target": ["mon-mgr", ""]}) v1 -- 0x7f515c1980f0 con 0x7f51440391e0 2024-03-21T13:22:01.801 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:01.801+0000 7f5158ff9700 1 -- 172.21.15.144:0/484046713 <== mgr.14152 v2:172.21.15.97:6800/1943212787 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f515c1980f0 con 0x7f51440391e0 2024-03-21T13:22:01.801 INFO:teuthology.orchestra.run.smithi144.stdout:Scheduled mon update... 2024-03-21T13:22:01.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:01.803+0000 7f5161fff700 1 -- 172.21.15.144:0/484046713 >> v2:172.21.15.97:6800/1943212787 conn(0x7f51440391e0 msgr2=0x7f514403b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:22:01.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:01.803+0000 7f5161fff700 1 --2- 172.21.15.144:0/484046713 >> v2:172.21.15.97:6800/1943212787 conn(0x7f51440391e0 0x7f514403b6a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5150006fd0 tx=0x7f5150006e40 comp rx=0 tx=0).stop 2024-03-21T13:22:01.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:01.803+0000 7f5161fff700 1 -- 172.21.15.144:0/484046713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f515c101dd0 msgr2=0x7f515c196d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:22:01.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:01.803+0000 7f5161fff700 1 --2- 172.21.15.144:0/484046713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f515c101dd0 0x7f515c196d90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f514c004f80 tx=0x7f514c005060 comp rx=0 tx=0).stop 2024-03-21T13:22:01.804 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:01.803+0000 7f5161fff700 1 -- 172.21.15.144:0/484046713 shutdown_connections 2024-03-21T13:22:01.804 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:01.803+0000 7f5161fff700 1 --2- 172.21.15.144:0/484046713 >> v2:172.21.15.97:6800/1943212787 conn(0x7f51440391e0 0x7f514403b6a0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:22:01.804 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:01.803+0000 7f5161fff700 1 --2- 172.21.15.144:0/484046713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f515c101dd0 0x7f515c196d90 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:22:01.804 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:01.803+0000 7f5161fff700 1 -- 172.21.15.144:0/484046713 >> 172.21.15.144:0/484046713 conn(0x7f515c0fd6a0 msgr2=0x7f515c1098c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:22:01.804 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:01.803+0000 7f5161fff700 1 -- 172.21.15.144:0/484046713 shutdown_connections 2024-03-21T13:22:01.804 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:01.803+0000 7f5161fff700 1 -- 172.21.15.144:0/484046713 wait complete. 2024-03-21T13:22:02.363 DEBUG:teuthology.orchestra.run.smithi097:mon.c> sudo journalctl -f -n 0 -u ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@mon.c.service 2024-03-21T13:22:02.365 DEBUG:teuthology.orchestra.run.smithi144:mon.b> sudo journalctl -f -n 0 -u ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@mon.b.service 2024-03-21T13:22:02.368 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-21T13:22:02.368 DEBUG:teuthology.orchestra.run.smithi144:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-21T13:22:02.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:02 smithi097 ceph-mon[30040]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:02.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:02 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:02.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:02 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:22:02.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:02 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:02.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:02 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:22:02.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:02 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:02.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:02 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T13:22:02.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:02 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:03.083 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T13:22:03.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:03 smithi097 ceph-mon[30040]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi097:172.21.15.97=a;smithi097:[v2:172.21.15.97:3301,v1:172.21.15.97:6790]=c;smithi144:172.21.15.144=b", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:22:03.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:03 smithi097 ceph-mon[30040]: Saving service mon spec with placement smithi097:172.21.15.97=a;smithi097:[v2:172.21.15.97:3301,v1:172.21.15.97:6790]=c;smithi144:172.21.15.144=b;count:3 2024-03-21T13:22:03.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:03 smithi097 ceph-mon[30040]: Deploying daemon mon.b on smithi144 2024-03-21T13:22:04.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:04 smithi097 ceph-mon[30040]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:05.890 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:05.888+0000 7f00ef788700 1 -- 172.21.15.144:0/4069283074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00e8106a50 msgr2=0x7f00e8106e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:22:05.890 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:05.888+0000 7f00ef788700 1 --2- 172.21.15.144:0/4069283074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00e8106a50 0x7f00e8106e30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f00d8009910 tx=0x7f00d8009c20 comp rx=0 tx=0).stop 2024-03-21T13:22:05.890 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:05.888+0000 7f00ef788700 1 -- 172.21.15.144:0/4069283074 shutdown_connections 2024-03-21T13:22:05.890 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:05.888+0000 7f00ef788700 1 --2- 172.21.15.144:0/4069283074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00e8106a50 0x7f00e8106e30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:22:05.890 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:05.888+0000 7f00ef788700 1 -- 172.21.15.144:0/4069283074 >> 172.21.15.144:0/4069283074 conn(0x7f00e80fc540 msgr2=0x7f00e80fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:22:05.891 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:05.888+0000 7f00ef788700 1 -- 172.21.15.144:0/4069283074 shutdown_connections 2024-03-21T13:22:05.891 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:05.888+0000 7f00ef788700 1 -- 172.21.15.144:0/4069283074 wait complete. 2024-03-21T13:22:05.891 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:05.888+0000 7f00ef788700 1 Processor -- start 2024-03-21T13:22:05.891 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:05.888+0000 7f00ef788700 1 -- start start 2024-03-21T13:22:05.891 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:05.889+0000 7f00ef788700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00e8106a50 0x7f00e8196d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:22:05.891 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:05.889+0000 7f00ef788700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e810a750 con 0x7f00e8106a50 2024-03-21T13:22:05.891 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:05.889+0000 7f00ed524700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00e8106a50 0x7f00e8196d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:22:05.891 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:05.889+0000 7f00ed524700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00e8106a50 0x7f00e8196d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.144:49120/0 (socket says 172.21.15.144:49120) 2024-03-21T13:22:05.891 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:05.889+0000 7f00ed524700 1 -- 172.21.15.144:0/1977632081 learned_addr learned my addr 172.21.15.144:0/1977632081 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-21T13:22:05.891 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:05.889+0000 7f00ed524700 1 -- 172.21.15.144:0/1977632081 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00e81972d0 con 0x7f00e8106a50 2024-03-21T13:22:05.891 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:05.889+0000 7f00ed524700 1 --2- 172.21.15.144:0/1977632081 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00e8106a50 0x7f00e8196d90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f00d8004f80 tx=0x7f00d8005060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:22:05.891 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:05.889+0000 7f00de7fc700 1 -- 172.21.15.144:0/1977632081 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f00d8026070 con 0x7f00e8106a50 2024-03-21T13:22:05.891 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:05.890+0000 7f00de7fc700 1 -- 172.21.15.144:0/1977632081 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f00d800bba0 con 0x7f00e8106a50 2024-03-21T13:22:05.891 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:05.890+0000 7f00ef788700 1 -- 172.21.15.144:0/1977632081 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00e81975c0 con 0x7f00e8106a50 2024-03-21T13:22:05.892 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:05.890+0000 7f00de7fc700 1 -- 172.21.15.144:0/1977632081 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f00d8003930 con 0x7f00e8106a50 2024-03-21T13:22:05.892 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:05.890+0000 7f00ef788700 1 -- 172.21.15.144:0/1977632081 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00e819b660 con 0x7f00e8106a50 2024-03-21T13:22:05.893 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:05.890+0000 7f00de7fc700 1 -- 172.21.15.144:0/1977632081 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 46315+0+0 (secure 0 0 0) 0x7f00d8029070 con 0x7f00e8106a50 2024-03-21T13:22:05.893 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:05.890+0000 7f00ef788700 1 -- 172.21.15.144:0/1977632081 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00e8102190 con 0x7f00e8106a50 2024-03-21T13:22:05.893 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:05.891+0000 7f00de7fc700 1 --2- 172.21.15.144:0/1977632081 >> v2:172.21.15.97:6800/1943212787 conn(0x7f00d40391e0 0x7f00d403b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:22:05.893 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:05.891+0000 7f00de7fc700 1 -- 172.21.15.144:0/1977632081 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f00d801e070 con 0x7f00e8106a50 2024-03-21T13:22:05.893 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:05.891+0000 7f00ecd23700 1 --2- 172.21.15.144:0/1977632081 >> v2:172.21.15.97:6800/1943212787 conn(0x7f00d40391e0 0x7f00d403b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:22:05.893 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:05.891+0000 7f00ecd23700 1 --2- 172.21.15.144:0/1977632081 >> v2:172.21.15.97:6800/1943212787 conn(0x7f00d40391e0 0x7f00d403b6a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f00e4006fd0 tx=0x7f00e4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:22:05.896 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:05.896+0000 7f00de7fc700 1 -- 172.21.15.144:0/1977632081 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f00d8013070 con 0x7f00e8106a50 2024-03-21T13:22:06.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:06.195+0000 7f00ef788700 1 -- 172.21.15.144:0/1977632081 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f00e80637c0 con 0x7f00e8106a50 2024-03-21T13:22:06.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:06.196+0000 7f00de7fc700 1 -- 172.21.15.144:0/1977632081 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+745 (secure 0 0 0) 0x7f00d8030090 con 0x7f00e8106a50 2024-03-21T13:22:06.196 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:22:06.196 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":1,"fsid":"a5dd1f32-e785-11ee-95cd-87774f69a715","modified":"2024-03-21T13:20:33.671111Z","created":"2024-03-21T13:20:33.671111Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:3300","nonce":0},{"type":"v1","addr":"172.21.15.97:6789","nonce":0}]},"addr":"172.21.15.97:6789/0","public_addr":"172.21.15.97:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-21T13:22:06.199 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:06.199+0000 7f00ef788700 1 -- 172.21.15.144:0/1977632081 >> v2:172.21.15.97:6800/1943212787 conn(0x7f00d40391e0 msgr2=0x7f00d403b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:22:06.200 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:06.199+0000 7f00ef788700 1 --2- 172.21.15.144:0/1977632081 >> v2:172.21.15.97:6800/1943212787 conn(0x7f00d40391e0 0x7f00d403b6a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f00e4006fd0 tx=0x7f00e4006e40 comp rx=0 tx=0).stop 2024-03-21T13:22:06.200 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:06.199+0000 7f00ef788700 1 -- 172.21.15.144:0/1977632081 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00e8106a50 msgr2=0x7f00e8196d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:22:06.200 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:06.199+0000 7f00ef788700 1 --2- 172.21.15.144:0/1977632081 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00e8106a50 0x7f00e8196d90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f00d8004f80 tx=0x7f00d8005060 comp rx=0 tx=0).stop 2024-03-21T13:22:06.200 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:06.199+0000 7f00ef788700 1 -- 172.21.15.144:0/1977632081 shutdown_connections 2024-03-21T13:22:06.200 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:06.199+0000 7f00ef788700 1 --2- 172.21.15.144:0/1977632081 >> v2:172.21.15.97:6800/1943212787 conn(0x7f00d40391e0 0x7f00d403b6a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:22:06.200 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:06.199+0000 7f00ef788700 1 --2- 172.21.15.144:0/1977632081 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00e8106a50 0x7f00e8196d90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:22:06.200 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:06.199+0000 7f00ef788700 1 -- 172.21.15.144:0/1977632081 >> 172.21.15.144:0/1977632081 conn(0x7f00e80fc540 msgr2=0x7f00e81087f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:22:06.200 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:06.199+0000 7f00ef788700 1 -- 172.21.15.144:0/1977632081 shutdown_connections 2024-03-21T13:22:06.200 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:06.199+0000 7f00ef788700 1 -- 172.21.15.144:0/1977632081 wait complete. 2024-03-21T13:22:06.200 INFO:teuthology.orchestra.run.smithi144.stderr:dumped monmap epoch 1 2024-03-21T13:22:06.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:06 smithi097 ceph-mon[30040]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:06.987 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:06 smithi144 podman[33595]: 2024-03-21 13:22:06.891248035 +0000 UTC m=+0.018803243 image pull 19e12293512ff7978c8adb5f72a829060dd47f07bfd6903b6b3178d2ebc29e4e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93 2024-03-21T13:22:07.262 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 podman[33595]: 2024-03-21 13:22:07.178670719 +0000 UTC m=+0.306225917 container init 97da8a80ab456ec9d311a685feccc89d5866182051c55185cd5302459da56b00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mon-b, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-21T13:22:07.262 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 podman[33595]: 2024-03-21 13:22:07.184435783 +0000 UTC m=+0.311990982 container start 97da8a80ab456ec9d311a685feccc89d5866182051c55185cd5302459da56b00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mon-b, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-21T13:22:07.262 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: set uid:gid to 167:167 (ceph:ceph) 2024-03-21T13:22:07.262 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: ceph version 19.0.0-2267-g960fbbe4 (960fbbe4645c2becc8084669be194c95183c7753) squid (dev), process ceph-mon, pid 7 2024-03-21T13:22:07.262 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: pidfile_write: ignore empty --pid-file 2024-03-21T13:22:07.263 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: load: jerasure load: lrc 2024-03-21T13:22:07.263 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: RocksDB version: 7.9.2 2024-03-21T13:22:07.263 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Git sha 0 2024-03-21T13:22:07.263 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Compile date 2024-03-21 08:53:26 2024-03-21T13:22:07.263 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: DB SUMMARY 2024-03-21T13:22:07.263 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: DB Session ID: 583B1ADRB5ODK6VQ7RNA 2024-03-21T13:22:07.263 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: CURRENT file: CURRENT 2024-03-21T13:22:07.263 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: IDENTITY file: IDENTITY 2024-03-21T13:22:07.263 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-03-21T13:22:07.263 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-03-21T13:22:07.263 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 511 ; 2024-03-21T13:22:07.263 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.error_if_exists: 0 2024-03-21T13:22:07.263 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.create_if_missing: 0 2024-03-21T13:22:07.263 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.paranoid_checks: 1 2024-03-21T13:22:07.264 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-21T13:22:07.264 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-21T13:22:07.264 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-21T13:22:07.264 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.env: 0x560d9e22db20 2024-03-21T13:22:07.264 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.fs: PosixFileSystem 2024-03-21T13:22:07.264 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.info_log: 0x560d9f10f920 2024-03-21T13:22:07.264 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.max_file_opening_threads: 16 2024-03-21T13:22:07.266 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.statistics: (nil) 2024-03-21T13:22:07.266 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.use_fsync: 0 2024-03-21T13:22:07.266 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.max_log_file_size: 0 2024-03-21T13:22:07.266 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-21T13:22:07.266 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-21T13:22:07.266 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.keep_log_file_num: 1000 2024-03-21T13:22:07.266 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.recycle_log_file_num: 0 2024-03-21T13:22:07.266 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.allow_fallocate: 1 2024-03-21T13:22:07.266 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.allow_mmap_reads: 0 2024-03-21T13:22:07.266 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.allow_mmap_writes: 0 2024-03-21T13:22:07.266 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.use_direct_reads: 0 2024-03-21T13:22:07.266 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-21T13:22:07.266 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.create_missing_column_families: 0 2024-03-21T13:22:07.267 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.db_log_dir: 2024-03-21T13:22:07.267 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.wal_dir: 2024-03-21T13:22:07.267 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-21T13:22:07.267 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-21T13:22:07.267 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-21T13:22:07.267 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-21T13:22:07.267 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-21T13:22:07.267 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-21T13:22:07.267 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.advise_random_on_open: 1 2024-03-21T13:22:07.267 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.db_write_buffer_size: 0 2024-03-21T13:22:07.267 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.write_buffer_manager: 0x560d9f0e01e0 2024-03-21T13:22:07.267 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-21T13:22:07.267 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-21T13:22:07.267 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-21T13:22:07.267 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.rate_limiter: (nil) 2024-03-21T13:22:07.268 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-21T13:22:07.268 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.wal_recovery_mode: 2 2024-03-21T13:22:07.268 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.enable_thread_tracking: 0 2024-03-21T13:22:07.268 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.enable_pipelined_write: 0 2024-03-21T13:22:07.268 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.unordered_write: 0 2024-03-21T13:22:07.268 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-21T13:22:07.268 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-21T13:22:07.268 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-21T13:22:07.268 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-21T13:22:07.268 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.row_cache: None 2024-03-21T13:22:07.268 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.wal_filter: None 2024-03-21T13:22:07.268 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-21T13:22:07.268 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.allow_ingest_behind: 0 2024-03-21T13:22:07.268 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.two_write_queues: 0 2024-03-21T13:22:07.268 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.manual_wal_flush: 0 2024-03-21T13:22:07.269 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.wal_compression: 0 2024-03-21T13:22:07.269 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.atomic_flush: 0 2024-03-21T13:22:07.269 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-21T13:22:07.269 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-21T13:22:07.269 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-21T13:22:07.269 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.log_readahead_size: 0 2024-03-21T13:22:07.269 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-21T13:22:07.269 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.best_efforts_recovery: 0 2024-03-21T13:22:07.269 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-21T13:22:07.269 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-21T13:22:07.269 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.allow_data_in_errors: 0 2024-03-21T13:22:07.269 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.db_host_id: __hostname__ 2024-03-21T13:22:07.269 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-21T13:22:07.269 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.max_background_jobs: 2 2024-03-21T13:22:07.269 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.max_background_compactions: -1 2024-03-21T13:22:07.270 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.max_subcompactions: 1 2024-03-21T13:22:07.270 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-21T13:22:07.270 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-21T13:22:07.270 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-21T13:22:07.270 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.max_total_wal_size: 0 2024-03-21T13:22:07.270 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-21T13:22:07.270 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-21T13:22:07.270 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-21T13:22:07.270 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-21T13:22:07.270 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.max_open_files: -1 2024-03-21T13:22:07.270 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.bytes_per_sync: 0 2024-03-21T13:22:07.270 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-21T13:22:07.270 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-21T13:22:07.270 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.compaction_readahead_size: 0 2024-03-21T13:22:07.270 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.max_background_flushes: -1 2024-03-21T13:22:07.271 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Compression algorithms supported: 2024-03-21T13:22:07.271 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-21T13:22:07.271 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: kZSTD supported: 0 2024-03-21T13:22:07.271 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: kXpressCompression supported: 0 2024-03-21T13:22:07.271 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: kLZ4HCCompression supported: 1 2024-03-21T13:22:07.271 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: kZlibCompression supported: 1 2024-03-21T13:22:07.271 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: kSnappyCompression supported: 1 2024-03-21T13:22:07.271 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: kLZ4Compression supported: 1 2024-03-21T13:22:07.271 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: kBZip2Compression supported: 0 2024-03-21T13:22:07.271 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-21T13:22:07.271 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-21T13:22:07.271 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-03-21T13:22:07.271 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-21T13:22:07.271 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-21T13:22:07.272 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.merge_operator: 2024-03-21T13:22:07.272 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.compaction_filter: None 2024-03-21T13:22:07.272 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.compaction_filter_factory: None 2024-03-21T13:22:07.272 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.sst_partitioner_factory: None 2024-03-21T13:22:07.272 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-21T13:22:07.272 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-21T13:22:07.272 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x560d9f10fa20) 2024-03-21T13:22:07.272 INFO:journalctl@ceph.mon.b.smithi144.stdout: cache_index_and_filter_blocks: 1 2024-03-21T13:22:07.272 INFO:journalctl@ceph.mon.b.smithi144.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-21T13:22:07.272 INFO:journalctl@ceph.mon.b.smithi144.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-21T13:22:07.272 INFO:journalctl@ceph.mon.b.smithi144.stdout: pin_top_level_index_and_filter: 1 2024-03-21T13:22:07.272 INFO:journalctl@ceph.mon.b.smithi144.stdout: index_type: 0 2024-03-21T13:22:07.272 INFO:journalctl@ceph.mon.b.smithi144.stdout: data_block_index_type: 0 2024-03-21T13:22:07.272 INFO:journalctl@ceph.mon.b.smithi144.stdout: index_shortening: 1 2024-03-21T13:22:07.272 INFO:journalctl@ceph.mon.b.smithi144.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-21T13:22:07.272 INFO:journalctl@ceph.mon.b.smithi144.stdout: checksum: 4 2024-03-21T13:22:07.273 INFO:journalctl@ceph.mon.b.smithi144.stdout: no_block_cache: 0 2024-03-21T13:22:07.273 INFO:journalctl@ceph.mon.b.smithi144.stdout: block_cache: 0x560d9f113090 2024-03-21T13:22:07.273 INFO:journalctl@ceph.mon.b.smithi144.stdout: block_cache_name: BinnedLRUCache 2024-03-21T13:22:07.273 INFO:journalctl@ceph.mon.b.smithi144.stdout: block_cache_options: 2024-03-21T13:22:07.273 INFO:journalctl@ceph.mon.b.smithi144.stdout: capacity : 536870912 2024-03-21T13:22:07.273 INFO:journalctl@ceph.mon.b.smithi144.stdout: num_shard_bits : 4 2024-03-21T13:22:07.273 INFO:journalctl@ceph.mon.b.smithi144.stdout: strict_capacity_limit : 0 2024-03-21T13:22:07.273 INFO:journalctl@ceph.mon.b.smithi144.stdout: high_pri_pool_ratio: 0.000 2024-03-21T13:22:07.273 INFO:journalctl@ceph.mon.b.smithi144.stdout: block_cache_compressed: (nil) 2024-03-21T13:22:07.273 INFO:journalctl@ceph.mon.b.smithi144.stdout: persistent_cache: (nil) 2024-03-21T13:22:07.273 INFO:journalctl@ceph.mon.b.smithi144.stdout: block_size: 4096 2024-03-21T13:22:07.273 INFO:journalctl@ceph.mon.b.smithi144.stdout: block_size_deviation: 10 2024-03-21T13:22:07.273 INFO:journalctl@ceph.mon.b.smithi144.stdout: block_restart_interval: 16 2024-03-21T13:22:07.273 INFO:journalctl@ceph.mon.b.smithi144.stdout: index_block_restart_interval: 1 2024-03-21T13:22:07.273 INFO:journalctl@ceph.mon.b.smithi144.stdout: metadata_block_size: 4096 2024-03-21T13:22:07.274 INFO:journalctl@ceph.mon.b.smithi144.stdout: partition_filters: 0 2024-03-21T13:22:07.274 INFO:journalctl@ceph.mon.b.smithi144.stdout: use_delta_encoding: 1 2024-03-21T13:22:07.274 INFO:journalctl@ceph.mon.b.smithi144.stdout: filter_policy: bloomfilter 2024-03-21T13:22:07.274 INFO:journalctl@ceph.mon.b.smithi144.stdout: whole_key_filtering: 1 2024-03-21T13:22:07.274 INFO:journalctl@ceph.mon.b.smithi144.stdout: verify_compression: 0 2024-03-21T13:22:07.274 INFO:journalctl@ceph.mon.b.smithi144.stdout: read_amp_bytes_per_bit: 0 2024-03-21T13:22:07.274 INFO:journalctl@ceph.mon.b.smithi144.stdout: format_version: 5 2024-03-21T13:22:07.274 INFO:journalctl@ceph.mon.b.smithi144.stdout: enable_index_compression: 1 2024-03-21T13:22:07.274 INFO:journalctl@ceph.mon.b.smithi144.stdout: block_align: 0 2024-03-21T13:22:07.274 INFO:journalctl@ceph.mon.b.smithi144.stdout: max_auto_readahead_size: 262144 2024-03-21T13:22:07.274 INFO:journalctl@ceph.mon.b.smithi144.stdout: prepopulate_block_cache: 0 2024-03-21T13:22:07.274 INFO:journalctl@ceph.mon.b.smithi144.stdout: initial_auto_readahead_size: 8192 2024-03-21T13:22:07.274 INFO:journalctl@ceph.mon.b.smithi144.stdout: num_file_reads_for_auto_readahead: 2 2024-03-21T13:22:07.274 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.write_buffer_size: 33554432 2024-03-21T13:22:07.274 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.max_write_buffer_number: 2 2024-03-21T13:22:07.275 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.compression: NoCompression 2024-03-21T13:22:07.275 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.bottommost_compression: Disabled 2024-03-21T13:22:07.275 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.prefix_extractor: nullptr 2024-03-21T13:22:07.275 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-21T13:22:07.275 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.num_levels: 7 2024-03-21T13:22:07.275 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-21T13:22:07.275 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-21T13:22:07.275 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-21T13:22:07.275 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-21T13:22:07.275 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-21T13:22:07.275 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-21T13:22:07.275 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-21T13:22:07.275 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-21T13:22:07.275 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-21T13:22:07.275 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-21T13:22:07.276 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-21T13:22:07.276 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-21T13:22:07.276 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-21T13:22:07.276 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.compression_opts.level: 32767 2024-03-21T13:22:07.276 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.compression_opts.strategy: 0 2024-03-21T13:22:07.276 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-21T13:22:07.276 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-21T13:22:07.276 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-21T13:22:07.276 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-21T13:22:07.276 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.compression_opts.enabled: false 2024-03-21T13:22:07.276 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-21T13:22:07.276 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-21T13:22:07.276 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-21T13:22:07.276 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-21T13:22:07.276 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.target_file_size_base: 67108864 2024-03-21T13:22:07.277 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-21T13:22:07.277 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-21T13:22:07.277 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-21T13:22:07.277 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-21T13:22:07.277 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-21T13:22:07.277 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-21T13:22:07.277 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-21T13:22:07.277 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-21T13:22:07.277 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-21T13:22:07.277 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-21T13:22:07.277 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-21T13:22:07.277 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-21T13:22:07.277 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-21T13:22:07.277 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-21T13:22:07.278 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.arena_block_size: 1048576 2024-03-21T13:22:07.278 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-21T13:22:07.278 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-21T13:22:07.278 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.disable_auto_compactions: 0 2024-03-21T13:22:07.278 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-21T13:22:07.278 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-21T13:22:07.278 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-21T13:22:07.278 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-21T13:22:07.278 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-21T13:22:07.278 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-21T13:22:07.278 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-21T13:22:07.278 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-21T13:22:07.278 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-21T13:22:07.278 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-21T13:22:07.278 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.table_properties_collectors: 2024-03-21T13:22:07.279 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.inplace_update_support: 0 2024-03-21T13:22:07.279 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-21T13:22:07.279 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-21T13:22:07.279 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-21T13:22:07.279 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-21T13:22:07.279 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.bloom_locality: 0 2024-03-21T13:22:07.279 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.max_successive_merges: 0 2024-03-21T13:22:07.279 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-21T13:22:07.279 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.paranoid_file_checks: 0 2024-03-21T13:22:07.279 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.force_consistency_checks: 1 2024-03-21T13:22:07.279 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.report_bg_io_stats: 0 2024-03-21T13:22:07.279 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.ttl: 2592000 2024-03-21T13:22:07.279 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-21T13:22:07.279 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-21T13:22:07.279 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-21T13:22:07.280 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.enable_blob_files: false 2024-03-21T13:22:07.280 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.min_blob_size: 0 2024-03-21T13:22:07.280 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.blob_file_size: 268435456 2024-03-21T13:22:07.280 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-21T13:22:07.280 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-21T13:22:07.280 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-21T13:22:07.280 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-21T13:22:07.280 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-21T13:22:07.280 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.blob_file_starting_level: 0 2024-03-21T13:22:07.280 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-21T13:22:07.280 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-21T13:22:07.280 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-21T13:22:07.280 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 38dc7f44-f24b-4938-8223-666ddbdf648d 2024-03-21T13:22:07.280 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711027327214331, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-21T13:22:07.280 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-21T13:22:07.281 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711027327214739, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711027327, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "38dc7f44-f24b-4938-8223-666ddbdf648d", "db_session_id": "583B1ADRB5ODK6VQ7RNA", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-21T13:22:07.281 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711027327214817, "job": 1, "event": "recovery_finished"} 2024-03-21T13:22:07.281 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-21T13:22:07.281 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-21T13:22:07.281 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x560d9f202000 2024-03-21T13:22:07.281 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: DB pointer 0x560d9f1ec000 2024-03-21T13:22:07.281 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-21T13:22:07.281 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-21T13:22:07.281 INFO:journalctl@ceph.mon.b.smithi144.stdout: ** DB Stats ** 2024-03-21T13:22:07.281 INFO:journalctl@ceph.mon.b.smithi144.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-21T13:22:07.281 INFO:journalctl@ceph.mon.b.smithi144.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-21T13:22:07.281 INFO:journalctl@ceph.mon.b.smithi144.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-21T13:22:07.281 INFO:journalctl@ceph.mon.b.smithi144.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-21T13:22:07.281 INFO:journalctl@ceph.mon.b.smithi144.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-21T13:22:07.282 INFO:journalctl@ceph.mon.b.smithi144.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-21T13:22:07.282 INFO:journalctl@ceph.mon.b.smithi144.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-21T13:22:07.282 INFO:journalctl@ceph.mon.b.smithi144.stdout: 2024-03-21T13:22:07.282 INFO:journalctl@ceph.mon.b.smithi144.stdout: ** Compaction Stats [default] ** 2024-03-21T13:22:07.282 INFO:journalctl@ceph.mon.b.smithi144.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-21T13:22:07.282 INFO:journalctl@ceph.mon.b.smithi144.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-21T13:22:07.282 INFO:journalctl@ceph.mon.b.smithi144.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-21T13:22:07.282 INFO:journalctl@ceph.mon.b.smithi144.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-21T13:22:07.282 INFO:journalctl@ceph.mon.b.smithi144.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-21T13:22:07.282 INFO:journalctl@ceph.mon.b.smithi144.stdout: 2024-03-21T13:22:07.282 INFO:journalctl@ceph.mon.b.smithi144.stdout: ** Compaction Stats [default] ** 2024-03-21T13:22:07.282 INFO:journalctl@ceph.mon.b.smithi144.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-21T13:22:07.282 INFO:journalctl@ceph.mon.b.smithi144.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-21T13:22:07.282 INFO:journalctl@ceph.mon.b.smithi144.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-21T13:22:07.282 INFO:journalctl@ceph.mon.b.smithi144.stdout: 2024-03-21T13:22:07.283 INFO:journalctl@ceph.mon.b.smithi144.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-21T13:22:07.283 INFO:journalctl@ceph.mon.b.smithi144.stdout: 2024-03-21T13:22:07.283 INFO:journalctl@ceph.mon.b.smithi144.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-21T13:22:07.283 INFO:journalctl@ceph.mon.b.smithi144.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-21T13:22:07.283 INFO:journalctl@ceph.mon.b.smithi144.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-21T13:22:07.283 INFO:journalctl@ceph.mon.b.smithi144.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-21T13:22:07.283 INFO:journalctl@ceph.mon.b.smithi144.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-21T13:22:07.283 INFO:journalctl@ceph.mon.b.smithi144.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-21T13:22:07.283 INFO:journalctl@ceph.mon.b.smithi144.stdout: Cumulative compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-21T13:22:07.283 INFO:journalctl@ceph.mon.b.smithi144.stdout: Interval compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-21T13:22:07.283 INFO:journalctl@ceph.mon.b.smithi144.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-21T13:22:07.283 INFO:journalctl@ceph.mon.b.smithi144.stdout: Block cache BinnedLRUCache@0x560d9f113090#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-03-21T13:22:07.283 INFO:journalctl@ceph.mon.b.smithi144.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-21T13:22:07.283 INFO:journalctl@ceph.mon.b.smithi144.stdout: 2024-03-21T13:22:07.284 INFO:journalctl@ceph.mon.b.smithi144.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-21T13:22:07.284 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-03-21T13:22:07.284 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: using public_addr v2:172.21.15.144:0/0 -> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] 2024-03-21T13:22:07.284 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: starting mon.b rank -1 at public addrs [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] at bind addrs [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid a5dd1f32-e785-11ee-95cd-87774f69a715 2024-03-21T13:22:07.284 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: mon.b@-1(???) e0 preinit fsid a5dd1f32-e785-11ee-95cd-87774f69a715 2024-03-21T13:22:07.284 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: mon.b@-1(synchronizing).mds e1 new map 2024-03-21T13:22:07.284 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: mon.b@-1(synchronizing).mds e1 print_map 2024-03-21T13:22:07.284 INFO:journalctl@ceph.mon.b.smithi144.stdout: e1 2024-03-21T13:22:07.284 INFO:journalctl@ceph.mon.b.smithi144.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T13:22:07.284 INFO:journalctl@ceph.mon.b.smithi144.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-21T13:22:07.284 INFO:journalctl@ceph.mon.b.smithi144.stdout: legacy client fscid: -1 2024-03-21T13:22:07.284 INFO:journalctl@ceph.mon.b.smithi144.stdout: 2024-03-21T13:22:07.284 INFO:journalctl@ceph.mon.b.smithi144.stdout: No filesystems configured 2024-03-21T13:22:07.284 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-21T13:22:07.284 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-21T13:22:07.284 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-21T13:22:07.285 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-21T13:22:07.285 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-21T13:22:07.285 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-21T13:22:07.285 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-21T13:22:07.285 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T13:22:07.285 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T13:22:07.285 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T13:22:07.285 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:07.285 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:07.285 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:07.285 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T13:22:07.285 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:07.285 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3601209568' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-21T13:22:07.285 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:07.285 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3601209568' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-21T13:22:07.286 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T13:22:07.286 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:07.286 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:07.286 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:07.286 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:07.286 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi144", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:22:07.286 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:07.286 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:22:07.286 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: Updating smithi144:/etc/ceph/ceph.conf 2024-03-21T13:22:07.286 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:07.286 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:07.286 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:07.286 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: Updating smithi144:/var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T13:22:07.286 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-03-21T13:22:07.287 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: Updating smithi144:/var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T13:22:07.287 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:07.287 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:07.287 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:22:07.287 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:07.287 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:22:07.287 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:07.287 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T13:22:07.287 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:07.287 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi097:172.21.15.97=a;smithi097:[v2:172.21.15.97:3301,v1:172.21.15.97:6790]=c;smithi144:172.21.15.144=b", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:22:07.287 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: Saving service mon spec with placement smithi097:172.21.15.97=a;smithi097:[v2:172.21.15.97:3301,v1:172.21.15.97:6790]=c;smithi144:172.21.15.144=b;count:3 2024-03-21T13:22:07.287 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: Deploying daemon mon.b on smithi144 2024-03-21T13:22:07.287 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:07.287 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:07.287 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: from='client.? 172.21.15.144:0/1977632081' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T13:22:07.288 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 ceph-mon[33633]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-21T13:22:07.288 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 bash[33595]: 97da8a80ab456ec9d311a685feccc89d5866182051c55185cd5302459da56b00 2024-03-21T13:22:07.288 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:07 smithi144 systemd[1]: Started Ceph mon.b for a5dd1f32-e785-11ee-95cd-87774f69a715. 2024-03-21T13:22:07.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:07 smithi097 ceph-mon[30040]: from='client.? 172.21.15.144:0/1977632081' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T13:22:07.728 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-21T13:22:07.728 DEBUG:teuthology.orchestra.run.smithi144:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-21T13:22:07.967 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T13:22:12.503 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:12.500+0000 7f4fbf7c6700 1 -- 172.21.15.144:0/989389796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fa40035c0 msgr2=0x7f4fa4005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:22:12.503 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:12.500+0000 7f4fbf7c6700 1 --2- 172.21.15.144:0/989389796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fa40035c0 0x7f4fa4005a50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8100a90 tx=0x7f4fa800feb0 comp rx=0 tx=0).stop 2024-03-21T13:22:12.503 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:12.500+0000 7f4fbf7c6700 1 -- 172.21.15.144:0/989389796 shutdown_connections 2024-03-21T13:22:12.503 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:12.500+0000 7f4fbf7c6700 1 --2- 172.21.15.144:0/989389796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fa40035c0 0x7f4fa4005a50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:22:12.503 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:12.500+0000 7f4fbf7c6700 1 -- 172.21.15.144:0/989389796 >> 172.21.15.144:0/989389796 conn(0x7f4fb80fc630 msgr2=0x7f4fb80fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:22:12.503 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:12.500+0000 7f4fbf7c6700 1 -- 172.21.15.144:0/989389796 shutdown_connections 2024-03-21T13:22:12.503 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:12.500+0000 7f4fbf7c6700 1 -- 172.21.15.144:0/989389796 wait complete. 2024-03-21T13:22:12.503 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:12.501+0000 7f4fbf7c6700 1 Processor -- start 2024-03-21T13:22:12.504 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:12.501+0000 7f4fbf7c6700 1 -- start start 2024-03-21T13:22:12.504 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:12.501+0000 7f4fbf7c6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fa40035c0 0x7f4fb8196f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:22:12.504 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:12.501+0000 7f4fbf7c6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fb81974a0 0x7f4fb819b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:22:12.504 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:12.501+0000 7f4fbf7c6700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb81082f0 con 0x7f4fb81974a0 2024-03-21T13:22:12.504 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:12.501+0000 7f4fbf7c6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb8108170 con 0x7f4fa40035c0 2024-03-21T13:22:12.504 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:12.501+0000 7f4fbd562700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fa40035c0 0x7f4fb8196f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:22:12.504 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:12.501+0000 7f4fbd562700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fa40035c0 0x7f4fb8196f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:56418/0 (socket says 172.21.15.144:56418) 2024-03-21T13:22:12.504 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:12.501+0000 7f4fbd562700 1 -- 172.21.15.144:0/2205476391 learned_addr learned my addr 172.21.15.144:0/2205476391 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-21T13:22:12.504 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:12.502+0000 7f4fbcd61700 1 --2- 172.21.15.144:0/2205476391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fb81974a0 0x7f4fb819b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:22:12.504 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:12.502+0000 7f4fbd562700 1 -- 172.21.15.144:0/2205476391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fa40035c0 msgr2=0x7f4fb8196f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-21T13:22:12.504 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:12.502+0000 7f4fbd562700 1 -- 172.21.15.144:0/2205476391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fa40035c0 msgr2=0x7f4fb8196f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-21T13:22:12.505 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:12.502+0000 7f4fbd562700 1 --2- 172.21.15.144:0/2205476391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fa40035c0 0x7f4fb8196f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-21T13:22:12.505 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:12.502+0000 7f4fbd562700 1 --2- 172.21.15.144:0/2205476391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fa40035c0 0x7f4fb8196f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T13:22:12.505 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:12.502+0000 7f4fbcd61700 1 -- 172.21.15.144:0/2205476391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fa40035c0 msgr2=0x7f4fb8196f60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T13:22:12.505 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:12.502+0000 7f4fbcd61700 1 --2- 172.21.15.144:0/2205476391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fa40035c0 0x7f4fb8196f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:22:12.505 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:12.502+0000 7f4fbcd61700 1 -- 172.21.15.144:0/2205476391 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb8197ae0 con 0x7f4fb81974a0 2024-03-21T13:22:12.505 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:12.502+0000 7f4fbcd61700 1 --2- 172.21.15.144:0/2205476391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fb81974a0 0x7f4fb819b830 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4fb400b4d0 tx=0x7f4fb400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:22:12.505 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:12.502+0000 7f4fae7fc700 1 -- 172.21.15.144:0/2205476391 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f4fb4011630 con 0x7f4fb81974a0 2024-03-21T13:22:12.505 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:12.502+0000 7f4fae7fc700 1 -- 172.21.15.144:0/2205476391 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f4fb4011c80 con 0x7f4fb81974a0 2024-03-21T13:22:12.505 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:12.502+0000 7f4fbf7c6700 1 -- 172.21.15.144:0/2205476391 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fb819bd70 con 0x7f4fb81974a0 2024-03-21T13:22:12.505 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:12.502+0000 7f4fae7fc700 1 -- 172.21.15.144:0/2205476391 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f4fb4010d70 con 0x7f4fb81974a0 2024-03-21T13:22:12.505 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:12.503+0000 7f4fbf7c6700 1 -- 172.21.15.144:0/2205476391 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fb819c2d0 con 0x7f4fb81974a0 2024-03-21T13:22:12.506 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:12.503+0000 7f4fae7fc700 1 -- 172.21.15.144:0/2205476391 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 46315+0+0 (secure 0 0 0) 0x7f4fb4018020 con 0x7f4fb81974a0 2024-03-21T13:22:12.506 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:12.503+0000 7f4fae7fc700 1 --2- 172.21.15.144:0/2205476391 >> v2:172.21.15.97:6800/1943212787 conn(0x7f4f9c038fd0 0x7f4f9c03b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:22:12.506 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:12.504+0000 7f4fae7fc700 1 -- 172.21.15.144:0/2205476391 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f4fb404bbc0 con 0x7f4fb81974a0 2024-03-21T13:22:12.507 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:12.504+0000 7f4fbf7c6700 1 -- 172.21.15.144:0/2205476391 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fb81022a0 con 0x7f4fb81974a0 2024-03-21T13:22:12.507 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:12.504+0000 7f4fbd562700 1 --2- 172.21.15.144:0/2205476391 >> v2:172.21.15.97:6800/1943212787 conn(0x7f4f9c038fd0 0x7f4f9c03b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:22:12.507 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:12.504+0000 7f4fbd562700 1 --2- 172.21.15.144:0/2205476391 >> v2:172.21.15.97:6800/1943212787 conn(0x7f4f9c038fd0 0x7f4f9c03b490 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4fa800fa60 tx=0x7f4fa8011470 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:22:12.509 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:12.509+0000 7f4fae7fc700 1 -- 172.21.15.144:0/2205476391 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4fb401f020 con 0x7f4fb81974a0 2024-03-21T13:22:12.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:12 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T13:22:12.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:12 smithi144 ceph-mon[33633]: mon.a calling monitor election 2024-03-21T13:22:12.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:12 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T13:22:12.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:12 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T13:22:12.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:12 smithi144 ceph-mon[33633]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:12.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:12 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T13:22:12.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:12 smithi144 ceph-mon[33633]: mon.b calling monitor election 2024-03-21T13:22:12.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:12 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T13:22:12.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:12 smithi144 ceph-mon[33633]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:12.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:12 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T13:22:12.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:12 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T13:22:12.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:12 smithi144 ceph-mon[33633]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-03-21T13:22:12.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:12 smithi144 ceph-mon[33633]: monmap e2: 2 mons at {a=[v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0],b=[v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T13:22:12.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:12 smithi144 ceph-mon[33633]: fsmap 2024-03-21T13:22:12.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:12 smithi144 ceph-mon[33633]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T13:22:12.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:12 smithi144 ceph-mon[33633]: mgrmap e13: y(active, since 41s) 2024-03-21T13:22:12.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:12 smithi144 ceph-mon[33633]: overall HEALTH_OK 2024-03-21T13:22:12.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:12 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:12.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:12 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:12.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:12 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:12.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:12 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T13:22:12.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:12 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:12.788 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:12.788+0000 7f4fbf7c6700 1 -- 172.21.15.144:0/2205476391 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4fb819c5e0 con 0x7f4fb81974a0 2024-03-21T13:22:12.789 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:12.788+0000 7f4fae7fc700 1 -- 172.21.15.144:0/2205476391 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1015 (secure 0 0 0) 0x7f4fb4018570 con 0x7f4fb81974a0 2024-03-21T13:22:12.789 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:22:12.789 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":2,"fsid":"a5dd1f32-e785-11ee-95cd-87774f69a715","modified":"2024-03-21T13:22:07.234596Z","created":"2024-03-21T13:20:33.671111Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:3300","nonce":0},{"type":"v1","addr":"172.21.15.97:6789","nonce":0}]},"addr":"172.21.15.97:6789/0","public_addr":"172.21.15.97:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:3300","nonce":0},{"type":"v1","addr":"172.21.15.144:6789","nonce":0}]},"addr":"172.21.15.144:6789/0","public_addr":"172.21.15.144:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-21T13:22:12.791 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:12.790+0000 7f4fbf7c6700 1 -- 172.21.15.144:0/2205476391 >> v2:172.21.15.97:6800/1943212787 conn(0x7f4f9c038fd0 msgr2=0x7f4f9c03b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:22:12.791 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:12.790+0000 7f4fbf7c6700 1 --2- 172.21.15.144:0/2205476391 >> v2:172.21.15.97:6800/1943212787 conn(0x7f4f9c038fd0 0x7f4f9c03b490 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4fa800fa60 tx=0x7f4fa8011470 comp rx=0 tx=0).stop 2024-03-21T13:22:12.791 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:12.790+0000 7f4fbf7c6700 1 -- 172.21.15.144:0/2205476391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fb81974a0 msgr2=0x7f4fb819b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:22:12.791 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:12.790+0000 7f4fbf7c6700 1 --2- 172.21.15.144:0/2205476391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fb81974a0 0x7f4fb819b830 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4fb400b4d0 tx=0x7f4fb400b890 comp rx=0 tx=0).stop 2024-03-21T13:22:12.791 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:12.790+0000 7f4fbf7c6700 1 -- 172.21.15.144:0/2205476391 shutdown_connections 2024-03-21T13:22:12.791 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:12.791+0000 7f4fbf7c6700 1 --2- 172.21.15.144:0/2205476391 >> v2:172.21.15.97:6800/1943212787 conn(0x7f4f9c038fd0 0x7f4f9c03b490 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:22:12.791 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:12.791+0000 7f4fbf7c6700 1 --2- 172.21.15.144:0/2205476391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fa40035c0 0x7f4fb8196f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:22:12.791 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:12.791+0000 7f4fbf7c6700 1 --2- 172.21.15.144:0/2205476391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fb81974a0 0x7f4fb819b830 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:22:12.791 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:12.791+0000 7f4fbf7c6700 1 -- 172.21.15.144:0/2205476391 >> 172.21.15.144:0/2205476391 conn(0x7f4fb80fc630 msgr2=0x7f4fb806ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:22:12.791 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:12.791+0000 7f4fbf7c6700 1 -- 172.21.15.144:0/2205476391 shutdown_connections 2024-03-21T13:22:12.791 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:12.791+0000 7f4fbf7c6700 1 -- 172.21.15.144:0/2205476391 wait complete. 2024-03-21T13:22:12.792 INFO:teuthology.orchestra.run.smithi144.stderr:dumped monmap epoch 2 2024-03-21T13:22:12.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:12 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T13:22:12.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:12 smithi097 ceph-mon[30040]: mon.a calling monitor election 2024-03-21T13:22:12.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:12 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T13:22:12.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:12 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T13:22:12.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:12 smithi097 ceph-mon[30040]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:12.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:12 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T13:22:12.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:12 smithi097 ceph-mon[30040]: mon.b calling monitor election 2024-03-21T13:22:12.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:12 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T13:22:12.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:12 smithi097 ceph-mon[30040]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:12.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:12 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T13:22:12.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:12 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T13:22:12.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:12 smithi097 ceph-mon[30040]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-03-21T13:22:12.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:12 smithi097 ceph-mon[30040]: monmap e2: 2 mons at {a=[v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0],b=[v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T13:22:12.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:12 smithi097 ceph-mon[30040]: fsmap 2024-03-21T13:22:12.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:12 smithi097 ceph-mon[30040]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T13:22:12.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:12 smithi097 ceph-mon[30040]: mgrmap e13: y(active, since 41s) 2024-03-21T13:22:12.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:12 smithi097 ceph-mon[30040]: overall HEALTH_OK 2024-03-21T13:22:12.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:12 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:12.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:12 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:12.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:12 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:12.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:12 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T13:22:12.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:12 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:13.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:13 smithi144 ceph-mon[33633]: Deploying daemon mon.c on smithi097 2024-03-21T13:22:13.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:13 smithi144 ceph-mon[33633]: from='client.? 172.21.15.144:0/2205476391' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T13:22:13.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:13 smithi144 ceph-mon[33633]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:13.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:13 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T13:22:13.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:13 smithi097 ceph-mon[30040]: Deploying daemon mon.c on smithi097 2024-03-21T13:22:13.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:13 smithi097 ceph-mon[30040]: from='client.? 172.21.15.144:0/2205476391' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T13:22:13.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:13 smithi097 ceph-mon[30040]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:13.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:13 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T13:22:14.359 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-21T13:22:14.359 DEBUG:teuthology.orchestra.run.smithi144:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-21T13:22:14.607 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T13:22:15.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 podman[39378]: 2024-03-21 13:22:15.470727778 +0000 UTC m=+0.135457693 container create e696409da010072945642e8c0109c725327d14b9c04a17faa462c3069fe12552 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mon-c, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319) 2024-03-21T13:22:16.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 podman[39378]: 2024-03-21 13:22:15.664775512 +0000 UTC m=+0.329505435 container init e696409da010072945642e8c0109c725327d14b9c04a17faa462c3069fe12552 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mon-c, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-03-21T13:22:16.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 podman[39378]: 2024-03-21 13:22:15.670158971 +0000 UTC m=+0.334888886 container start e696409da010072945642e8c0109c725327d14b9c04a17faa462c3069fe12552 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mon-c, org.label-schema.build-date=20240319, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS) 2024-03-21T13:22:16.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: set uid:gid to 167:167 (ceph:ceph) 2024-03-21T13:22:16.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: ceph version 19.0.0-2267-g960fbbe4 (960fbbe4645c2becc8084669be194c95183c7753) squid (dev), process ceph-mon, pid 7 2024-03-21T13:22:16.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: pidfile_write: ignore empty --pid-file 2024-03-21T13:22:16.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: load: jerasure load: lrc 2024-03-21T13:22:16.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: RocksDB version: 7.9.2 2024-03-21T13:22:16.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Git sha 0 2024-03-21T13:22:16.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Compile date 2024-03-21 08:53:26 2024-03-21T13:22:16.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: DB SUMMARY 2024-03-21T13:22:16.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: DB Session ID: EH21ZYF016Q820HHAPPU 2024-03-21T13:22:16.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: CURRENT file: CURRENT 2024-03-21T13:22:16.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: IDENTITY file: IDENTITY 2024-03-21T13:22:16.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-03-21T13:22:16.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-03-21T13:22:16.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 636 ; 2024-03-21T13:22:16.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.error_if_exists: 0 2024-03-21T13:22:16.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.create_if_missing: 0 2024-03-21T13:22:16.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.paranoid_checks: 1 2024-03-21T13:22:16.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-21T13:22:16.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-21T13:22:16.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-21T13:22:16.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.env: 0x560fb06a4b20 2024-03-21T13:22:16.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.fs: PosixFileSystem 2024-03-21T13:22:16.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.info_log: 0x560fb1f05920 2024-03-21T13:22:16.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.max_file_opening_threads: 16 2024-03-21T13:22:16.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.statistics: (nil) 2024-03-21T13:22:16.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.use_fsync: 0 2024-03-21T13:22:16.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.max_log_file_size: 0 2024-03-21T13:22:16.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-21T13:22:16.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-21T13:22:16.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.keep_log_file_num: 1000 2024-03-21T13:22:16.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.recycle_log_file_num: 0 2024-03-21T13:22:16.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.allow_fallocate: 1 2024-03-21T13:22:16.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.allow_mmap_reads: 0 2024-03-21T13:22:16.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.allow_mmap_writes: 0 2024-03-21T13:22:16.057 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.use_direct_reads: 0 2024-03-21T13:22:16.057 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-21T13:22:16.057 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.create_missing_column_families: 0 2024-03-21T13:22:16.057 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.db_log_dir: 2024-03-21T13:22:16.057 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.wal_dir: 2024-03-21T13:22:16.057 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-21T13:22:16.057 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-21T13:22:16.057 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-21T13:22:16.057 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-21T13:22:16.057 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-21T13:22:16.057 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-21T13:22:16.057 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.advise_random_on_open: 1 2024-03-21T13:22:16.057 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.db_write_buffer_size: 0 2024-03-21T13:22:16.057 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.write_buffer_manager: 0x560fb1ed61e0 2024-03-21T13:22:16.057 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-21T13:22:16.057 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-21T13:22:16.058 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-21T13:22:16.058 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.rate_limiter: (nil) 2024-03-21T13:22:16.058 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-21T13:22:16.058 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.wal_recovery_mode: 2 2024-03-21T13:22:16.058 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.enable_thread_tracking: 0 2024-03-21T13:22:16.058 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.enable_pipelined_write: 0 2024-03-21T13:22:16.058 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.unordered_write: 0 2024-03-21T13:22:16.058 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-21T13:22:16.058 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-21T13:22:16.058 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-21T13:22:16.058 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-21T13:22:16.058 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.row_cache: None 2024-03-21T13:22:16.059 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.wal_filter: None 2024-03-21T13:22:16.060 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-21T13:22:16.060 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.allow_ingest_behind: 0 2024-03-21T13:22:16.060 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.two_write_queues: 0 2024-03-21T13:22:16.060 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.manual_wal_flush: 0 2024-03-21T13:22:16.060 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.wal_compression: 0 2024-03-21T13:22:16.060 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.atomic_flush: 0 2024-03-21T13:22:16.060 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-21T13:22:16.061 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-21T13:22:16.061 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-21T13:22:16.061 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.log_readahead_size: 0 2024-03-21T13:22:16.061 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-21T13:22:16.061 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.best_efforts_recovery: 0 2024-03-21T13:22:16.061 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-21T13:22:16.061 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-21T13:22:16.061 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.allow_data_in_errors: 0 2024-03-21T13:22:16.061 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.db_host_id: __hostname__ 2024-03-21T13:22:16.061 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-21T13:22:16.061 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.max_background_jobs: 2 2024-03-21T13:22:16.061 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.max_background_compactions: -1 2024-03-21T13:22:16.061 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.max_subcompactions: 1 2024-03-21T13:22:16.061 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-21T13:22:16.061 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-21T13:22:16.061 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-21T13:22:16.062 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.max_total_wal_size: 0 2024-03-21T13:22:16.062 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-21T13:22:16.062 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-21T13:22:16.062 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-21T13:22:16.062 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-21T13:22:16.062 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.max_open_files: -1 2024-03-21T13:22:16.062 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.bytes_per_sync: 0 2024-03-21T13:22:16.062 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-21T13:22:16.062 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-21T13:22:16.062 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.compaction_readahead_size: 0 2024-03-21T13:22:16.062 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.max_background_flushes: -1 2024-03-21T13:22:16.062 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Compression algorithms supported: 2024-03-21T13:22:16.062 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-21T13:22:16.062 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: kZSTD supported: 0 2024-03-21T13:22:16.063 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: kXpressCompression supported: 0 2024-03-21T13:22:16.063 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: kLZ4HCCompression supported: 1 2024-03-21T13:22:16.063 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: kZlibCompression supported: 1 2024-03-21T13:22:16.063 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: kSnappyCompression supported: 1 2024-03-21T13:22:16.063 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: kLZ4Compression supported: 1 2024-03-21T13:22:16.063 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: kBZip2Compression supported: 0 2024-03-21T13:22:16.063 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-21T13:22:16.063 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-21T13:22:16.063 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-03-21T13:22:16.063 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-21T13:22:16.063 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-21T13:22:16.063 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.merge_operator: 2024-03-21T13:22:16.063 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.compaction_filter: None 2024-03-21T13:22:16.063 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.compaction_filter_factory: None 2024-03-21T13:22:16.063 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.sst_partitioner_factory: None 2024-03-21T13:22:16.064 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-21T13:22:16.064 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-21T13:22:16.064 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x560fb1f05a20) 2024-03-21T13:22:16.064 INFO:journalctl@ceph.mon.c.smithi097.stdout: cache_index_and_filter_blocks: 1 2024-03-21T13:22:16.064 INFO:journalctl@ceph.mon.c.smithi097.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-21T13:22:16.064 INFO:journalctl@ceph.mon.c.smithi097.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-21T13:22:16.064 INFO:journalctl@ceph.mon.c.smithi097.stdout: pin_top_level_index_and_filter: 1 2024-03-21T13:22:16.064 INFO:journalctl@ceph.mon.c.smithi097.stdout: index_type: 0 2024-03-21T13:22:16.064 INFO:journalctl@ceph.mon.c.smithi097.stdout: data_block_index_type: 0 2024-03-21T13:22:16.064 INFO:journalctl@ceph.mon.c.smithi097.stdout: index_shortening: 1 2024-03-21T13:22:16.064 INFO:journalctl@ceph.mon.c.smithi097.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-21T13:22:16.064 INFO:journalctl@ceph.mon.c.smithi097.stdout: checksum: 4 2024-03-21T13:22:16.064 INFO:journalctl@ceph.mon.c.smithi097.stdout: no_block_cache: 0 2024-03-21T13:22:16.064 INFO:journalctl@ceph.mon.c.smithi097.stdout: block_cache: 0x560fb1f09090 2024-03-21T13:22:16.064 INFO:journalctl@ceph.mon.c.smithi097.stdout: block_cache_name: BinnedLRUCache 2024-03-21T13:22:16.065 INFO:journalctl@ceph.mon.c.smithi097.stdout: block_cache_options: 2024-03-21T13:22:16.065 INFO:journalctl@ceph.mon.c.smithi097.stdout: capacity : 536870912 2024-03-21T13:22:16.065 INFO:journalctl@ceph.mon.c.smithi097.stdout: num_shard_bits : 4 2024-03-21T13:22:16.065 INFO:journalctl@ceph.mon.c.smithi097.stdout: strict_capacity_limit : 0 2024-03-21T13:22:16.065 INFO:journalctl@ceph.mon.c.smithi097.stdout: high_pri_pool_ratio: 0.000 2024-03-21T13:22:16.065 INFO:journalctl@ceph.mon.c.smithi097.stdout: block_cache_compressed: (nil) 2024-03-21T13:22:16.065 INFO:journalctl@ceph.mon.c.smithi097.stdout: persistent_cache: (nil) 2024-03-21T13:22:16.065 INFO:journalctl@ceph.mon.c.smithi097.stdout: block_size: 4096 2024-03-21T13:22:16.065 INFO:journalctl@ceph.mon.c.smithi097.stdout: block_size_deviation: 10 2024-03-21T13:22:16.065 INFO:journalctl@ceph.mon.c.smithi097.stdout: block_restart_interval: 16 2024-03-21T13:22:16.065 INFO:journalctl@ceph.mon.c.smithi097.stdout: index_block_restart_interval: 1 2024-03-21T13:22:16.065 INFO:journalctl@ceph.mon.c.smithi097.stdout: metadata_block_size: 4096 2024-03-21T13:22:16.065 INFO:journalctl@ceph.mon.c.smithi097.stdout: partition_filters: 0 2024-03-21T13:22:16.066 INFO:journalctl@ceph.mon.c.smithi097.stdout: use_delta_encoding: 1 2024-03-21T13:22:16.066 INFO:journalctl@ceph.mon.c.smithi097.stdout: filter_policy: bloomfilter 2024-03-21T13:22:16.066 INFO:journalctl@ceph.mon.c.smithi097.stdout: whole_key_filtering: 1 2024-03-21T13:22:16.066 INFO:journalctl@ceph.mon.c.smithi097.stdout: verify_compression: 0 2024-03-21T13:22:16.066 INFO:journalctl@ceph.mon.c.smithi097.stdout: read_amp_bytes_per_bit: 0 2024-03-21T13:22:16.066 INFO:journalctl@ceph.mon.c.smithi097.stdout: format_version: 5 2024-03-21T13:22:16.066 INFO:journalctl@ceph.mon.c.smithi097.stdout: enable_index_compression: 1 2024-03-21T13:22:16.066 INFO:journalctl@ceph.mon.c.smithi097.stdout: block_align: 0 2024-03-21T13:22:16.066 INFO:journalctl@ceph.mon.c.smithi097.stdout: max_auto_readahead_size: 262144 2024-03-21T13:22:16.066 INFO:journalctl@ceph.mon.c.smithi097.stdout: prepopulate_block_cache: 0 2024-03-21T13:22:16.066 INFO:journalctl@ceph.mon.c.smithi097.stdout: initial_auto_readahead_size: 8192 2024-03-21T13:22:16.066 INFO:journalctl@ceph.mon.c.smithi097.stdout: num_file_reads_for_auto_readahead: 2 2024-03-21T13:22:16.067 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.write_buffer_size: 33554432 2024-03-21T13:22:16.067 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.max_write_buffer_number: 2 2024-03-21T13:22:16.067 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.compression: NoCompression 2024-03-21T13:22:16.067 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.bottommost_compression: Disabled 2024-03-21T13:22:16.067 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.prefix_extractor: nullptr 2024-03-21T13:22:16.067 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-21T13:22:16.067 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.num_levels: 7 2024-03-21T13:22:16.067 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-21T13:22:16.067 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-21T13:22:16.067 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-21T13:22:16.067 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-21T13:22:16.067 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-21T13:22:16.067 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-21T13:22:16.067 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-21T13:22:16.067 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-21T13:22:16.068 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-21T13:22:16.068 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-21T13:22:16.068 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-21T13:22:16.068 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-21T13:22:16.068 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-21T13:22:16.068 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.compression_opts.level: 32767 2024-03-21T13:22:16.068 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.compression_opts.strategy: 0 2024-03-21T13:22:16.068 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-21T13:22:16.068 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-21T13:22:16.068 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-21T13:22:16.068 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-21T13:22:16.068 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.compression_opts.enabled: false 2024-03-21T13:22:16.068 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-21T13:22:16.068 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-21T13:22:16.068 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-21T13:22:16.069 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-21T13:22:16.069 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.target_file_size_base: 67108864 2024-03-21T13:22:16.069 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-21T13:22:16.069 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-21T13:22:16.069 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-21T13:22:16.069 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-21T13:22:16.069 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-21T13:22:16.069 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-21T13:22:16.069 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-21T13:22:16.069 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-21T13:22:16.069 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-21T13:22:16.069 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-21T13:22:16.069 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-21T13:22:16.069 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-21T13:22:16.069 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-21T13:22:16.070 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-21T13:22:16.070 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.arena_block_size: 1048576 2024-03-21T13:22:16.070 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-21T13:22:16.070 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-21T13:22:16.070 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.disable_auto_compactions: 0 2024-03-21T13:22:16.070 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-21T13:22:16.070 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-21T13:22:16.070 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-21T13:22:16.070 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-21T13:22:16.070 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-21T13:22:16.070 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-21T13:22:16.070 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-21T13:22:16.070 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-21T13:22:16.070 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-21T13:22:16.071 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-21T13:22:16.071 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.table_properties_collectors: 2024-03-21T13:22:16.071 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.inplace_update_support: 0 2024-03-21T13:22:16.071 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-21T13:22:16.071 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-21T13:22:16.071 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-21T13:22:16.071 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-21T13:22:16.071 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.bloom_locality: 0 2024-03-21T13:22:16.071 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.max_successive_merges: 0 2024-03-21T13:22:16.071 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-21T13:22:16.071 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.paranoid_file_checks: 0 2024-03-21T13:22:16.071 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.force_consistency_checks: 1 2024-03-21T13:22:16.071 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.report_bg_io_stats: 0 2024-03-21T13:22:16.071 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.ttl: 2592000 2024-03-21T13:22:16.072 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-21T13:22:16.072 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-21T13:22:16.072 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-21T13:22:16.072 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.enable_blob_files: false 2024-03-21T13:22:16.072 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.min_blob_size: 0 2024-03-21T13:22:16.072 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.blob_file_size: 268435456 2024-03-21T13:22:16.072 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-21T13:22:16.072 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-21T13:22:16.072 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-21T13:22:16.072 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-21T13:22:16.072 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-21T13:22:16.072 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.blob_file_starting_level: 0 2024-03-21T13:22:16.072 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-21T13:22:16.072 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-21T13:22:16.073 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-21T13:22:16.073 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: f71b0071-5c48-4ab0-8ae4-d25996086d5d 2024-03-21T13:22:16.073 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711027335699467, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-21T13:22:16.073 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-21T13:22:16.073 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711027335699845, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711027335, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "f71b0071-5c48-4ab0-8ae4-d25996086d5d", "db_session_id": "EH21ZYF016Q820HHAPPU", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-21T13:22:16.073 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711027335699923, "job": 1, "event": "recovery_finished"} 2024-03-21T13:22:16.073 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-21T13:22:16.073 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-21T13:22:16.073 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x560fb1ff8000 2024-03-21T13:22:16.073 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: DB pointer 0x560fb1fe2000 2024-03-21T13:22:16.073 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-21T13:22:16.073 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-21T13:22:16.073 INFO:journalctl@ceph.mon.c.smithi097.stdout: ** DB Stats ** 2024-03-21T13:22:16.074 INFO:journalctl@ceph.mon.c.smithi097.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-21T13:22:16.074 INFO:journalctl@ceph.mon.c.smithi097.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-21T13:22:16.074 INFO:journalctl@ceph.mon.c.smithi097.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-21T13:22:16.074 INFO:journalctl@ceph.mon.c.smithi097.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-21T13:22:16.074 INFO:journalctl@ceph.mon.c.smithi097.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-21T13:22:16.074 INFO:journalctl@ceph.mon.c.smithi097.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-21T13:22:16.074 INFO:journalctl@ceph.mon.c.smithi097.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-21T13:22:16.074 INFO:journalctl@ceph.mon.c.smithi097.stdout: 2024-03-21T13:22:16.074 INFO:journalctl@ceph.mon.c.smithi097.stdout: ** Compaction Stats [default] ** 2024-03-21T13:22:16.074 INFO:journalctl@ceph.mon.c.smithi097.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-21T13:22:16.074 INFO:journalctl@ceph.mon.c.smithi097.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-21T13:22:16.074 INFO:journalctl@ceph.mon.c.smithi097.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-21T13:22:16.074 INFO:journalctl@ceph.mon.c.smithi097.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-21T13:22:16.074 INFO:journalctl@ceph.mon.c.smithi097.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-21T13:22:16.075 INFO:journalctl@ceph.mon.c.smithi097.stdout: 2024-03-21T13:22:16.075 INFO:journalctl@ceph.mon.c.smithi097.stdout: ** Compaction Stats [default] ** 2024-03-21T13:22:16.075 INFO:journalctl@ceph.mon.c.smithi097.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-21T13:22:16.075 INFO:journalctl@ceph.mon.c.smithi097.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-21T13:22:16.075 INFO:journalctl@ceph.mon.c.smithi097.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-21T13:22:16.075 INFO:journalctl@ceph.mon.c.smithi097.stdout: 2024-03-21T13:22:16.075 INFO:journalctl@ceph.mon.c.smithi097.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-21T13:22:16.075 INFO:journalctl@ceph.mon.c.smithi097.stdout: 2024-03-21T13:22:16.075 INFO:journalctl@ceph.mon.c.smithi097.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-21T13:22:16.075 INFO:journalctl@ceph.mon.c.smithi097.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-21T13:22:16.075 INFO:journalctl@ceph.mon.c.smithi097.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-21T13:22:16.075 INFO:journalctl@ceph.mon.c.smithi097.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-21T13:22:16.075 INFO:journalctl@ceph.mon.c.smithi097.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-21T13:22:16.075 INFO:journalctl@ceph.mon.c.smithi097.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-21T13:22:16.076 INFO:journalctl@ceph.mon.c.smithi097.stdout: Cumulative compaction: 0.00 GB write, 0.70 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-21T13:22:16.076 INFO:journalctl@ceph.mon.c.smithi097.stdout: Interval compaction: 0.00 GB write, 0.70 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-21T13:22:16.076 INFO:journalctl@ceph.mon.c.smithi097.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-21T13:22:16.076 INFO:journalctl@ceph.mon.c.smithi097.stdout: Block cache BinnedLRUCache@0x560fb1f09090#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-03-21T13:22:16.076 INFO:journalctl@ceph.mon.c.smithi097.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-21T13:22:16.076 INFO:journalctl@ceph.mon.c.smithi097.stdout: 2024-03-21T13:22:16.076 INFO:journalctl@ceph.mon.c.smithi097.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-21T13:22:16.076 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-03-21T13:22:16.076 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: using public_addrv [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] 2024-03-21T13:22:16.076 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: starting mon.c rank -1 at public addrs [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] at bind addrs [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] mon_data /var/lib/ceph/mon/ceph-c fsid a5dd1f32-e785-11ee-95cd-87774f69a715 2024-03-21T13:22:16.076 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: mon.c@-1(???) e0 preinit fsid a5dd1f32-e785-11ee-95cd-87774f69a715 2024-03-21T13:22:16.076 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: mon.c@-1(synchronizing).mds e1 new map 2024-03-21T13:22:16.076 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: mon.c@-1(synchronizing).mds e1 print_map 2024-03-21T13:22:16.076 INFO:journalctl@ceph.mon.c.smithi097.stdout: e1 2024-03-21T13:22:16.077 INFO:journalctl@ceph.mon.c.smithi097.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T13:22:16.077 INFO:journalctl@ceph.mon.c.smithi097.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-21T13:22:16.077 INFO:journalctl@ceph.mon.c.smithi097.stdout: legacy client fscid: -1 2024-03-21T13:22:16.077 INFO:journalctl@ceph.mon.c.smithi097.stdout: 2024-03-21T13:22:16.077 INFO:journalctl@ceph.mon.c.smithi097.stdout: No filesystems configured 2024-03-21T13:22:16.077 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-21T13:22:16.077 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-21T13:22:16.077 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-21T13:22:16.077 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-21T13:22:16.077 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-21T13:22:16.077 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-21T13:22:16.077 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-21T13:22:16.077 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T13:22:16.077 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T13:22:16.078 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T13:22:16.078 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:16.078 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:16.078 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:16.078 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T13:22:16.078 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:16.078 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3601209568' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-21T13:22:16.078 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:16.078 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3601209568' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-21T13:22:16.078 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T13:22:16.078 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:16.078 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:16.078 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:16.078 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:16.078 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi144", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:22:16.079 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:16.079 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:22:16.079 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: Updating smithi144:/etc/ceph/ceph.conf 2024-03-21T13:22:16.079 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:16.079 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:16.079 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:16.079 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: Updating smithi144:/var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T13:22:16.079 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-03-21T13:22:16.079 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: Updating smithi144:/var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T13:22:16.079 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:16.079 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:16.079 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:22:16.079 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:16.079 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:22:16.080 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:16.080 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T13:22:16.080 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:16.080 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi097:172.21.15.97=a;smithi097:[v2:172.21.15.97:3301,v1:172.21.15.97:6790]=c;smithi144:172.21.15.144=b", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:22:16.080 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: Saving service mon spec with placement smithi097:172.21.15.97=a;smithi097:[v2:172.21.15.97:3301,v1:172.21.15.97:6790]=c;smithi144:172.21.15.144=b;count:3 2024-03-21T13:22:16.080 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: Deploying daemon mon.b on smithi144 2024-03-21T13:22:16.080 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:16.080 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:16.080 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: from='client.? 172.21.15.144:0/1977632081' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T13:22:16.080 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T13:22:16.080 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: mon.a calling monitor election 2024-03-21T13:22:16.080 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T13:22:16.080 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T13:22:16.080 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:16.080 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T13:22:16.081 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: mon.b calling monitor election 2024-03-21T13:22:16.081 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T13:22:16.081 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:16.081 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T13:22:16.081 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T13:22:16.081 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-03-21T13:22:16.081 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: monmap e2: 2 mons at {a=[v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0],b=[v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T13:22:16.081 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: fsmap 2024-03-21T13:22:16.081 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T13:22:16.081 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: mgrmap e13: y(active, since 41s) 2024-03-21T13:22:16.081 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: overall HEALTH_OK 2024-03-21T13:22:16.081 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:16.081 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:16.081 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:16.082 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T13:22:16.082 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:16.082 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: Deploying daemon mon.c on smithi097 2024-03-21T13:22:16.082 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: from='client.? 172.21.15.144:0/2205476391' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T13:22:16.082 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:16.082 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T13:22:16.082 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 ceph-mon[39416]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-21T13:22:16.082 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 bash[39378]: e696409da010072945642e8c0109c725327d14b9c04a17faa462c3069fe12552 2024-03-21T13:22:16.082 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:15 smithi097 systemd[1]: Started Ceph mon.c for a5dd1f32-e785-11ee-95cd-87774f69a715. 2024-03-21T13:22:21.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:20 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T13:22:21.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:20 smithi144 ceph-mon[33633]: mon.a calling monitor election 2024-03-21T13:22:21.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:20 smithi144 ceph-mon[33633]: mon.b calling monitor election 2024-03-21T13:22:21.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:20 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T13:22:21.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:20 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T13:22:21.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:20 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T13:22:21.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:20 smithi144 ceph-mon[33633]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:21.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:20 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T13:22:21.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:20 smithi144 ceph-mon[33633]: mon.c calling monitor election 2024-03-21T13:22:21.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:20 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T13:22:21.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:20 smithi144 ceph-mon[33633]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:21.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:20 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T13:22:21.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:20 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T13:22:21.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:20 smithi144 ceph-mon[33633]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-03-21T13:22:21.123 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:20 smithi144 ceph-mon[33633]: monmap e3: 3 mons at {a=[v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0],b=[v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0],c=[v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T13:22:21.123 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:20 smithi144 ceph-mon[33633]: fsmap 2024-03-21T13:22:21.123 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:20 smithi144 ceph-mon[33633]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T13:22:21.123 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:20 smithi144 ceph-mon[33633]: mgrmap e13: y(active, since 49s) 2024-03-21T13:22:21.123 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:20 smithi144 ceph-mon[33633]: overall HEALTH_OK 2024-03-21T13:22:21.123 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:20 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:21.123 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:20 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:21.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:20 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T13:22:21.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:20 smithi097 ceph-mon[30040]: mon.a calling monitor election 2024-03-21T13:22:21.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:20 smithi097 ceph-mon[30040]: mon.b calling monitor election 2024-03-21T13:22:21.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:20 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T13:22:21.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:20 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T13:22:21.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:20 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T13:22:21.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:20 smithi097 ceph-mon[30040]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:21.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:20 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T13:22:21.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:20 smithi097 ceph-mon[30040]: mon.c calling monitor election 2024-03-21T13:22:21.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:20 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T13:22:21.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:20 smithi097 ceph-mon[30040]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:21.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:20 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T13:22:21.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:20 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T13:22:21.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:20 smithi097 ceph-mon[30040]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-03-21T13:22:21.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:20 smithi097 ceph-mon[30040]: monmap e3: 3 mons at {a=[v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0],b=[v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0],c=[v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T13:22:21.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:20 smithi097 ceph-mon[30040]: fsmap 2024-03-21T13:22:21.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:20 smithi097 ceph-mon[30040]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T13:22:21.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:20 smithi097 ceph-mon[30040]: mgrmap e13: y(active, since 49s) 2024-03-21T13:22:21.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:20 smithi097 ceph-mon[30040]: overall HEALTH_OK 2024-03-21T13:22:21.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:20 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:21.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:20 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:21.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:20 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T13:22:21.306 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:20 smithi097 ceph-mon[39416]: mon.a calling monitor election 2024-03-21T13:22:21.306 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:20 smithi097 ceph-mon[39416]: mon.b calling monitor election 2024-03-21T13:22:21.306 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:20 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T13:22:21.306 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:20 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T13:22:21.306 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:20 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T13:22:21.306 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:20 smithi097 ceph-mon[39416]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:21.306 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:20 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T13:22:21.306 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:20 smithi097 ceph-mon[39416]: mon.c calling monitor election 2024-03-21T13:22:21.306 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:20 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T13:22:21.306 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:20 smithi097 ceph-mon[39416]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:21.306 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:20 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T13:22:21.306 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:20 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T13:22:21.306 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:20 smithi097 ceph-mon[39416]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-03-21T13:22:21.307 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:20 smithi097 ceph-mon[39416]: monmap e3: 3 mons at {a=[v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0],b=[v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0],c=[v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T13:22:21.307 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:20 smithi097 ceph-mon[39416]: fsmap 2024-03-21T13:22:21.307 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:20 smithi097 ceph-mon[39416]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T13:22:21.307 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:20 smithi097 ceph-mon[39416]: mgrmap e13: y(active, since 49s) 2024-03-21T13:22:21.307 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:20 smithi097 ceph-mon[39416]: overall HEALTH_OK 2024-03-21T13:22:21.307 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:20 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:21.307 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:20 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:22.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:21 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:22.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:21 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:22.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:21 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:22:22.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:21 smithi097 ceph-mon[30040]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:22.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:21 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:22.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:21 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T13:22:22.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:21 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:22.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:21 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:22.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:21 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:22:22.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:21 smithi097 ceph-mon[39416]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:22.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:21 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:22.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:21 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T13:22:22.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:21 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:22.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:21 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:22.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:21 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:22:22.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:21 smithi144 ceph-mon[33633]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:22.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:21 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:22.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:21 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T13:22:22.226 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.224+0000 7fe12d3b3700 1 -- 172.21.15.144:0/3972960233 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1100062a0 msgr2=0x7fe110008710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:22:22.226 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.224+0000 7fe12d3b3700 1 --2- 172.21.15.144:0/3972960233 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1100062a0 0x7fe110008710 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe128101c40 tx=0x7fe11c00fe90 comp rx=0 tx=0).stop 2024-03-21T13:22:22.226 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.224+0000 7fe12d3b3700 1 -- 172.21.15.144:0/3972960233 shutdown_connections 2024-03-21T13:22:22.227 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.224+0000 7fe12d3b3700 1 --2- 172.21.15.144:0/3972960233 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1100062a0 0x7fe110008710 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:22:22.227 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.224+0000 7fe12d3b3700 1 -- 172.21.15.144:0/3972960233 >> 172.21.15.144:0/3972960233 conn(0x7fe1280fc690 msgr2=0x7fe1280feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:22:22.227 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.224+0000 7fe12d3b3700 1 -- 172.21.15.144:0/3972960233 shutdown_connections 2024-03-21T13:22:22.227 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.224+0000 7fe12d3b3700 1 -- 172.21.15.144:0/3972960233 wait complete. 2024-03-21T13:22:22.227 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.224+0000 7fe12d3b3700 1 Processor -- start 2024-03-21T13:22:22.227 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.225+0000 7fe12d3b3700 1 -- start start 2024-03-21T13:22:22.227 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.225+0000 7fe12d3b3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1100062a0 0x7fe128198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:22:22.227 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.225+0000 7fe12d3b3700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe128198570 0x7fe1281989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:22:22.227 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.225+0000 7fe12d3b3700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1281a10a0 0x7fe1281a3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:22:22.227 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.225+0000 7fe12d3b3700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe128108310 con 0x7fe1281a10a0 2024-03-21T13:22:22.228 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.225+0000 7fe12d3b3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe128108190 con 0x7fe1100062a0 2024-03-21T13:22:22.228 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.225+0000 7fe12d3b3700 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe128108490 con 0x7fe128198570 2024-03-21T13:22:22.228 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.225+0000 7fe126ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1100062a0 0x7fe128198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:22:22.228 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.225+0000 7fe126ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1100062a0 0x7fe128198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:55740/0 (socket says 172.21.15.144:55740) 2024-03-21T13:22:22.228 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.225+0000 7fe126ffd700 1 -- 172.21.15.144:0/838350460 learned_addr learned my addr 172.21.15.144:0/838350460 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-21T13:22:22.228 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.225+0000 7fe1267fc700 1 --2- 172.21.15.144:0/838350460 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe128198570 0x7fe1281989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:22:22.228 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.225+0000 7fe1277fe700 1 --2- 172.21.15.144:0/838350460 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1281a10a0 0x7fe1281a3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:22:22.228 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.225+0000 7fe126ffd700 1 -- 172.21.15.144:0/838350460 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe128198570 msgr2=0x7fe1281989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:22:22.228 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.225+0000 7fe126ffd700 1 --2- 172.21.15.144:0/838350460 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe128198570 0x7fe1281989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:22:22.228 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.225+0000 7fe126ffd700 1 -- 172.21.15.144:0/838350460 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1281a10a0 msgr2=0x7fe1281a3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:22:22.228 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.225+0000 7fe126ffd700 1 --2- 172.21.15.144:0/838350460 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1281a10a0 0x7fe1281a3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:22:22.229 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.225+0000 7fe126ffd700 1 -- 172.21.15.144:0/838350460 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1281a3c60 con 0x7fe1100062a0 2024-03-21T13:22:22.229 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.225+0000 7fe1267fc700 1 --2- 172.21.15.144:0/838350460 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe128198570 0x7fe1281989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T13:22:22.229 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.226+0000 7fe126ffd700 1 --2- 172.21.15.144:0/838350460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1100062a0 0x7fe128198030 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe11c005920 tx=0x7fe11c01d9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:22:22.229 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.226+0000 7fe10ffff700 1 -- 172.21.15.144:0/838350460 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11c005250 con 0x7fe1100062a0 2024-03-21T13:22:22.229 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.226+0000 7fe10ffff700 1 -- 172.21.15.144:0/838350460 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fe11c01de20 con 0x7fe1100062a0 2024-03-21T13:22:22.229 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.226+0000 7fe12d3b3700 1 -- 172.21.15.144:0/838350460 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1281a3ef0 con 0x7fe1100062a0 2024-03-21T13:22:22.229 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.226+0000 7fe10ffff700 1 -- 172.21.15.144:0/838350460 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11c0116a0 con 0x7fe1100062a0 2024-03-21T13:22:22.229 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.226+0000 7fe12d3b3700 1 -- 172.21.15.144:0/838350460 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1281a4340 con 0x7fe1100062a0 2024-03-21T13:22:22.231 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.227+0000 7fe10ffff700 1 -- 172.21.15.144:0/838350460 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 13) v1 ==== 46315+0+0 (secure 0 0 0) 0x7fe11c011840 con 0x7fe1100062a0 2024-03-21T13:22:22.231 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.227+0000 7fe10ffff700 1 --2- 172.21.15.144:0/838350460 >> v2:172.21.15.97:6800/1943212787 conn(0x7fe114039670 0x7fe11403bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:22:22.231 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.227+0000 7fe10ffff700 1 -- 172.21.15.144:0/838350460 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (secure 0 0 0) 0x7fe11c0054f0 con 0x7fe1100062a0 2024-03-21T13:22:22.231 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.227+0000 7fe12d3b3700 1 -- 172.21.15.144:0/838350460 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1280637c0 con 0x7fe1100062a0 2024-03-21T13:22:22.231 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.227+0000 7fe1267fc700 1 --2- 172.21.15.144:0/838350460 >> v2:172.21.15.97:6800/1943212787 conn(0x7fe114039670 0x7fe11403bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:22:22.231 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.227+0000 7fe1267fc700 1 --2- 172.21.15.144:0/838350460 >> v2:172.21.15.97:6800/1943212787 conn(0x7fe114039670 0x7fe11403bb30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe118009ec0 tx=0x7fe118009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:22:22.232 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.232+0000 7fe10ffff700 1 -- 172.21.15.144:0/838350460 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe11c024310 con 0x7fe1100062a0 2024-03-21T13:22:22.519 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.518+0000 7fe12d3b3700 1 -- 172.21.15.144:0/838350460 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe1281a4570 con 0x7fe1100062a0 2024-03-21T13:22:22.519 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.519+0000 7fe10ffff700 1 -- 172.21.15.144:0/838350460 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1281 (secure 0 0 0) 0x7fe11c02e310 con 0x7fe1100062a0 2024-03-21T13:22:22.520 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-21T13:22:22.520 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":3,"fsid":"a5dd1f32-e785-11ee-95cd-87774f69a715","modified":"2024-03-21T13:22:15.719170Z","created":"2024-03-21T13:20:33.671111Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:3300","nonce":0},{"type":"v1","addr":"172.21.15.97:6789","nonce":0}]},"addr":"172.21.15.97:6789/0","public_addr":"172.21.15.97:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:3300","nonce":0},{"type":"v1","addr":"172.21.15.144:6789","nonce":0}]},"addr":"172.21.15.144:6789/0","public_addr":"172.21.15.144:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:3301","nonce":0},{"type":"v1","addr":"172.21.15.97:6790","nonce":0}]},"addr":"172.21.15.97:6790/0","public_addr":"172.21.15.97:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-03-21T13:22:22.522 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.521+0000 7fe12d3b3700 1 -- 172.21.15.144:0/838350460 >> v2:172.21.15.97:6800/1943212787 conn(0x7fe114039670 msgr2=0x7fe11403bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:22:22.522 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.521+0000 7fe12d3b3700 1 --2- 172.21.15.144:0/838350460 >> v2:172.21.15.97:6800/1943212787 conn(0x7fe114039670 0x7fe11403bb30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe118009ec0 tx=0x7fe118009210 comp rx=0 tx=0).stop 2024-03-21T13:22:22.522 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.521+0000 7fe12d3b3700 1 -- 172.21.15.144:0/838350460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1100062a0 msgr2=0x7fe128198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:22:22.522 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.521+0000 7fe12d3b3700 1 --2- 172.21.15.144:0/838350460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1100062a0 0x7fe128198030 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe11c005920 tx=0x7fe11c01d9b0 comp rx=0 tx=0).stop 2024-03-21T13:22:22.522 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.521+0000 7fe12d3b3700 1 -- 172.21.15.144:0/838350460 shutdown_connections 2024-03-21T13:22:22.522 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.521+0000 7fe12d3b3700 1 --2- 172.21.15.144:0/838350460 >> v2:172.21.15.97:6800/1943212787 conn(0x7fe114039670 0x7fe11403bb30 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:22:22.522 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.521+0000 7fe12d3b3700 1 --2- 172.21.15.144:0/838350460 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1281a10a0 0x7fe1281a3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:22:22.522 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.521+0000 7fe12d3b3700 1 --2- 172.21.15.144:0/838350460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1100062a0 0x7fe128198030 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:22:22.522 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.521+0000 7fe12d3b3700 1 --2- 172.21.15.144:0/838350460 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe128198570 0x7fe1281989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:22:22.522 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.521+0000 7fe12d3b3700 1 -- 172.21.15.144:0/838350460 >> 172.21.15.144:0/838350460 conn(0x7fe1280fc690 msgr2=0x7fe1281047c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:22:22.522 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.522+0000 7fe12d3b3700 1 -- 172.21.15.144:0/838350460 shutdown_connections 2024-03-21T13:22:22.522 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:22.522+0000 7fe12d3b3700 1 -- 172.21.15.144:0/838350460 wait complete. 2024-03-21T13:22:22.523 INFO:teuthology.orchestra.run.smithi144.stderr:dumped monmap epoch 3 2024-03-21T13:22:23.197 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-21T13:22:23.198 DEBUG:teuthology.orchestra.run.smithi097:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph config generate-minimal-conf 2024-03-21T13:22:23.444 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T13:22:23.530 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:23 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:23.530 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:23 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:23.531 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:23 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:23.531 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:23 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:23.531 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:23 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:23.531 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:23 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:22:23.531 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:23 smithi097 ceph-mon[30040]: from='client.? 172.21.15.144:0/838350460' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T13:22:23.531 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:23 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:23.531 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:23 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:23.531 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:23 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:23.531 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:23 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:23.531 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:23 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:23.531 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:23 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:23.531 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:23 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:23.531 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:23 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:23.531 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:23 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:23.532 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:23 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T13:22:23.532 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:23 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T13:22:23.532 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:23 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:23.532 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:23 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:23.532 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:23 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:23.532 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:23 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:23.532 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:23 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:23.532 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:23 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:23.532 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:23 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:22:23.532 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:23 smithi097 ceph-mon[39416]: from='client.? 172.21.15.144:0/838350460' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T13:22:23.532 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:23 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:23.532 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:23 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:23.533 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:23 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:23.533 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:23 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:23.533 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:23 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:23.533 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:23 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:23.533 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:23 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:23.533 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:23 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:23.533 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:23 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:23.533 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:23 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T13:22:23.533 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:23 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T13:22:23.533 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:23 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:23.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:23 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:23.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:23 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:23.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:23 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:23.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:23 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:23.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:23 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:23.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:23 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:22:23.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:23 smithi144 ceph-mon[33633]: from='client.? 172.21.15.144:0/838350460' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T13:22:23.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:23 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:23.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:23 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:23.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:23 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:23.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:23 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:23.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:23 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:23.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:23 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:23.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:23 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:23.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:23 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:23.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:23 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:23.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:23 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T13:22:23.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:23 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T13:22:23.624 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:23 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:24.384 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:24 smithi097 ceph-mon[30040]: Updating smithi097:/etc/ceph/ceph.conf 2024-03-21T13:22:24.384 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:24 smithi097 ceph-mon[30040]: Updating smithi144:/etc/ceph/ceph.conf 2024-03-21T13:22:24.384 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:24 smithi097 ceph-mon[30040]: Updating smithi097:/var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T13:22:24.385 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:24 smithi097 ceph-mon[30040]: Updating smithi144:/var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T13:22:24.385 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:24 smithi097 ceph-mon[30040]: Reconfiguring mon.a (unknown last config time)... 2024-03-21T13:22:24.385 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:24 smithi097 ceph-mon[30040]: Reconfiguring daemon mon.a on smithi097 2024-03-21T13:22:24.385 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:24 smithi097 ceph-mon[30040]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:24.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:24 smithi097 ceph-mon[39416]: Updating smithi097:/etc/ceph/ceph.conf 2024-03-21T13:22:24.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:24 smithi097 ceph-mon[39416]: Updating smithi144:/etc/ceph/ceph.conf 2024-03-21T13:22:24.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:24 smithi097 ceph-mon[39416]: Updating smithi097:/var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T13:22:24.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:24 smithi097 ceph-mon[39416]: Updating smithi144:/var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T13:22:24.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:24 smithi097 ceph-mon[39416]: Reconfiguring mon.a (unknown last config time)... 2024-03-21T13:22:24.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:24 smithi097 ceph-mon[39416]: Reconfiguring daemon mon.a on smithi097 2024-03-21T13:22:24.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:24 smithi097 ceph-mon[39416]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:24.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:24 smithi144 ceph-mon[33633]: Updating smithi097:/etc/ceph/ceph.conf 2024-03-21T13:22:24.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:24 smithi144 ceph-mon[33633]: Updating smithi144:/etc/ceph/ceph.conf 2024-03-21T13:22:24.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:24 smithi144 ceph-mon[33633]: Updating smithi097:/var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T13:22:24.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:24 smithi144 ceph-mon[33633]: Updating smithi144:/var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T13:22:24.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:24 smithi144 ceph-mon[33633]: Reconfiguring mon.a (unknown last config time)... 2024-03-21T13:22:24.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:24 smithi144 ceph-mon[33633]: Reconfiguring daemon mon.a on smithi097 2024-03-21T13:22:24.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:24 smithi144 ceph-mon[33633]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:25.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:25 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:25.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:25 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:25.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:25 smithi097 ceph-mon[30040]: Reconfiguring mon.c (monmap changed)... 2024-03-21T13:22:25.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:25 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T13:22:25.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:25 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T13:22:25.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:25 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:25.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:25 smithi097 ceph-mon[30040]: Reconfiguring daemon mon.c on smithi097 2024-03-21T13:22:25.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:25 smithi097 ceph-mon[30040]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:25.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:25 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:25.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:25 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:25.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:25 smithi097 ceph-mon[39416]: Reconfiguring mon.c (monmap changed)... 2024-03-21T13:22:25.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:25 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T13:22:25.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:25 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T13:22:25.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:25 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:25.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:25 smithi097 ceph-mon[39416]: Reconfiguring daemon mon.c on smithi097 2024-03-21T13:22:25.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:25 smithi097 ceph-mon[39416]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:26.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:25 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:26.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:25 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:26.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:25 smithi144 ceph-mon[33633]: Reconfiguring mon.c (monmap changed)... 2024-03-21T13:22:26.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:25 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T13:22:26.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:25 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T13:22:26.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:25 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:26.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:25 smithi144 ceph-mon[33633]: Reconfiguring daemon mon.c on smithi097 2024-03-21T13:22:26.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:25 smithi144 ceph-mon[33633]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:26.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.255+0000 7fdb11b8f700 1 -- 172.21.15.97:0/201426017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb0c1016d0 msgr2=0x7fdb0c106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:22:26.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.255+0000 7fdb11b8f700 1 --2- 172.21.15.97:0/201426017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb0c1016d0 0x7fdb0c106e10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdb00009920 tx=0x7fdb00009c30 comp rx=0 tx=0).stop 2024-03-21T13:22:26.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.255+0000 7fdb11b8f700 1 -- 172.21.15.97:0/201426017 shutdown_connections 2024-03-21T13:22:26.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.255+0000 7fdb11b8f700 1 --2- 172.21.15.97:0/201426017 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdb0c107540 0x7fdb0c10f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:22:26.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.255+0000 7fdb11b8f700 1 --2- 172.21.15.97:0/201426017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb0c100d20 0x7fdb0c101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:22:26.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.255+0000 7fdb11b8f700 1 --2- 172.21.15.97:0/201426017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb0c1016d0 0x7fdb0c106e10 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:22:26.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.255+0000 7fdb11b8f700 1 -- 172.21.15.97:0/201426017 >> 172.21.15.97:0/201426017 conn(0x7fdb0c0763c0 msgr2=0x7fdb0c0767d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:22:26.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.255+0000 7fdb11b8f700 1 -- 172.21.15.97:0/201426017 shutdown_connections 2024-03-21T13:22:26.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.255+0000 7fdb11b8f700 1 -- 172.21.15.97:0/201426017 wait complete. 2024-03-21T13:22:26.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.256+0000 7fdb11b8f700 1 Processor -- start 2024-03-21T13:22:26.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.256+0000 7fdb11b8f700 1 -- start start 2024-03-21T13:22:26.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.256+0000 7fdb11b8f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb0c100d20 0x7fdb0c101ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:22:26.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.256+0000 7fdb11b8f700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdb0c1016d0 0x7fdb0c1021e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:22:26.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.256+0000 7fdb11b8f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb0c107540 0x7fdb0c102720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:22:26.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.256+0000 7fdb11b8f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c111bb0 con 0x7fdb0c100d20 2024-03-21T13:22:26.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.256+0000 7fdb11b8f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c111a30 con 0x7fdb0c107540 2024-03-21T13:22:26.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.256+0000 7fdb11b8f700 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c111d30 con 0x7fdb0c1016d0 2024-03-21T13:22:26.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.257+0000 7fdb0affd700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdb0c1016d0 0x7fdb0c1021e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:22:26.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.257+0000 7fdb0b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb0c100d20 0x7fdb0c101ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:22:26.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.257+0000 7fdb0affd700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdb0c1016d0 0x7fdb0c1021e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.97:49832/0 (socket says 172.21.15.97:49832) 2024-03-21T13:22:26.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.257+0000 7fdb0affd700 1 -- 172.21.15.97:0/4166365899 learned_addr learned my addr 172.21.15.97:0/4166365899 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:22:26.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.257+0000 7fdb0bfff700 1 --2- 172.21.15.97:0/4166365899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb0c107540 0x7fdb0c102720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:22:26.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.257+0000 7fdb0b7fe700 1 -- 172.21.15.97:0/4166365899 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdb0c1016d0 msgr2=0x7fdb0c1021e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:22:26.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.257+0000 7fdb0b7fe700 1 --2- 172.21.15.97:0/4166365899 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdb0c1016d0 0x7fdb0c1021e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:22:26.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.257+0000 7fdb0b7fe700 1 -- 172.21.15.97:0/4166365899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb0c107540 msgr2=0x7fdb0c102720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:22:26.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.257+0000 7fdb0b7fe700 1 --2- 172.21.15.97:0/4166365899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb0c107540 0x7fdb0c102720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:22:26.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.257+0000 7fdb0b7fe700 1 -- 172.21.15.97:0/4166365899 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb0c1a6650 con 0x7fdb0c100d20 2024-03-21T13:22:26.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.257+0000 7fdb0affd700 1 --2- 172.21.15.97:0/4166365899 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdb0c1016d0 0x7fdb0c1021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T13:22:26.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.257+0000 7fdb0b7fe700 1 --2- 172.21.15.97:0/4166365899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb0c100d20 0x7fdb0c101ca0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdafc00c970 tx=0x7fdafc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:22:26.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.257+0000 7fdb08ff9700 1 -- 172.21.15.97:0/4166365899 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdafc00ce60 con 0x7fdb0c100d20 2024-03-21T13:22:26.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.257+0000 7fdb08ff9700 1 -- 172.21.15.97:0/4166365899 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fdafc004830 con 0x7fdb0c100d20 2024-03-21T13:22:26.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.257+0000 7fdb11b8f700 1 -- 172.21.15.97:0/4166365899 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb0c1a6940 con 0x7fdb0c100d20 2024-03-21T13:22:26.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.257+0000 7fdb08ff9700 1 -- 172.21.15.97:0/4166365899 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdafc0054a0 con 0x7fdb0c100d20 2024-03-21T13:22:26.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.257+0000 7fdb11b8f700 1 -- 172.21.15.97:0/4166365899 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb0c1a6e80 con 0x7fdb0c100d20 2024-03-21T13:22:26.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.258+0000 7fdb08ff9700 1 -- 172.21.15.97:0/4166365899 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 46315+0+0 (secure 0 0 0) 0x7fdafc00f450 con 0x7fdb0c100d20 2024-03-21T13:22:26.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.258+0000 7fdb08ff9700 1 --2- 172.21.15.97:0/4166365899 >> v2:172.21.15.97:6800/1943212787 conn(0x7fdaf4039420 0x7fdaf403b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:22:26.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.258+0000 7fdb08ff9700 1 -- 172.21.15.97:0/4166365899 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (secure 0 0 0) 0x7fdafc04bf60 con 0x7fdb0c100d20 2024-03-21T13:22:26.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.258+0000 7fdb11b8f700 1 -- 172.21.15.97:0/4166365899 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb0c068e50 con 0x7fdb0c100d20 2024-03-21T13:22:26.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.258+0000 7fdb0affd700 1 --2- 172.21.15.97:0/4166365899 >> v2:172.21.15.97:6800/1943212787 conn(0x7fdaf4039420 0x7fdaf403b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:22:26.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.259+0000 7fdb0affd700 1 --2- 172.21.15.97:0/4166365899 >> v2:172.21.15.97:6800/1943212787 conn(0x7fdaf4039420 0x7fdaf403b8e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdb00009680 tx=0x7fdb00019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:22:26.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.263+0000 7fdb08ff9700 1 -- 172.21.15.97:0/4166365899 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdafc014030 con 0x7fdb0c100d20 2024-03-21T13:22:26.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.466+0000 7fdb11b8f700 1 -- 172.21.15.97:0/4166365899 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fdb0c1a70b0 con 0x7fdb0c100d20 2024-03-21T13:22:26.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.467+0000 7fdb08ff9700 1 -- 172.21.15.97:0/4166365899 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+273 (secure 0 0 0) 0x7fdafc005640 con 0x7fdb0c100d20 2024-03-21T13:22:26.468 INFO:teuthology.orchestra.run.smithi097.stdout:# minimal ceph.conf for a5dd1f32-e785-11ee-95cd-87774f69a715 2024-03-21T13:22:26.468 INFO:teuthology.orchestra.run.smithi097.stdout:[global] 2024-03-21T13:22:26.468 INFO:teuthology.orchestra.run.smithi097.stdout: fsid = a5dd1f32-e785-11ee-95cd-87774f69a715 2024-03-21T13:22:26.468 INFO:teuthology.orchestra.run.smithi097.stdout: mon_host = [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] 2024-03-21T13:22:26.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.469+0000 7fdb11b8f700 1 -- 172.21.15.97:0/4166365899 >> v2:172.21.15.97:6800/1943212787 conn(0x7fdaf4039420 msgr2=0x7fdaf403b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:22:26.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.469+0000 7fdb11b8f700 1 --2- 172.21.15.97:0/4166365899 >> v2:172.21.15.97:6800/1943212787 conn(0x7fdaf4039420 0x7fdaf403b8e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdb00009680 tx=0x7fdb00019040 comp rx=0 tx=0).stop 2024-03-21T13:22:26.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.469+0000 7fdb11b8f700 1 -- 172.21.15.97:0/4166365899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb0c100d20 msgr2=0x7fdb0c101ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:22:26.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.469+0000 7fdb11b8f700 1 --2- 172.21.15.97:0/4166365899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb0c100d20 0x7fdb0c101ca0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdafc00c970 tx=0x7fdafc00cc80 comp rx=0 tx=0).stop 2024-03-21T13:22:26.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.469+0000 7fdb11b8f700 1 -- 172.21.15.97:0/4166365899 shutdown_connections 2024-03-21T13:22:26.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.469+0000 7fdb11b8f700 1 --2- 172.21.15.97:0/4166365899 >> v2:172.21.15.97:6800/1943212787 conn(0x7fdaf4039420 0x7fdaf403b8e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:22:26.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.469+0000 7fdb11b8f700 1 --2- 172.21.15.97:0/4166365899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb0c107540 0x7fdb0c102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:22:26.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.469+0000 7fdb11b8f700 1 --2- 172.21.15.97:0/4166365899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb0c100d20 0x7fdb0c101ca0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:22:26.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.469+0000 7fdb11b8f700 1 --2- 172.21.15.97:0/4166365899 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdb0c1016d0 0x7fdb0c1021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:22:26.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.469+0000 7fdb11b8f700 1 -- 172.21.15.97:0/4166365899 >> 172.21.15.97:0/4166365899 conn(0x7fdb0c0763c0 msgr2=0x7fdb0c0fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:22:26.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.469+0000 7fdb11b8f700 1 -- 172.21.15.97:0/4166365899 shutdown_connections 2024-03-21T13:22:26.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:26.469+0000 7fdb11b8f700 1 -- 172.21.15.97:0/4166365899 wait complete. 2024-03-21T13:22:26.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:26 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4166365899' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:26.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:26 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4166365899' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:27.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:26 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4166365899' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:27.301 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-21T13:22:27.301 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-21T13:22:27.301 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/ceph/ceph.conf 2024-03-21T13:22:27.330 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-21T13:22:27.330 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-21T13:22:27.397 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-21T13:22:27.397 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/ceph/ceph.conf 2024-03-21T13:22:27.432 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-21T13:22:27.432 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-21T13:22:27.503 INFO:tasks.cephadm:Adding mgr.y on smithi097 2024-03-21T13:22:27.503 INFO:tasks.cephadm:Adding mgr.x on smithi144 2024-03-21T13:22:27.503 DEBUG:teuthology.orchestra.run.smithi144:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph orch apply mgr '2;smithi097=y;smithi144=x' 2024-03-21T13:22:27.781 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T13:22:28.001 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:27 smithi144 ceph-mon[33633]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:28.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:27 smithi097 ceph-mon[30040]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:28.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:27 smithi097 ceph-mon[39416]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:28.863 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:28 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:28.863 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:28 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:28.863 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:28 smithi144 ceph-mon[33633]: Reconfiguring mon.b (monmap changed)... 2024-03-21T13:22:28.863 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:28 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T13:22:28.863 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:28 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T13:22:28.863 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:28 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:28.863 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:28 smithi144 ceph-mon[33633]: Reconfiguring daemon mon.b on smithi144 2024-03-21T13:22:29.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:28 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:29.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:28 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:29.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:28 smithi097 ceph-mon[30040]: Reconfiguring mon.b (monmap changed)... 2024-03-21T13:22:29.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:28 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T13:22:29.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:28 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T13:22:29.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:28 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:29.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:28 smithi097 ceph-mon[30040]: Reconfiguring daemon mon.b on smithi144 2024-03-21T13:22:29.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:28 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:29.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:28 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:29.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:28 smithi097 ceph-mon[39416]: Reconfiguring mon.b (monmap changed)... 2024-03-21T13:22:29.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:28 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T13:22:29.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:28 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T13:22:29.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:28 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:29.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:28 smithi097 ceph-mon[39416]: Reconfiguring daemon mon.b on smithi144 2024-03-21T13:22:29.436 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:29.434+0000 7fd57a2ef700 1 -- 172.21.15.144:0/4033351352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd574100bb0 msgr2=0x7fd574100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:22:29.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:29.434+0000 7fd57a2ef700 1 --2- 172.21.15.144:0/4033351352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd574100bb0 0x7fd574100fd0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd564009910 tx=0x7fd564009c20 comp rx=0 tx=0).stop 2024-03-21T13:22:29.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:29.434+0000 7fd57a2ef700 1 -- 172.21.15.144:0/4033351352 shutdown_connections 2024-03-21T13:22:29.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:29.434+0000 7fd57a2ef700 1 --2- 172.21.15.144:0/4033351352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd574100bb0 0x7fd574100fd0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:22:29.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:29.434+0000 7fd57a2ef700 1 -- 172.21.15.144:0/4033351352 >> 172.21.15.144:0/4033351352 conn(0x7fd5740fc170 msgr2=0x7fd5740fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:22:29.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:29.434+0000 7fd57a2ef700 1 -- 172.21.15.144:0/4033351352 shutdown_connections 2024-03-21T13:22:29.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:29.434+0000 7fd57a2ef700 1 -- 172.21.15.144:0/4033351352 wait complete. 2024-03-21T13:22:29.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:29.435+0000 7fd57a2ef700 1 Processor -- start 2024-03-21T13:22:29.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:29.435+0000 7fd57a2ef700 1 -- start start 2024-03-21T13:22:29.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:29.435+0000 7fd57a2ef700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd574100bb0 0x7fd574196c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:22:29.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:29.435+0000 7fd57a2ef700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5741971b0 0x7fd57419c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:22:29.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:29.435+0000 7fd57a2ef700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd574197630 0x7fd57419e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:22:29.438 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:29.435+0000 7fd57a2ef700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5741062a0 con 0x7fd5741971b0 2024-03-21T13:22:29.438 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:29.435+0000 7fd57a2ef700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd574106120 con 0x7fd574197630 2024-03-21T13:22:29.438 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:29.435+0000 7fd57a2ef700 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd574106420 con 0x7fd574100bb0 2024-03-21T13:22:29.438 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:29.435+0000 7fd57888c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd574197630 0x7fd57419e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:22:29.438 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:29.436+0000 7fd57888c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd574197630 0x7fd57419e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:55752/0 (socket says 172.21.15.144:55752) 2024-03-21T13:22:29.438 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:29.436+0000 7fd57888c700 1 -- 172.21.15.144:0/3407878124 learned_addr learned my addr 172.21.15.144:0/3407878124 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-21T13:22:29.438 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:29.436+0000 7fd573fff700 1 --2- 172.21.15.144:0/3407878124 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd574100bb0 0x7fd574196c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:22:29.438 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:29.436+0000 7fd5737fe700 1 --2- 172.21.15.144:0/3407878124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5741971b0 0x7fd57419c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:22:29.438 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:29.436+0000 7fd57888c700 1 -- 172.21.15.144:0/3407878124 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd574100bb0 msgr2=0x7fd574196c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:22:29.438 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:29.436+0000 7fd57888c700 1 --2- 172.21.15.144:0/3407878124 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd574100bb0 0x7fd574196c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:22:29.438 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:29.436+0000 7fd57888c700 1 -- 172.21.15.144:0/3407878124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5741971b0 msgr2=0x7fd57419c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:22:29.438 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:29.436+0000 7fd57888c700 1 --2- 172.21.15.144:0/3407878124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5741971b0 0x7fd57419c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:22:29.438 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:29.436+0000 7fd57888c700 1 -- 172.21.15.144:0/3407878124 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd57419f090 con 0x7fd574197630 2024-03-21T13:22:29.438 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:29.436+0000 7fd57888c700 1 --2- 172.21.15.144:0/3407878124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd574197630 0x7fd57419e790 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd56c00ba50 tx=0x7fd56c00bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:22:29.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:29.436+0000 7fd5717fa700 1 -- 172.21.15.144:0/3407878124 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56c019070 con 0x7fd574197630 2024-03-21T13:22:29.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:29.436+0000 7fd5717fa700 1 -- 172.21.15.144:0/3407878124 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fd56c00cde0 con 0x7fd574197630 2024-03-21T13:22:29.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:29.436+0000 7fd57a2ef700 1 -- 172.21.15.144:0/3407878124 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd57419f380 con 0x7fd574197630 2024-03-21T13:22:29.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:29.436+0000 7fd5717fa700 1 -- 172.21.15.144:0/3407878124 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56c007350 con 0x7fd574197630 2024-03-21T13:22:29.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:29.436+0000 7fd57a2ef700 1 -- 172.21.15.144:0/3407878124 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd57419f990 con 0x7fd574197630 2024-03-21T13:22:29.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:29.437+0000 7fd5717fa700 1 -- 172.21.15.144:0/3407878124 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 13) v1 ==== 46315+0+0 (secure 0 0 0) 0x7fd56c01c070 con 0x7fd574197630 2024-03-21T13:22:29.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:29.437+0000 7fd57a2ef700 1 -- 172.21.15.144:0/3407878124 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5740637c0 con 0x7fd574197630 2024-03-21T13:22:29.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:29.437+0000 7fd5717fa700 1 --2- 172.21.15.144:0/3407878124 >> v2:172.21.15.97:6800/1943212787 conn(0x7fd55c039670 0x7fd55c03bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:22:29.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:29.437+0000 7fd5717fa700 1 -- 172.21.15.144:0/3407878124 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (secure 0 0 0) 0x7fd56c04b730 con 0x7fd574197630 2024-03-21T13:22:29.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:29.438+0000 7fd573fff700 1 --2- 172.21.15.144:0/3407878124 >> v2:172.21.15.97:6800/1943212787 conn(0x7fd55c039670 0x7fd55c03bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:22:29.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:29.438+0000 7fd573fff700 1 --2- 172.21.15.144:0/3407878124 >> v2:172.21.15.97:6800/1943212787 conn(0x7fd55c039670 0x7fd55c03bb30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd564000c00 tx=0x7fd56401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:22:29.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:29.442+0000 7fd5717fa700 1 -- 172.21.15.144:0/3407878124 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd56c0145b0 con 0x7fd574197630 2024-03-21T13:22:29.662 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:29.662+0000 7fd57a2ef700 1 -- 172.21.15.144:0/3407878124 --> v2:172.21.15.97:6800/1943212787 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi097=y;smithi144=x", "target": ["mon-mgr", ""]}) v1 -- 0x7fd5740625d0 con 0x7fd55c039670 2024-03-21T13:22:29.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:29.668+0000 7fd5717fa700 1 -- 172.21.15.144:0/3407878124 <== mgr.14152 v2:172.21.15.97:6800/1943212787 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fd5740625d0 con 0x7fd55c039670 2024-03-21T13:22:29.668 INFO:teuthology.orchestra.run.smithi144.stdout:Scheduled mgr update... 2024-03-21T13:22:29.671 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:29.670+0000 7fd57a2ef700 1 -- 172.21.15.144:0/3407878124 >> v2:172.21.15.97:6800/1943212787 conn(0x7fd55c039670 msgr2=0x7fd55c03bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:22:29.671 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:29.670+0000 7fd57a2ef700 1 --2- 172.21.15.144:0/3407878124 >> v2:172.21.15.97:6800/1943212787 conn(0x7fd55c039670 0x7fd55c03bb30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd564000c00 tx=0x7fd56401a040 comp rx=0 tx=0).stop 2024-03-21T13:22:29.671 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:29.670+0000 7fd57a2ef700 1 -- 172.21.15.144:0/3407878124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd574197630 msgr2=0x7fd57419e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:22:29.671 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:29.670+0000 7fd57a2ef700 1 --2- 172.21.15.144:0/3407878124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd574197630 0x7fd57419e790 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd56c00ba50 tx=0x7fd56c00bd60 comp rx=0 tx=0).stop 2024-03-21T13:22:29.671 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:29.670+0000 7fd57a2ef700 1 -- 172.21.15.144:0/3407878124 shutdown_connections 2024-03-21T13:22:29.671 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:29.670+0000 7fd57a2ef700 1 --2- 172.21.15.144:0/3407878124 >> v2:172.21.15.97:6800/1943212787 conn(0x7fd55c039670 0x7fd55c03bb30 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:22:29.671 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:29.670+0000 7fd57a2ef700 1 --2- 172.21.15.144:0/3407878124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd574197630 0x7fd57419e790 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:22:29.671 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:29.670+0000 7fd57a2ef700 1 --2- 172.21.15.144:0/3407878124 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd574100bb0 0x7fd574196c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:22:29.671 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:29.670+0000 7fd57a2ef700 1 --2- 172.21.15.144:0/3407878124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5741971b0 0x7fd57419c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:22:29.671 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:29.670+0000 7fd57a2ef700 1 -- 172.21.15.144:0/3407878124 >> 172.21.15.144:0/3407878124 conn(0x7fd5740fc170 msgr2=0x7fd574103e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:22:29.671 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:29.670+0000 7fd57a2ef700 1 -- 172.21.15.144:0/3407878124 shutdown_connections 2024-03-21T13:22:29.671 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:22:29.670+0000 7fd57a2ef700 1 -- 172.21.15.144:0/3407878124 wait complete. 2024-03-21T13:22:30.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:29 smithi144 ceph-mon[33633]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:30.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:29 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:30.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:29 smithi097 ceph-mon[30040]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:30.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:29 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:30.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:29 smithi097 ceph-mon[39416]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:30.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:29 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:30.474 DEBUG:teuthology.orchestra.run.smithi144:mgr.x> sudo journalctl -f -n 0 -u ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@mgr.x.service 2024-03-21T13:22:30.476 INFO:tasks.cephadm:Deploying OSDs... 2024-03-21T13:22:30.477 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-21T13:22:30.477 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/scratch_devs of=/dev/stdout 2024-03-21T13:22:30.495 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-21T13:22:30.496 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_1 2024-03-21T13:22:30.554 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-21T13:22:30.554 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T13:22:30.554 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-03-21T13:22:30.554 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T13:22:30.554 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T13:22:30.554 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-03-21 13:22:15.180242450 +0000 2024-03-21T13:22:30.554 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-03-21 13:16:20.069189022 +0000 2024-03-21T13:22:30.554 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-03-21 13:16:20.069189022 +0000 2024-03-21T13:22:30.554 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-03-21 13:16:20.069189022 +0000 2024-03-21T13:22:30.554 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-21T13:22:30.625 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-03-21T13:22:30.625 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-03-21T13:22:30.625 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000287665 s, 1.8 MB/s 2024-03-21T13:22:30.626 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-21T13:22:30.684 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_2 2024-03-21T13:22:30.740 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-21T13:22:30.741 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T13:22:30.741 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-03-21T13:22:30.741 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T13:22:30.741 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T13:22:30.741 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-03-21 13:22:15.180242450 +0000 2024-03-21T13:22:30.741 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-03-21 13:16:20.395183091 +0000 2024-03-21T13:22:30.741 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-03-21 13:16:20.395183091 +0000 2024-03-21T13:22:30.741 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-03-21 13:16:20.395183091 +0000 2024-03-21T13:22:30.741 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-21T13:22:30.810 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-03-21T13:22:30.810 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-03-21T13:22:30.810 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.00028216 s, 1.8 MB/s 2024-03-21T13:22:30.811 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-21T13:22:30.868 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_3 2024-03-21T13:22:30.924 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-21T13:22:30.924 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T13:22:30.925 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-03-21T13:22:30.925 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T13:22:30.925 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T13:22:30.925 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-03-21 13:22:15.181242432 +0000 2024-03-21T13:22:30.925 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-03-21 13:16:20.729177015 +0000 2024-03-21T13:22:30.925 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-03-21 13:16:20.729177015 +0000 2024-03-21T13:22:30.925 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-03-21 13:16:20.729177015 +0000 2024-03-21T13:22:30.925 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-21T13:22:30.989 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:30 smithi097 ceph-mon[30040]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi097=y;smithi144=x", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:22:30.989 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:30 smithi097 ceph-mon[30040]: Saving service mgr spec with placement smithi097=y;smithi144=x;count:2 2024-03-21T13:22:30.989 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:30 smithi097 ceph-mon[39416]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi097=y;smithi144=x", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:22:30.989 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:30 smithi097 ceph-mon[39416]: Saving service mgr spec with placement smithi097=y;smithi144=x;count:2 2024-03-21T13:22:30.991 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-03-21T13:22:30.991 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-03-21T13:22:30.991 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000189114 s, 2.7 MB/s 2024-03-21T13:22:30.992 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-21T13:22:31.050 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_4 2024-03-21T13:22:31.104 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-21T13:22:31.105 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T13:22:31.105 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-03-21T13:22:31.105 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T13:22:31.105 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T13:22:31.105 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-03-21 13:22:15.181242432 +0000 2024-03-21T13:22:31.105 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-03-21 13:16:21.074170738 +0000 2024-03-21T13:22:31.105 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-03-21 13:16:21.074170738 +0000 2024-03-21T13:22:31.105 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-03-21 13:16:21.074170738 +0000 2024-03-21T13:22:31.105 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-21T13:22:31.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:30 smithi144 ceph-mon[33633]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi097=y;smithi144=x", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:22:31.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:30 smithi144 ceph-mon[33633]: Saving service mgr spec with placement smithi097=y;smithi144=x;count:2 2024-03-21T13:22:31.170 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-03-21T13:22:31.170 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-03-21T13:22:31.170 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000226031 s, 2.3 MB/s 2024-03-21T13:22:31.171 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-21T13:22:31.227 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-21T13:22:31.227 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/scratch_devs of=/dev/stdout 2024-03-21T13:22:31.242 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-21T13:22:31.243 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_1 2024-03-21T13:22:31.298 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-21T13:22:31.298 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T13:22:31.298 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-03-21T13:22:31.298 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T13:22:31.298 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T13:22:31.298 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-21 13:22:06.228780538 +0000 2024-03-21T13:22:31.298 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-21 13:16:20.213832637 +0000 2024-03-21T13:22:31.299 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-21 13:16:20.213832637 +0000 2024-03-21T13:22:31.299 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-21 13:16:20.213832637 +0000 2024-03-21T13:22:31.299 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-21T13:22:31.366 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-03-21T13:22:31.366 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-03-21T13:22:31.366 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000274862 s, 1.9 MB/s 2024-03-21T13:22:31.367 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-21T13:22:31.422 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_2 2024-03-21T13:22:31.476 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-21T13:22:31.476 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T13:22:31.476 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-03-21T13:22:31.476 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T13:22:31.477 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T13:22:31.477 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-21 13:22:06.229780520 +0000 2024-03-21T13:22:31.477 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-21 13:16:20.564826317 +0000 2024-03-21T13:22:31.477 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-21 13:16:20.564826317 +0000 2024-03-21T13:22:31.477 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-21 13:16:20.564826317 +0000 2024-03-21T13:22:31.477 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-21T13:22:31.543 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-03-21T13:22:31.543 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-03-21T13:22:31.543 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000173918 s, 2.9 MB/s 2024-03-21T13:22:31.544 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-21T13:22:31.599 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_3 2024-03-21T13:22:31.655 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-21T13:22:31.655 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T13:22:31.655 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-03-21T13:22:31.655 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T13:22:31.655 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T13:22:31.655 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-21 13:22:06.229780520 +0000 2024-03-21T13:22:31.655 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-21 13:16:20.905820178 +0000 2024-03-21T13:22:31.655 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-21 13:16:20.905820178 +0000 2024-03-21T13:22:31.656 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-21 13:16:20.905820178 +0000 2024-03-21T13:22:31.656 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-21T13:22:31.720 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-03-21T13:22:31.720 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-03-21T13:22:31.720 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000194194 s, 2.6 MB/s 2024-03-21T13:22:31.722 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-21T13:22:31.778 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_4 2024-03-21T13:22:31.833 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-21T13:22:31.833 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T13:22:31.834 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-03-21T13:22:31.834 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T13:22:31.834 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T13:22:31.834 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-21 13:22:06.230780502 +0000 2024-03-21T13:22:31.834 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-21 13:16:21.239814164 +0000 2024-03-21T13:22:31.834 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-21 13:16:21.239814164 +0000 2024-03-21T13:22:31.834 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-21 13:16:21.239814164 +0000 2024-03-21T13:22:31.834 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-21T13:22:31.891 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:31 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:31.891 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:31 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:31.892 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:31 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:22:31.892 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:31 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:31.892 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:31 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:22:31.892 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:31 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:31.892 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:31 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T13:22:31.892 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:31 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-21T13:22:31.892 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:31 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T13:22:31.892 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:31 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:31.892 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:31 smithi144 ceph-mon[33633]: Deploying daemon mgr.x on smithi144 2024-03-21T13:22:31.892 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:31 smithi144 ceph-mon[33633]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:31.899 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-03-21T13:22:31.900 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-03-21T13:22:31.900 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000181482 s, 2.8 MB/s 2024-03-21T13:22:31.901 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-21T13:22:31.956 INFO:tasks.cephadm:Deploying osd.0 on smithi097 with /dev/vg_nvme/lv_4... 2024-03-21T13:22:31.957 DEBUG:teuthology.orchestra.run.smithi097:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-21T13:22:32.197 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T13:22:32.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:31 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:32.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:31 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:32.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:31 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:22:32.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:31 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:32.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:31 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:22:32.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:31 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:32.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:31 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T13:22:32.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:31 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-21T13:22:32.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:31 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T13:22:32.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:31 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:32.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:31 smithi097 ceph-mon[30040]: Deploying daemon mgr.x on smithi144 2024-03-21T13:22:32.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:31 smithi097 ceph-mon[30040]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:32.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:31 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:32.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:31 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:32.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:31 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:22:32.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:31 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:32.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:31 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:22:32.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:31 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:32.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:31 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T13:22:32.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:31 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-21T13:22:32.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:31 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T13:22:32.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:31 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:32.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:31 smithi097 ceph-mon[39416]: Deploying daemon mgr.x on smithi144 2024-03-21T13:22:32.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:31 smithi097 ceph-mon[39416]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:33.201 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:22:33 smithi144 podman[35694]: 2024-03-21 13:22:33.034340423 +0000 UTC m=+0.017547331 image pull 19e12293512ff7978c8adb5f72a829060dd47f07bfd6903b6b3178d2ebc29e4e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93 2024-03-21T13:22:33.621 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:22:33 smithi144 podman[35694]: 2024-03-21 13:22:33.308323791 +0000 UTC m=+0.291530696 container init 7b6751b4e086b5441292b2e07303cdd91e646bf423bed6b6d60e487ac1a0b70f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5) 2024-03-21T13:22:33.621 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:22:33 smithi144 podman[35694]: 2024-03-21 13:22:33.314129297 +0000 UTC m=+0.297336192 container start 7b6751b4e086b5441292b2e07303cdd91e646bf423bed6b6d60e487ac1a0b70f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=) 2024-03-21T13:22:33.621 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:22:33 smithi144 ceph-mgr[35732]: -- 172.21.15.144:0/4210167220 <== mon.1 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559bb710c5a0 con 0x559bb70e0800 2024-03-21T13:22:33.621 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:22:33 smithi144 bash[35694]: 7b6751b4e086b5441292b2e07303cdd91e646bf423bed6b6d60e487ac1a0b70f 2024-03-21T13:22:33.621 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:22:33 smithi144 systemd[1]: Started Ceph mgr.x for a5dd1f32-e785-11ee-95cd-87774f69a715. 2024-03-21T13:22:34.135 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:22:33 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: 2024-03-21T13:22:33.873+0000 7f2094230200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-21T13:22:34.135 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:22:33 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: 2024-03-21T13:22:33.969+0000 7f2094230200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-21T13:22:34.157 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-21T13:22:34.157 INFO:teuthology.orchestra.run.smithi097.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-21T13:22:34.158 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10+0 records in 2024-03-21T13:22:34.158 INFO:teuthology.orchestra.run.smithi097.stderr:10+0 records out 2024-03-21T13:22:34.158 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115811 s, 905 MB/s 2024-03-21T13:22:34.158 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping successful for: 2024-03-21T13:22:34.443 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:34 smithi144 ceph-mon[33633]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:34.443 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:34 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:34.443 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:34 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:34.444 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:34 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:34.444 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:34 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:34.444 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:34 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:22:34.472 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:34 smithi097 ceph-mon[30040]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:34.473 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:34 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:34.473 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:34 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:34.473 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:34 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:34.473 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:34 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:34.473 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:34 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:22:34.473 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:34 smithi097 ceph-mon[39416]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:34.473 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:34 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:34.473 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:34 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:34.473 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:34 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:34.473 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:34 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:34.473 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:34 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:22:34.778 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:22:34 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: 2024-03-21T13:22:34.685+0000 7f2094230200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-21T13:22:34.778 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:22:34 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: 2024-03-21T13:22:34.777+0000 7f2094230200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-21T13:22:34.784 DEBUG:teuthology.orchestra.run.smithi097:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi097:vg_nvme/lv_4 2024-03-21T13:22:35.027 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T13:22:35.109 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:22:34 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: 2024-03-21T13:22:34.926+0000 7f2094230200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-21T13:22:35.109 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:22:35 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: 2024-03-21T13:22:35.023+0000 7f2094230200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-21T13:22:35.109 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:22:35 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: 2024-03-21T13:22:35.108+0000 7f2094230200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-21T13:22:35.663 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:22:35 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: 2024-03-21T13:22:35.373+0000 7f2094230200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-21T13:22:35.663 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:22:35 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: 2024-03-21T13:22:35.483+0000 7f2094230200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-21T13:22:35.663 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:22:35 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: 2024-03-21T13:22:35.568+0000 7f2094230200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-21T13:22:35.663 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:22:35 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: 2024-03-21T13:22:35.662+0000 7f2094230200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-21T13:22:35.950 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:22:35 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: 2024-03-21T13:22:35.838+0000 7f2094230200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-21T13:22:36.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:35 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:36.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:35 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:36.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:35 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:36.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:35 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:22:36.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:35 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:36.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:35 smithi097 ceph-mon[39416]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:36.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:35 smithi097 ceph-mon[39416]: Reconfiguring mgr.y (unknown last config time)... 2024-03-21T13:22:36.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:35 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T13:22:36.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:35 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T13:22:36.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:35 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:36.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:35 smithi097 ceph-mon[39416]: Reconfiguring daemon mgr.y on smithi097 2024-03-21T13:22:36.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:35 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:36.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:35 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:36.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:35 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:36.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:35 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:22:36.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:35 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:36.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:35 smithi097 ceph-mon[30040]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:36.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:35 smithi097 ceph-mon[30040]: Reconfiguring mgr.y (unknown last config time)... 2024-03-21T13:22:36.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:35 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T13:22:36.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:35 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T13:22:36.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:35 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:36.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:35 smithi097 ceph-mon[30040]: Reconfiguring daemon mgr.y on smithi097 2024-03-21T13:22:36.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:35 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:36.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:35 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:36.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:35 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:36.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:35 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:22:36.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:35 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:36.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:35 smithi144 ceph-mon[33633]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:36.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:35 smithi144 ceph-mon[33633]: Reconfiguring mgr.y (unknown last config time)... 2024-03-21T13:22:36.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:35 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T13:22:36.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:35 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T13:22:36.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:35 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:36.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:35 smithi144 ceph-mon[33633]: Reconfiguring daemon mgr.y on smithi097 2024-03-21T13:22:37.121 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:22:36 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: 2024-03-21T13:22:36.807+0000 7f2094230200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-21T13:22:37.121 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:22:36 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: 2024-03-21T13:22:36.937+0000 7f2094230200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-21T13:22:37.400 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:37 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:37.400 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:37 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:37.400 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:37 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:37.400 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:37 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:37.400 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:37 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:37.400 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:37 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:37.560 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:37 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:37.560 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:37 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:37.560 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:37 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:37.560 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:22:37 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: 2024-03-21T13:22:37.279+0000 7f2094230200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-21T13:22:37.560 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:22:37 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: 2024-03-21T13:22:37.367+0000 7f2094230200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-21T13:22:37.560 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:22:37 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: 2024-03-21T13:22:37.559+0000 7f2094230200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-21T13:22:37.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:37.697+0000 7fb7f68fc700 1 -- 172.21.15.97:0/2992441049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7f0102cd0 msgr2=0x7fb7f01050c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:22:37.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:37.697+0000 7fb7f68fc700 1 --2- 172.21.15.97:0/2992441049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7f0102cd0 0x7fb7f01050c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb7e0009990 tx=0x7fb7e0009ca0 comp rx=0 tx=0).stop 2024-03-21T13:22:37.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:37.698+0000 7fb7f68fc700 1 -- 172.21.15.97:0/2992441049 shutdown_connections 2024-03-21T13:22:37.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:37.698+0000 7fb7f68fc700 1 --2- 172.21.15.97:0/2992441049 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7f0107f30 0x7fb7f010a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:22:37.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:37.698+0000 7fb7f68fc700 1 --2- 172.21.15.97:0/2992441049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7f0102cd0 0x7fb7f01050c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:22:37.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:37.698+0000 7fb7f68fc700 1 --2- 172.21.15.97:0/2992441049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7f0105600 0x7fb7f01079f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:22:37.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:37.698+0000 7fb7f68fc700 1 -- 172.21.15.97:0/2992441049 >> 172.21.15.97:0/2992441049 conn(0x7fb7f00fc660 msgr2=0x7fb7f00feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:22:37.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:37.698+0000 7fb7f68fc700 1 -- 172.21.15.97:0/2992441049 shutdown_connections 2024-03-21T13:22:37.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:37.698+0000 7fb7f68fc700 1 -- 172.21.15.97:0/2992441049 wait complete. 2024-03-21T13:22:37.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:37.698+0000 7fb7f68fc700 1 Processor -- start 2024-03-21T13:22:37.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:37.698+0000 7fb7f68fc700 1 -- start start 2024-03-21T13:22:37.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:37.699+0000 7fb7f68fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7f0102cd0 0x7fb7f01959c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:22:37.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:37.699+0000 7fb7f68fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7f0105600 0x7fb7f0195f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:22:37.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:37.699+0000 7fb7f68fc700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7f0107f30 0x7fb7f019cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:22:37.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:37.699+0000 7fb7f68fc700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f010b3e0 con 0x7fb7f0105600 2024-03-21T13:22:37.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:37.699+0000 7fb7f68fc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f010b260 con 0x7fb7f0102cd0 2024-03-21T13:22:37.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:37.699+0000 7fb7f68fc700 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f010b560 con 0x7fb7f0107f30 2024-03-21T13:22:37.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:37.699+0000 7fb7f4e99700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7f0107f30 0x7fb7f019cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:22:37.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:37.699+0000 7fb7f4e99700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7f0107f30 0x7fb7f019cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.97:38102/0 (socket says 172.21.15.97:38102) 2024-03-21T13:22:37.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:37.699+0000 7fb7f4e99700 1 -- 172.21.15.97:0/3677506153 learned_addr learned my addr 172.21.15.97:0/3677506153 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:22:37.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:37.699+0000 7fb7ef7fe700 1 --2- 172.21.15.97:0/3677506153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7f0105600 0x7fb7f0195f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:22:37.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:37.699+0000 7fb7effff700 1 --2- 172.21.15.97:0/3677506153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7f0102cd0 0x7fb7f01959c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:22:37.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:37.699+0000 7fb7f4e99700 1 -- 172.21.15.97:0/3677506153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7f0102cd0 msgr2=0x7fb7f01959c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:22:37.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:37.699+0000 7fb7f4e99700 1 --2- 172.21.15.97:0/3677506153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7f0102cd0 0x7fb7f01959c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:22:37.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:37.699+0000 7fb7f4e99700 1 -- 172.21.15.97:0/3677506153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7f0105600 msgr2=0x7fb7f0195f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:22:37.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:37.699+0000 7fb7f4e99700 1 --2- 172.21.15.97:0/3677506153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7f0105600 0x7fb7f0195f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:22:37.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:37.699+0000 7fb7f4e99700 1 -- 172.21.15.97:0/3677506153 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f019d620 con 0x7fb7f0107f30 2024-03-21T13:22:37.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:37.699+0000 7fb7ef7fe700 1 --2- 172.21.15.97:0/3677506153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7f0105600 0x7fb7f0195f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T13:22:37.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:37.699+0000 7fb7f4e99700 1 --2- 172.21.15.97:0/3677506153 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7f0107f30 0x7fb7f019cf80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb7e80124b0 tx=0x7fb7e80127c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:22:37.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:37.700+0000 7fb7ed7fa700 1 -- 172.21.15.97:0/3677506153 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e80090d0 con 0x7fb7f0107f30 2024-03-21T13:22:37.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:37.700+0000 7fb7f68fc700 1 -- 172.21.15.97:0/3677506153 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7f019d8b0 con 0x7fb7f0107f30 2024-03-21T13:22:37.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:37.700+0000 7fb7ed7fa700 1 -- 172.21.15.97:0/3677506153 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fb7e8009270 con 0x7fb7f0107f30 2024-03-21T13:22:37.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:37.700+0000 7fb7ed7fa700 1 -- 172.21.15.97:0/3677506153 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8007500 con 0x7fb7f0107f30 2024-03-21T13:22:37.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:37.700+0000 7fb7f68fc700 1 -- 172.21.15.97:0/3677506153 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7f019dd80 con 0x7fb7f0107f30 2024-03-21T13:22:37.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:37.701+0000 7fb7ed7fa700 1 -- 172.21.15.97:0/3677506153 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 13) v1 ==== 46315+0+0 (secure 0 0 0) 0x7fb7e801f030 con 0x7fb7f0107f30 2024-03-21T13:22:37.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:37.701+0000 7fb7ed7fa700 1 --2- 172.21.15.97:0/3677506153 >> v2:172.21.15.97:6800/1943212787 conn(0x7fb7d8039420 0x7fb7d803b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:22:37.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:37.701+0000 7fb7ed7fa700 1 -- 172.21.15.97:0/3677506153 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (secure 0 0 0) 0x7fb7e804ff70 con 0x7fb7f0107f30 2024-03-21T13:22:37.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:37.701+0000 7fb7effff700 1 --2- 172.21.15.97:0/3677506153 >> v2:172.21.15.97:6800/1943212787 conn(0x7fb7d8039420 0x7fb7d803b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:22:37.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:37.701+0000 7fb7effff700 1 --2- 172.21.15.97:0/3677506153 >> v2:172.21.15.97:6800/1943212787 conn(0x7fb7d8039420 0x7fb7d803b8e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb7e000b2e0 tx=0x7fb7e0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:22:37.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:37.701+0000 7fb7f68fc700 1 -- 172.21.15.97:0/3677506153 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7f0068e50 con 0x7fb7f0107f30 2024-03-21T13:22:37.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:37.707+0000 7fb7ed7fa700 1 -- 172.21.15.97:0/3677506153 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb7e8026020 con 0x7fb7f0107f30 2024-03-21T13:22:37.871 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:22:37 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: 2024-03-21T13:22:37.741+0000 7f2094230200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-21T13:22:37.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:37.947+0000 7fb7f68fc700 1 -- 172.21.15.97:0/3677506153 --> v2:172.21.15.97:6800/1943212787 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fb7f002db40 con 0x7fb7d8039420 2024-03-21T13:22:38.290 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:22:38 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: 2024-03-21T13:22:38.027+0000 7f2094230200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-21T13:22:38.291 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:22:38 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: 2024-03-21T13:22:38.290+0000 7f2094230200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-21T13:22:38.291 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:38 smithi144 ceph-mon[33633]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:38.291 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:38 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:22:38.291 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:38 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:38.291 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:38 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:22:38.291 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:38 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:38.291 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:38 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T13:22:38.291 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:38 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T13:22:38.291 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:38 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:38.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:38 smithi097 ceph-mon[30040]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:38.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:38 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:22:38.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:38 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:38.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:38 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:22:38.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:38 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:38.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:38 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T13:22:38.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:38 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T13:22:38.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:38 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:38.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:38 smithi097 ceph-mon[39416]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:38.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:38 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:22:38.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:38 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:38.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:38 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:22:38.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:38 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:38.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:38 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T13:22:38.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:38 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T13:22:38.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:38 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:38.621 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:22:38 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: 2024-03-21T13:22:38.525+0000 7f2094230200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-21T13:22:39.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:39 smithi097 ceph-mon[30040]: from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:22:39.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:39 smithi097 ceph-mon[39416]: from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:22:39.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:39 smithi144 ceph-mon[33633]: from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:22:39.621 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:22:39 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: 2024-03-21T13:22:39.326+0000 7f2094230200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-21T13:22:40.148 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:22:39 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: 2024-03-21T13:22:39.893+0000 7f2094230200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-21T13:22:40.148 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:22:39 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: 2024-03-21T13:22:39.990+0000 7f2094230200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-21T13:22:40.408 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:40 smithi144 ceph-mon[33633]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:40.409 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:22:40 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: 2024-03-21T13:22:40.208+0000 7f2094230200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-21T13:22:40.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:40 smithi097 ceph-mon[30040]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:40.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:40 smithi097 ceph-mon[39416]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:41.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:41.158+0000 7fb7ed7fa700 1 -- 172.21.15.97:0/3677506153 <== mon.2 v2:172.21.15.97:3301/0 7 ==== mgrmap(e 14) v1 ==== 92039+0+0 (secure 0 0 0) 0x7fb7e8017070 con 0x7fb7f0107f30 2024-03-21T13:22:41.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:41 smithi097 ceph-mon[30040]: Standby manager daemon x started 2024-03-21T13:22:41.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:41 smithi097 ceph-mon[30040]: from='mgr.? 172.21.15.144:0/3116876657' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-21T13:22:41.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:41 smithi097 ceph-mon[30040]: from='mgr.? 172.21.15.144:0/3116876657' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T13:22:41.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:41 smithi097 ceph-mon[30040]: from='mgr.? 172.21.15.144:0/3116876657' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-21T13:22:41.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:41 smithi097 ceph-mon[30040]: from='mgr.? 172.21.15.144:0/3116876657' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T13:22:41.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:41 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2711017008' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "da68a71b-2182-4471-a6f8-4aa689009b20"}]: dispatch 2024-03-21T13:22:41.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:41 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2711017008' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "da68a71b-2182-4471-a6f8-4aa689009b20"}]': finished 2024-03-21T13:22:41.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:41 smithi097 ceph-mon[30040]: osdmap e5: 1 total, 0 up, 1 in 2024-03-21T13:22:41.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:41 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T13:22:41.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:41 smithi097 ceph-mon[39416]: Standby manager daemon x started 2024-03-21T13:22:41.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:41 smithi097 ceph-mon[39416]: from='mgr.? 172.21.15.144:0/3116876657' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-21T13:22:41.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:41 smithi097 ceph-mon[39416]: from='mgr.? 172.21.15.144:0/3116876657' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T13:22:41.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:41 smithi097 ceph-mon[39416]: from='mgr.? 172.21.15.144:0/3116876657' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-21T13:22:41.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:41 smithi097 ceph-mon[39416]: from='mgr.? 172.21.15.144:0/3116876657' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T13:22:41.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:41 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2711017008' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "da68a71b-2182-4471-a6f8-4aa689009b20"}]: dispatch 2024-03-21T13:22:41.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:41 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2711017008' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "da68a71b-2182-4471-a6f8-4aa689009b20"}]': finished 2024-03-21T13:22:41.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:41 smithi097 ceph-mon[39416]: osdmap e5: 1 total, 0 up, 1 in 2024-03-21T13:22:41.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:41 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T13:22:41.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:41 smithi144 ceph-mon[33633]: Standby manager daemon x started 2024-03-21T13:22:41.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:41 smithi144 ceph-mon[33633]: from='mgr.? 172.21.15.144:0/3116876657' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-21T13:22:41.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:41 smithi144 ceph-mon[33633]: from='mgr.? 172.21.15.144:0/3116876657' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T13:22:41.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:41 smithi144 ceph-mon[33633]: from='mgr.? 172.21.15.144:0/3116876657' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-21T13:22:41.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:41 smithi144 ceph-mon[33633]: from='mgr.? 172.21.15.144:0/3116876657' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T13:22:41.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:41 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2711017008' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "da68a71b-2182-4471-a6f8-4aa689009b20"}]: dispatch 2024-03-21T13:22:41.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:41 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2711017008' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "da68a71b-2182-4471-a6f8-4aa689009b20"}]': finished 2024-03-21T13:22:41.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:41 smithi144 ceph-mon[33633]: osdmap e5: 1 total, 0 up, 1 in 2024-03-21T13:22:41.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:41 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T13:22:42.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:42 smithi097 ceph-mon[30040]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:42.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:42 smithi097 ceph-mon[30040]: mgrmap e14: y(active, since 70s), standbys: x 2024-03-21T13:22:42.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:42 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-21T13:22:42.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:42 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/881592129' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T13:22:42.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:42 smithi097 ceph-mon[39416]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:42.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:42 smithi097 ceph-mon[39416]: mgrmap e14: y(active, since 70s), standbys: x 2024-03-21T13:22:42.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:42 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-21T13:22:42.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:42 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/881592129' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T13:22:42.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:42 smithi144 ceph-mon[33633]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:42.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:42 smithi144 ceph-mon[33633]: mgrmap e14: y(active, since 70s), standbys: x 2024-03-21T13:22:42.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:42 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-21T13:22:42.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:42 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/881592129' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T13:22:44.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:44 smithi097 ceph-mon[30040]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:44.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:44 smithi097 ceph-mon[39416]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:44.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:44 smithi144 ceph-mon[33633]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:46.542 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:46 smithi097 ceph-mon[30040]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:46.542 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:46 smithi097 ceph-mon[39416]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:46.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:46 smithi144 ceph-mon[33633]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:47.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:47 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-21T13:22:47.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:47 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:47.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:47 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-21T13:22:47.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:47 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:47.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:47 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-21T13:22:47.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:47 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:48.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:48 smithi097 ceph-mon[30040]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:48.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:48 smithi097 ceph-mon[30040]: Deploying daemon osd.0 on smithi097 2024-03-21T13:22:48.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:48 smithi097 ceph-mon[39416]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:48.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:48 smithi097 ceph-mon[39416]: Deploying daemon osd.0 on smithi097 2024-03-21T13:22:48.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:48 smithi144 ceph-mon[33633]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:48.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:48 smithi144 ceph-mon[33633]: Deploying daemon osd.0 on smithi097 2024-03-21T13:22:50.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:50 smithi097 ceph-mon[30040]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:50.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:50 smithi097 ceph-mon[39416]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:50.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:50 smithi144 ceph-mon[33633]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:52.440 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:52 smithi097 ceph-mon[30040]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:52.440 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:52 smithi097 ceph-mon[39416]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:52.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:52 smithi144 ceph-mon[33633]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:54.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:53 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:54.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:53 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:54.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:53 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:22:54.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:53 smithi097 ceph-mon[30040]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:54.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:53 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:54.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:53 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:54.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:53 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:22:54.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:53 smithi097 ceph-mon[39416]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:54.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:53 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:54.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:53 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:54.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:53 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:22:54.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:53 smithi144 ceph-mon[33633]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:56.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:56 smithi097 ceph-mon[30040]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:56.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:56 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:56.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:56 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:56.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:56 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:56.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:56 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:22:56.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:56 smithi097 ceph-mon[30040]: from='osd.0 v2:172.21.15.97:6801/2691001097' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-21T13:22:56.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:56 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:56.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:56 smithi097 ceph-mon[39416]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:56.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:56 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:56.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:56 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:56.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:56 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:56.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:56 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:22:56.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:56 smithi097 ceph-mon[39416]: from='osd.0 v2:172.21.15.97:6801/2691001097' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-21T13:22:56.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:56 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:56.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:56 smithi144 ceph-mon[33633]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:56.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:56 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:56.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:56 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:56.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:56 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:56.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:56 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:22:56.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:56 smithi144 ceph-mon[33633]: from='osd.0 v2:172.21.15.97:6801/2691001097' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-21T13:22:56.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:56 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:56.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:56.700+0000 7fb7ed7fa700 1 -- 172.21.15.97:0/3677506153 <== mgr.14152 v2:172.21.15.97:6800/1943212787 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb7f002db40 con 0x7fb7d8039420 2024-03-21T13:22:56.701 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 0 on host 'smithi097' 2024-03-21T13:22:56.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:56.702+0000 7fb7f68fc700 1 -- 172.21.15.97:0/3677506153 >> v2:172.21.15.97:6800/1943212787 conn(0x7fb7d8039420 msgr2=0x7fb7d803b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:22:56.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:56.702+0000 7fb7f68fc700 1 --2- 172.21.15.97:0/3677506153 >> v2:172.21.15.97:6800/1943212787 conn(0x7fb7d8039420 0x7fb7d803b8e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb7e000b2e0 tx=0x7fb7e0005fb0 comp rx=0 tx=0).stop 2024-03-21T13:22:56.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:56.702+0000 7fb7f68fc700 1 -- 172.21.15.97:0/3677506153 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7f0107f30 msgr2=0x7fb7f019cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:22:56.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:56.702+0000 7fb7f68fc700 1 --2- 172.21.15.97:0/3677506153 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7f0107f30 0x7fb7f019cf80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb7e80124b0 tx=0x7fb7e80127c0 comp rx=0 tx=0).stop 2024-03-21T13:22:56.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:56.703+0000 7fb7f68fc700 1 -- 172.21.15.97:0/3677506153 shutdown_connections 2024-03-21T13:22:56.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:56.703+0000 7fb7f68fc700 1 --2- 172.21.15.97:0/3677506153 >> v2:172.21.15.97:6800/1943212787 conn(0x7fb7d8039420 0x7fb7d803b8e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:22:56.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:56.703+0000 7fb7f68fc700 1 --2- 172.21.15.97:0/3677506153 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7f0107f30 0x7fb7f019cf80 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:22:56.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:56.703+0000 7fb7f68fc700 1 --2- 172.21.15.97:0/3677506153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7f0102cd0 0x7fb7f01959c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:22:56.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:56.703+0000 7fb7f68fc700 1 --2- 172.21.15.97:0/3677506153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7f0105600 0x7fb7f0195f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:22:56.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:56.703+0000 7fb7f68fc700 1 -- 172.21.15.97:0/3677506153 >> 172.21.15.97:0/3677506153 conn(0x7fb7f00fc660 msgr2=0x7fb7f01036d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:22:56.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:56.703+0000 7fb7f68fc700 1 -- 172.21.15.97:0/3677506153 shutdown_connections 2024-03-21T13:22:56.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:22:56.703+0000 7fb7f68fc700 1 -- 172.21.15.97:0/3677506153 wait complete. 2024-03-21T13:22:57.358 DEBUG:teuthology.orchestra.run.smithi097:osd.0> sudo journalctl -f -n 0 -u ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@osd.0.service 2024-03-21T13:22:57.361 INFO:tasks.cephadm:Deploying osd.1 on smithi097 with /dev/vg_nvme/lv_3... 2024-03-21T13:22:57.361 DEBUG:teuthology.orchestra.run.smithi097:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-21T13:22:57.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:57 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:57.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:57 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:57.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:57 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:22:57.889 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:57 smithi097 ceph-mon[30040]: from='osd.0 v2:172.21.15.97:6801/2691001097' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-21T13:22:57.889 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:57 smithi097 ceph-mon[30040]: osdmap e6: 1 total, 0 up, 1 in 2024-03-21T13:22:57.889 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:57 smithi097 ceph-mon[30040]: from='osd.0 v2:172.21.15.97:6801/2691001097' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-03-21T13:22:57.889 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:57 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T13:22:57.889 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:57 smithi097 ceph-mon[30040]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:57.889 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:57 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:57.889 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:57 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:57.889 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:57 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:22:57.889 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:57 smithi097 ceph-mon[39416]: from='osd.0 v2:172.21.15.97:6801/2691001097' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-21T13:22:57.889 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:57 smithi097 ceph-mon[39416]: osdmap e6: 1 total, 0 up, 1 in 2024-03-21T13:22:57.890 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:57 smithi097 ceph-mon[39416]: from='osd.0 v2:172.21.15.97:6801/2691001097' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-03-21T13:22:57.890 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:57 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T13:22:57.890 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:57 smithi097 ceph-mon[39416]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:58.104 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T13:22:58.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:57 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:58.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:57 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:22:58.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:57 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:22:58.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:57 smithi144 ceph-mon[33633]: from='osd.0 v2:172.21.15.97:6801/2691001097' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-21T13:22:58.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:57 smithi144 ceph-mon[33633]: osdmap e6: 1 total, 0 up, 1 in 2024-03-21T13:22:58.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:57 smithi144 ceph-mon[33633]: from='osd.0 v2:172.21.15.97:6801/2691001097' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-03-21T13:22:58.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:57 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T13:22:58.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:57 smithi144 ceph-mon[33633]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:58.474 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 21 13:22:58 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-0[43810]: 2024-03-21T13:22:58.266+0000 7f4674d2d700 -1 osd.0 0 waiting for initial osdmap 2024-03-21T13:22:58.475 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 21 13:22:58 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-0[43810]: 2024-03-21T13:22:58.273+0000 7f466cb18700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T13:22:59.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:58 smithi097 ceph-mon[39416]: purged_snaps scrub starts 2024-03-21T13:22:59.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:58 smithi097 ceph-mon[39416]: purged_snaps scrub ok 2024-03-21T13:22:59.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:58 smithi097 ceph-mon[39416]: from='osd.0 v2:172.21.15.97:6801/2691001097' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-03-21T13:22:59.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:58 smithi097 ceph-mon[39416]: osdmap e7: 1 total, 0 up, 1 in 2024-03-21T13:22:59.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:58 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T13:22:59.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:58 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T13:22:59.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:58 smithi097 ceph-mon[39416]: from='osd.0 v2:172.21.15.97:6801/2691001097' entity='osd.0' 2024-03-21T13:22:59.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:58 smithi097 ceph-mon[30040]: purged_snaps scrub starts 2024-03-21T13:22:59.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:58 smithi097 ceph-mon[30040]: purged_snaps scrub ok 2024-03-21T13:22:59.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:58 smithi097 ceph-mon[30040]: from='osd.0 v2:172.21.15.97:6801/2691001097' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-03-21T13:22:59.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:58 smithi097 ceph-mon[30040]: osdmap e7: 1 total, 0 up, 1 in 2024-03-21T13:22:59.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:58 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T13:22:59.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:58 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T13:22:59.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:58 smithi097 ceph-mon[30040]: from='osd.0 v2:172.21.15.97:6801/2691001097' entity='osd.0' 2024-03-21T13:22:59.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:58 smithi144 ceph-mon[33633]: purged_snaps scrub starts 2024-03-21T13:22:59.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:58 smithi144 ceph-mon[33633]: purged_snaps scrub ok 2024-03-21T13:22:59.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:58 smithi144 ceph-mon[33633]: from='osd.0 v2:172.21.15.97:6801/2691001097' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-03-21T13:22:59.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:58 smithi144 ceph-mon[33633]: osdmap e7: 1 total, 0 up, 1 in 2024-03-21T13:22:59.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:58 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T13:22:59.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:58 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T13:22:59.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:58 smithi144 ceph-mon[33633]: from='osd.0 v2:172.21.15.97:6801/2691001097' entity='osd.0' 2024-03-21T13:22:59.994 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:59 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T13:22:59.994 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:59 smithi097 ceph-mon[39416]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:59.994 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:59 smithi097 ceph-mon[39416]: osd.0 v2:172.21.15.97:6801/2691001097 boot 2024-03-21T13:22:59.994 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:59 smithi097 ceph-mon[39416]: osdmap e8: 1 total, 1 up, 1 in 2024-03-21T13:22:59.994 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:22:59 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T13:22:59.994 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:59 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T13:22:59.994 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:59 smithi097 ceph-mon[30040]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:22:59.994 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:59 smithi097 ceph-mon[30040]: osd.0 v2:172.21.15.97:6801/2691001097 boot 2024-03-21T13:22:59.994 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:59 smithi097 ceph-mon[30040]: osdmap e8: 1 total, 1 up, 1 in 2024-03-21T13:22:59.995 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:22:59 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T13:23:00.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:59 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T13:23:00.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:59 smithi144 ceph-mon[33633]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T13:23:00.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:59 smithi144 ceph-mon[33633]: osd.0 v2:172.21.15.97:6801/2691001097 boot 2024-03-21T13:23:00.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:59 smithi144 ceph-mon[33633]: osdmap e8: 1 total, 1 up, 1 in 2024-03-21T13:23:00.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:22:59 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T13:23:01.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:01 smithi097 ceph-mon[30040]: osdmap e9: 1 total, 1 up, 1 in 2024-03-21T13:23:01.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:01 smithi097 ceph-mon[30040]: Detected new or changed devices on smithi097 2024-03-21T13:23:01.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:01 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:01.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:01 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:01.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:01 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:23:01.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:01 smithi097 ceph-mon[30040]: Adjusting osd_memory_target on smithi097 to 16064M 2024-03-21T13:23:01.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:01 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:01.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:01 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:23:01.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:01 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:23:01.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:01 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:01.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:01 smithi097 ceph-mon[30040]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T13:23:01.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:01 smithi097 ceph-mon[39416]: osdmap e9: 1 total, 1 up, 1 in 2024-03-21T13:23:01.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:01 smithi097 ceph-mon[39416]: Detected new or changed devices on smithi097 2024-03-21T13:23:01.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:01 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:01.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:01 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:01.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:01 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:23:01.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:01 smithi097 ceph-mon[39416]: Adjusting osd_memory_target on smithi097 to 16064M 2024-03-21T13:23:01.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:01 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:01.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:01 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:23:01.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:01 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:23:01.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:01 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:01.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:01 smithi097 ceph-mon[39416]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T13:23:01.576 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-21T13:23:01.576 INFO:teuthology.orchestra.run.smithi097.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-21T13:23:01.576 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10+0 records in 2024-03-21T13:23:01.576 INFO:teuthology.orchestra.run.smithi097.stderr:10+0 records out 2024-03-21T13:23:01.576 INFO:teuthology.orchestra.run.smithi097.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0147072 s, 713 MB/s 2024-03-21T13:23:01.576 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping successful for: 2024-03-21T13:23:01.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:01 smithi144 ceph-mon[33633]: osdmap e9: 1 total, 1 up, 1 in 2024-03-21T13:23:01.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:01 smithi144 ceph-mon[33633]: Detected new or changed devices on smithi097 2024-03-21T13:23:01.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:01 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:01.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:01 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:01.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:01 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:23:01.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:01 smithi144 ceph-mon[33633]: Adjusting osd_memory_target on smithi097 to 16064M 2024-03-21T13:23:01.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:01 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:01.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:01 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:23:01.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:01 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:23:01.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:01 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:01.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:01 smithi144 ceph-mon[33633]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T13:23:02.283 DEBUG:teuthology.orchestra.run.smithi097:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi097:vg_nvme/lv_3 2024-03-21T13:23:02.520 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T13:23:04.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:04.025+0000 7f106f991700 1 -- 172.21.15.97:0/715690246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1068107fc0 msgr2=0x7f106810a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:23:04.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:04.025+0000 7f106f991700 1 --2- 172.21.15.97:0/715690246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1068107fc0 0x7f106810a3b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f106000b0a0 tx=0x7f106000b3b0 comp rx=0 tx=0).stop 2024-03-21T13:23:04.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:04.025+0000 7f106f991700 1 -- 172.21.15.97:0/715690246 shutdown_connections 2024-03-21T13:23:04.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:04.025+0000 7f106f991700 1 --2- 172.21.15.97:0/715690246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1068107fc0 0x7f106810a3b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:23:04.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:04.025+0000 7f106f991700 1 --2- 172.21.15.97:0/715690246 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1068102d60 0x7f1068105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:23:04.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:04.025+0000 7f106f991700 1 --2- 172.21.15.97:0/715690246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1068105690 0x7f1068107a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:23:04.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:04.025+0000 7f106f991700 1 -- 172.21.15.97:0/715690246 >> 172.21.15.97:0/715690246 conn(0x7f10680fc6f0 msgr2=0x7f10680feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:23:04.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:04.025+0000 7f106f991700 1 -- 172.21.15.97:0/715690246 shutdown_connections 2024-03-21T13:23:04.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:04.025+0000 7f106f991700 1 -- 172.21.15.97:0/715690246 wait complete. 2024-03-21T13:23:04.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:04.026+0000 7f106f991700 1 Processor -- start 2024-03-21T13:23:04.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:04.026+0000 7f106f991700 1 -- start start 2024-03-21T13:23:04.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:04.026+0000 7f106f991700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1068102d60 0x7f1068199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:23:04.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:04.026+0000 7f106f991700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1068105690 0x7f106819a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:23:04.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:04.026+0000 7f106f991700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1068107fc0 0x7f10681a1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:23:04.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:04.026+0000 7f106f991700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106810d480 con 0x7f1068105690 2024-03-21T13:23:04.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:04.026+0000 7f106f991700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106810d300 con 0x7f1068107fc0 2024-03-21T13:23:04.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:04.026+0000 7f106f991700 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f106810d600 con 0x7f1068102d60 2024-03-21T13:23:04.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:04.026+0000 7f106d72d700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1068102d60 0x7f1068199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:23:04.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:04.026+0000 7f106cf2c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1068105690 0x7f106819a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:23:04.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:04.026+0000 7f106cf2c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1068105690 0x7f106819a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53094/0 (socket says 172.21.15.97:53094) 2024-03-21T13:23:04.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:04.026+0000 7f106cf2c700 1 -- 172.21.15.97:0/597605424 learned_addr learned my addr 172.21.15.97:0/597605424 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:23:04.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:04.026+0000 7f106d72d700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1068102d60 0x7f1068199c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.97:33790/0 (socket says 172.21.15.97:33790) 2024-03-21T13:23:04.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:04.026+0000 7f106cf2c700 1 -- 172.21.15.97:0/597605424 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1068102d60 msgr2=0x7f1068199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:23:04.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:04.026+0000 7f106df2e700 1 --2- 172.21.15.97:0/597605424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1068107fc0 0x7f10681a1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:23:04.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:04.026+0000 7f106cf2c700 1 --2- 172.21.15.97:0/597605424 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1068102d60 0x7f1068199c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:23:04.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:04.026+0000 7f106cf2c700 1 -- 172.21.15.97:0/597605424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1068107fc0 msgr2=0x7f10681a1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:23:04.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:04.026+0000 7f106cf2c700 1 --2- 172.21.15.97:0/597605424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1068107fc0 0x7f10681a1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:23:04.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:04.026+0000 7f106cf2c700 1 -- 172.21.15.97:0/597605424 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10681a18e0 con 0x7f1068105690 2024-03-21T13:23:04.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:04.026+0000 7f106d72d700 1 --2- 172.21.15.97:0/597605424 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1068102d60 0x7f1068199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T13:23:04.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:04.027+0000 7f106cf2c700 1 --2- 172.21.15.97:0/597605424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1068105690 0x7f106819a1a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f106400c920 tx=0x7f106400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:23:04.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:04.027+0000 7f105e7fc700 1 -- 172.21.15.97:0/597605424 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1064007b00 con 0x7f1068105690 2024-03-21T13:23:04.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:04.027+0000 7f105e7fc700 1 -- 172.21.15.97:0/597605424 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f1064004ce0 con 0x7f1068105690 2024-03-21T13:23:04.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:04.027+0000 7f106f991700 1 -- 172.21.15.97:0/597605424 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10681a1bd0 con 0x7f1068105690 2024-03-21T13:23:04.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:04.027+0000 7f105e7fc700 1 -- 172.21.15.97:0/597605424 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1064005580 con 0x7f1068105690 2024-03-21T13:23:04.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:04.027+0000 7f106f991700 1 -- 172.21.15.97:0/597605424 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10681a2110 con 0x7f1068105690 2024-03-21T13:23:04.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:04.028+0000 7f106f991700 1 -- 172.21.15.97:0/597605424 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1068068e50 con 0x7f1068105690 2024-03-21T13:23:04.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:04.028+0000 7f105e7fc700 1 -- 172.21.15.97:0/597605424 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 14) v1 ==== 92039+0+0 (secure 0 0 0) 0x7f1064004830 con 0x7f1068105690 2024-03-21T13:23:04.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:04.029+0000 7f105e7fc700 1 --2- 172.21.15.97:0/597605424 >> v2:172.21.15.97:6800/1943212787 conn(0x7f105406e180 0x7f1054070640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:23:04.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:04.029+0000 7f105e7fc700 1 -- 172.21.15.97:0/597605424 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1541+0+0 (secure 0 0 0) 0x7f106408ce30 con 0x7f1068105690 2024-03-21T13:23:04.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:04.029+0000 7f106d72d700 1 --2- 172.21.15.97:0/597605424 >> v2:172.21.15.97:6800/1943212787 conn(0x7f105406e180 0x7f1054070640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:23:04.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:04.029+0000 7f106d72d700 1 --2- 172.21.15.97:0/597605424 >> v2:172.21.15.97:6800/1943212787 conn(0x7f105406e180 0x7f1054070640 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f106819ab40 tx=0x7f1058009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:23:04.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:04.033+0000 7f105e7fc700 1 -- 172.21.15.97:0/597605424 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1064014030 con 0x7f1068105690 2024-03-21T13:23:04.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:04.241+0000 7f106f991700 1 -- 172.21.15.97:0/597605424 --> v2:172.21.15.97:6800/1943212787 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f10680625d0 con 0x7f105406e180 2024-03-21T13:23:04.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:04 smithi097 ceph-mon[30040]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T13:23:04.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:04 smithi097 ceph-mon[39416]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T13:23:04.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:04 smithi144 ceph-mon[33633]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T13:23:05.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:05 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T13:23:05.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:05 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T13:23:05.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:05 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:23:05.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:05 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T13:23:05.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:05 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T13:23:05.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:05 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:23:05.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:05 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T13:23:05.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:05 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T13:23:05.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:05 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:23:06.412 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:06 smithi097 ceph-mon[30040]: from='client.14247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:23:06.413 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:06 smithi097 ceph-mon[30040]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T13:23:06.413 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:06 smithi097 ceph-mon[39416]: from='client.14247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:23:06.413 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:06 smithi097 ceph-mon[39416]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T13:23:06.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:06 smithi144 ceph-mon[33633]: from='client.14247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:23:06.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:06 smithi144 ceph-mon[33633]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T13:23:07.400 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:07 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1070643099' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "19b99056-a65c-47ec-9392-cc0d40b3b399"}]: dispatch 2024-03-21T13:23:07.400 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:07 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1070643099' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "19b99056-a65c-47ec-9392-cc0d40b3b399"}]': finished 2024-03-21T13:23:07.400 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:07 smithi097 ceph-mon[39416]: osdmap e10: 2 total, 1 up, 2 in 2024-03-21T13:23:07.400 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:07 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T13:23:07.400 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:07 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1070643099' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "19b99056-a65c-47ec-9392-cc0d40b3b399"}]: dispatch 2024-03-21T13:23:07.401 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:07 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1070643099' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "19b99056-a65c-47ec-9392-cc0d40b3b399"}]': finished 2024-03-21T13:23:07.401 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:07 smithi097 ceph-mon[30040]: osdmap e10: 2 total, 1 up, 2 in 2024-03-21T13:23:07.401 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:07 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T13:23:07.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:07 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1070643099' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "19b99056-a65c-47ec-9392-cc0d40b3b399"}]: dispatch 2024-03-21T13:23:07.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:07 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1070643099' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "19b99056-a65c-47ec-9392-cc0d40b3b399"}]': finished 2024-03-21T13:23:07.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:07 smithi144 ceph-mon[33633]: osdmap e10: 2 total, 1 up, 2 in 2024-03-21T13:23:07.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:07 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T13:23:08.442 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:08 smithi097 ceph-mon[30040]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T13:23:08.442 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:08 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4192716513' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T13:23:08.442 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:08 smithi097 ceph-mon[39416]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T13:23:08.442 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:08 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4192716513' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T13:23:08.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:08 smithi144 ceph-mon[33633]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T13:23:08.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:08 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4192716513' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T13:23:10.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:10 smithi097 ceph-mon[30040]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T13:23:10.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:10 smithi097 ceph-mon[39416]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T13:23:10.423 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:10 smithi144 ceph-mon[33633]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T13:23:12.366 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:12 smithi097 ceph-mon[30040]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T13:23:12.366 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:12 smithi097 ceph-mon[39416]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T13:23:12.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:12 smithi144 ceph-mon[33633]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T13:23:14.447 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:14 smithi097 ceph-mon[30040]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T13:23:14.448 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:14 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-21T13:23:14.448 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:14 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:23:14.448 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:14 smithi097 ceph-mon[39416]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T13:23:14.448 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:14 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-21T13:23:14.448 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:14 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:23:14.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:14 smithi144 ceph-mon[33633]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T13:23:14.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:14 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-21T13:23:14.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:14 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:23:15.420 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:15 smithi097 ceph-mon[30040]: Deploying daemon osd.1 on smithi097 2024-03-21T13:23:15.420 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:15 smithi097 ceph-mon[39416]: Deploying daemon osd.1 on smithi097 2024-03-21T13:23:15.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:15 smithi144 ceph-mon[33633]: Deploying daemon osd.1 on smithi097 2024-03-21T13:23:16.434 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:16 smithi097 ceph-mon[39416]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T13:23:16.434 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:16 smithi097 ceph-mon[30040]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T13:23:16.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:16 smithi144 ceph-mon[33633]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T13:23:18.500 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:18 smithi097 ceph-mon[30040]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T13:23:18.500 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:18 smithi097 ceph-mon[39416]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T13:23:18.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:18 smithi144 ceph-mon[33633]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T13:23:20.299 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:20 smithi097 ceph-mon[30040]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T13:23:20.299 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:20 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:20.299 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:20 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:20.299 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:20 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:23:20.299 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:20 smithi097 ceph-mon[39416]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T13:23:20.300 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:20 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:20.300 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:20 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:20.300 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:20 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:23:20.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:20 smithi144 ceph-mon[33633]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T13:23:20.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:20 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:20.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:20 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:20.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:20 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:23:22.273 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:22 smithi097 ceph-mon[30040]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T13:23:22.273 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:22 smithi097 ceph-mon[39416]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T13:23:22.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:22 smithi144 ceph-mon[33633]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T13:23:22.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:22.992+0000 7f105e7fc700 1 -- 172.21.15.97:0/597605424 <== mgr.14152 v2:172.21.15.97:6800/1943212787 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f10680625d0 con 0x7f105406e180 2024-03-21T13:23:22.993 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 1 on host 'smithi097' 2024-03-21T13:23:22.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:22.995+0000 7f106f991700 1 -- 172.21.15.97:0/597605424 >> v2:172.21.15.97:6800/1943212787 conn(0x7f105406e180 msgr2=0x7f1054070640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:23:22.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:22.995+0000 7f106f991700 1 --2- 172.21.15.97:0/597605424 >> v2:172.21.15.97:6800/1943212787 conn(0x7f105406e180 0x7f1054070640 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f106819ab40 tx=0x7f1058009260 comp rx=0 tx=0).stop 2024-03-21T13:23:22.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:22.995+0000 7f106f991700 1 -- 172.21.15.97:0/597605424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1068105690 msgr2=0x7f106819a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:23:22.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:22.995+0000 7f106f991700 1 --2- 172.21.15.97:0/597605424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1068105690 0x7f106819a1a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f106400c920 tx=0x7f106400cc30 comp rx=0 tx=0).stop 2024-03-21T13:23:22.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:22.995+0000 7f106f991700 1 -- 172.21.15.97:0/597605424 shutdown_connections 2024-03-21T13:23:22.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:22.995+0000 7f106f991700 1 --2- 172.21.15.97:0/597605424 >> v2:172.21.15.97:6800/1943212787 conn(0x7f105406e180 0x7f1054070640 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:23:22.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:22.995+0000 7f106f991700 1 --2- 172.21.15.97:0/597605424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1068107fc0 0x7f10681a1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:23:22.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:22.995+0000 7f106f991700 1 --2- 172.21.15.97:0/597605424 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1068102d60 0x7f1068199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:23:22.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:22.995+0000 7f106f991700 1 --2- 172.21.15.97:0/597605424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1068105690 0x7f106819a1a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:23:22.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:22.995+0000 7f106f991700 1 -- 172.21.15.97:0/597605424 >> 172.21.15.97:0/597605424 conn(0x7f10680fc6f0 msgr2=0x7f1068100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:23:22.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:22.995+0000 7f106f991700 1 -- 172.21.15.97:0/597605424 shutdown_connections 2024-03-21T13:23:22.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:22.995+0000 7f106f991700 1 -- 172.21.15.97:0/597605424 wait complete. 2024-03-21T13:23:23.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:23 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:23.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:23 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:23.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:23 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:23:23.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:23 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:23:23.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:23 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:23.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:23 smithi097 ceph-mon[39416]: from='osd.1 v2:172.21.15.97:6805/287967678' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-21T13:23:23.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:23 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:23.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:23 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:23.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:23 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:23:23.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:23 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:23.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:23 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:23.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:23 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:23:23.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:23 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:23:23.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:23 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:23.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:23 smithi097 ceph-mon[30040]: from='osd.1 v2:172.21.15.97:6805/287967678' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-21T13:23:23.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:23 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:23.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:23 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:23.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:23 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:23:23.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:23 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:23.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:23 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:23.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:23 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:23:23.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:23 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:23:23.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:23 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:23.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:23 smithi144 ceph-mon[33633]: from='osd.1 v2:172.21.15.97:6805/287967678' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-21T13:23:23.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:23 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:23.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:23 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:23.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:23 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:23:23.625 DEBUG:teuthology.orchestra.run.smithi097:osd.1> sudo journalctl -f -n 0 -u ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@osd.1.service 2024-03-21T13:23:23.627 INFO:tasks.cephadm:Deploying osd.2 on smithi097 with /dev/vg_nvme/lv_2... 2024-03-21T13:23:23.627 DEBUG:teuthology.orchestra.run.smithi097:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-21T13:23:24.419 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T13:23:24.541 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:24 smithi097 ceph-mon[30040]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T13:23:24.541 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:24 smithi097 ceph-mon[30040]: from='osd.1 v2:172.21.15.97:6805/287967678' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-21T13:23:24.541 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:24 smithi097 ceph-mon[30040]: osdmap e11: 2 total, 1 up, 2 in 2024-03-21T13:23:24.542 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:24 smithi097 ceph-mon[30040]: from='osd.1 v2:172.21.15.97:6805/287967678' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-03-21T13:23:24.542 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:24 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T13:23:24.542 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:24 smithi097 ceph-mon[39416]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T13:23:24.542 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:24 smithi097 ceph-mon[39416]: from='osd.1 v2:172.21.15.97:6805/287967678' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-21T13:23:24.542 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:24 smithi097 ceph-mon[39416]: osdmap e11: 2 total, 1 up, 2 in 2024-03-21T13:23:24.542 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:24 smithi097 ceph-mon[39416]: from='osd.1 v2:172.21.15.97:6805/287967678' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-03-21T13:23:24.542 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:24 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T13:23:24.543 INFO:journalctl@ceph.osd.1.smithi097.stdout:Mar 21 13:23:24 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-1[49765]: 2024-03-21T13:23:24.242+0000 7fc976768700 -1 osd.1 0 waiting for initial osdmap 2024-03-21T13:23:24.543 INFO:journalctl@ceph.osd.1.smithi097.stdout:Mar 21 13:23:24 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-1[49765]: 2024-03-21T13:23:24.245+0000 7fc97255b700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T13:23:24.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:24 smithi144 ceph-mon[33633]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T13:23:24.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:24 smithi144 ceph-mon[33633]: from='osd.1 v2:172.21.15.97:6805/287967678' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-21T13:23:24.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:24 smithi144 ceph-mon[33633]: osdmap e11: 2 total, 1 up, 2 in 2024-03-21T13:23:24.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:24 smithi144 ceph-mon[33633]: from='osd.1 v2:172.21.15.97:6805/287967678' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-03-21T13:23:24.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:24 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T13:23:25.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:25 smithi097 ceph-mon[30040]: from='osd.1 v2:172.21.15.97:6805/287967678' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-03-21T13:23:25.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:25 smithi097 ceph-mon[30040]: osdmap e12: 2 total, 1 up, 2 in 2024-03-21T13:23:25.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:25 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T13:23:25.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:25 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T13:23:25.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:25 smithi097 ceph-mon[30040]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T13:23:25.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:25 smithi097 ceph-mon[39416]: from='osd.1 v2:172.21.15.97:6805/287967678' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-03-21T13:23:25.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:25 smithi097 ceph-mon[39416]: osdmap e12: 2 total, 1 up, 2 in 2024-03-21T13:23:25.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:25 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T13:23:25.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:25 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T13:23:25.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:25 smithi097 ceph-mon[39416]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T13:23:25.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:25 smithi144 ceph-mon[33633]: from='osd.1 v2:172.21.15.97:6805/287967678' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-03-21T13:23:25.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:25 smithi144 ceph-mon[33633]: osdmap e12: 2 total, 1 up, 2 in 2024-03-21T13:23:25.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:25 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T13:23:25.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:25 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T13:23:25.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:25 smithi144 ceph-mon[33633]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T13:23:26.518 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:26 smithi097 ceph-mon[30040]: purged_snaps scrub starts 2024-03-21T13:23:26.518 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:26 smithi097 ceph-mon[30040]: purged_snaps scrub ok 2024-03-21T13:23:26.518 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:26 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T13:23:26.518 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:26 smithi097 ceph-mon[30040]: osd.1 v2:172.21.15.97:6805/287967678 boot 2024-03-21T13:23:26.518 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:26 smithi097 ceph-mon[30040]: osdmap e13: 2 total, 2 up, 2 in 2024-03-21T13:23:26.518 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:26 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T13:23:26.519 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:26 smithi097 ceph-mon[39416]: purged_snaps scrub starts 2024-03-21T13:23:26.519 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:26 smithi097 ceph-mon[39416]: purged_snaps scrub ok 2024-03-21T13:23:26.519 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:26 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T13:23:26.519 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:26 smithi097 ceph-mon[39416]: osd.1 v2:172.21.15.97:6805/287967678 boot 2024-03-21T13:23:26.519 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:26 smithi097 ceph-mon[39416]: osdmap e13: 2 total, 2 up, 2 in 2024-03-21T13:23:26.519 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:26 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T13:23:26.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:26 smithi144 ceph-mon[33633]: purged_snaps scrub starts 2024-03-21T13:23:26.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:26 smithi144 ceph-mon[33633]: purged_snaps scrub ok 2024-03-21T13:23:26.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:26 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T13:23:26.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:26 smithi144 ceph-mon[33633]: osd.1 v2:172.21.15.97:6805/287967678 boot 2024-03-21T13:23:26.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:26 smithi144 ceph-mon[33633]: osdmap e13: 2 total, 2 up, 2 in 2024-03-21T13:23:26.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:26 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T13:23:27.096 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-21T13:23:27.096 INFO:teuthology.orchestra.run.smithi097.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-21T13:23:27.096 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10+0 records in 2024-03-21T13:23:27.096 INFO:teuthology.orchestra.run.smithi097.stderr:10+0 records out 2024-03-21T13:23:27.096 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0130843 s, 801 MB/s 2024-03-21T13:23:27.096 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping successful for: 2024-03-21T13:23:27.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:27 smithi097 ceph-mon[30040]: osdmap e14: 2 total, 2 up, 2 in 2024-03-21T13:23:27.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:27 smithi097 ceph-mon[30040]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T13:23:27.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:27 smithi097 ceph-mon[39416]: osdmap e14: 2 total, 2 up, 2 in 2024-03-21T13:23:27.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:27 smithi097 ceph-mon[39416]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T13:23:27.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:27 smithi144 ceph-mon[33633]: osdmap e14: 2 total, 2 up, 2 in 2024-03-21T13:23:27.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:27 smithi144 ceph-mon[33633]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T13:23:27.960 DEBUG:teuthology.orchestra.run.smithi097:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi097:vg_nvme/lv_2 2024-03-21T13:23:28.532 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T13:23:29.616 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:29 smithi097 ceph-mon[30040]: Detected new or changed devices on smithi097 2024-03-21T13:23:29.616 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:29 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:29.616 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:29 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:29.616 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:29 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:23:29.616 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:29 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:23:29.617 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:29 smithi097 ceph-mon[30040]: Adjusting osd_memory_target on smithi097 to 8032M 2024-03-21T13:23:29.617 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:29 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:29.617 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:29 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:23:29.617 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:29 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:23:29.617 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:29 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:29.617 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:29 smithi097 ceph-mon[30040]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T13:23:29.618 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:29 smithi097 ceph-mon[39416]: Detected new or changed devices on smithi097 2024-03-21T13:23:29.618 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:29 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:29.618 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:29 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:29.618 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:29 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:23:29.618 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:29 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:23:29.618 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:29 smithi097 ceph-mon[39416]: Adjusting osd_memory_target on smithi097 to 8032M 2024-03-21T13:23:29.619 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:29 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:29.619 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:29 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:23:29.619 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:29 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:23:29.619 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:29 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:29.619 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:29 smithi097 ceph-mon[39416]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T13:23:29.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:29 smithi144 ceph-mon[33633]: Detected new or changed devices on smithi097 2024-03-21T13:23:29.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:29 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:29.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:29 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:29.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:29 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:23:29.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:29 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:23:29.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:29 smithi144 ceph-mon[33633]: Adjusting osd_memory_target on smithi097 to 8032M 2024-03-21T13:23:29.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:29 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:29.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:29 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:23:29.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:29 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:23:29.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:29 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:29.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:29 smithi144 ceph-mon[33633]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T13:23:30.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:30.158+0000 7f59b12a6700 1 -- 172.21.15.97:0/3594783596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59ac105570 msgr2=0x7f59ac107960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:23:30.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:30.158+0000 7f59b12a6700 1 --2- 172.21.15.97:0/3594783596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59ac105570 0x7f59ac107960 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f59940098f0 tx=0x7f5994009c00 comp rx=0 tx=0).stop 2024-03-21T13:23:30.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:30.158+0000 7f59b12a6700 1 -- 172.21.15.97:0/3594783596 shutdown_connections 2024-03-21T13:23:30.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:30.158+0000 7f59b12a6700 1 --2- 172.21.15.97:0/3594783596 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f59ac107ea0 0x7f59ac10a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:23:30.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:30.158+0000 7f59b12a6700 1 --2- 172.21.15.97:0/3594783596 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59ac102c40 0x7f59ac105030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:23:30.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:30.158+0000 7f59b12a6700 1 --2- 172.21.15.97:0/3594783596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59ac105570 0x7f59ac107960 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:23:30.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:30.158+0000 7f59b12a6700 1 -- 172.21.15.97:0/3594783596 >> 172.21.15.97:0/3594783596 conn(0x7f59ac0fc650 msgr2=0x7f59ac0fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:23:30.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:30.158+0000 7f59b12a6700 1 -- 172.21.15.97:0/3594783596 shutdown_connections 2024-03-21T13:23:30.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:30.158+0000 7f59b12a6700 1 -- 172.21.15.97:0/3594783596 wait complete. 2024-03-21T13:23:30.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:30.158+0000 7f59b12a6700 1 Processor -- start 2024-03-21T13:23:30.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:30.159+0000 7f59b12a6700 1 -- start start 2024-03-21T13:23:30.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:30.159+0000 7f59b12a6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59ac102c40 0x7f59ac197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:23:30.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:30.159+0000 7f59b12a6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59ac105570 0x7f59ac197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:23:30.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:30.159+0000 7f59b12a6700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f59ac107ea0 0x7f59ac19f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:23:30.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:30.159+0000 7f59b12a6700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ac10d4d0 con 0x7f59ac102c40 2024-03-21T13:23:30.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:30.159+0000 7f59b12a6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ac10d350 con 0x7f59ac105570 2024-03-21T13:23:30.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:30.159+0000 7f59b12a6700 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59ac10d650 con 0x7f59ac107ea0 2024-03-21T13:23:30.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:30.159+0000 7f59aaffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59ac102c40 0x7f59ac197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:23:30.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:30.159+0000 7f59aaffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59ac102c40 0x7f59ac197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55704/0 (socket says 172.21.15.97:55704) 2024-03-21T13:23:30.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:30.159+0000 7f59aaffd700 1 -- 172.21.15.97:0/2169354033 learned_addr learned my addr 172.21.15.97:0/2169354033 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:23:30.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:30.159+0000 7f59ab7fe700 1 --2- 172.21.15.97:0/2169354033 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f59ac107ea0 0x7f59ac19f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:23:30.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:30.159+0000 7f59a27fc700 1 --2- 172.21.15.97:0/2169354033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59ac105570 0x7f59ac197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:23:30.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:30.159+0000 7f59aaffd700 1 -- 172.21.15.97:0/2169354033 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f59ac107ea0 msgr2=0x7f59ac19f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:23:30.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:30.159+0000 7f59aaffd700 1 --2- 172.21.15.97:0/2169354033 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f59ac107ea0 0x7f59ac19f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:23:30.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:30.159+0000 7f59aaffd700 1 -- 172.21.15.97:0/2169354033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59ac105570 msgr2=0x7f59ac197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:23:30.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:30.159+0000 7f59aaffd700 1 --2- 172.21.15.97:0/2169354033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59ac105570 0x7f59ac197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:23:30.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:30.159+0000 7f59aaffd700 1 -- 172.21.15.97:0/2169354033 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59ac19f6f0 con 0x7f59ac102c40 2024-03-21T13:23:30.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:30.159+0000 7f59ab7fe700 1 --2- 172.21.15.97:0/2169354033 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f59ac107ea0 0x7f59ac19f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T13:23:30.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:30.159+0000 7f59a27fc700 1 --2- 172.21.15.97:0/2169354033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59ac105570 0x7f59ac197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T13:23:30.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:30.160+0000 7f59aaffd700 1 --2- 172.21.15.97:0/2169354033 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59ac102c40 0x7f59ac197a90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f599c00c9e0 tx=0x7f599c00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:23:30.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:30.160+0000 7f59a8ff9700 1 -- 172.21.15.97:0/2169354033 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599c007ac0 con 0x7f59ac102c40 2024-03-21T13:23:30.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:30.160+0000 7f59a8ff9700 1 -- 172.21.15.97:0/2169354033 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f599c007c60 con 0x7f59ac102c40 2024-03-21T13:23:30.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:30.160+0000 7f59a8ff9700 1 -- 172.21.15.97:0/2169354033 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599c018660 con 0x7f59ac102c40 2024-03-21T13:23:30.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:30.160+0000 7f59b12a6700 1 -- 172.21.15.97:0/2169354033 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59ac19f9e0 con 0x7f59ac102c40 2024-03-21T13:23:30.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:30.160+0000 7f59b12a6700 1 -- 172.21.15.97:0/2169354033 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59ac19ff70 con 0x7f59ac102c40 2024-03-21T13:23:30.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:30.161+0000 7f59b12a6700 1 -- 172.21.15.97:0/2169354033 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59ac068e50 con 0x7f59ac102c40 2024-03-21T13:23:30.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:30.161+0000 7f59a8ff9700 1 -- 172.21.15.97:0/2169354033 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 14) v1 ==== 92039+0+0 (secure 0 0 0) 0x7f599c00f450 con 0x7f59ac102c40 2024-03-21T13:23:30.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:30.162+0000 7f59a8ff9700 1 --2- 172.21.15.97:0/2169354033 >> v2:172.21.15.97:6800/1943212787 conn(0x7f599806e180 0x7f5998070640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:23:30.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:30.162+0000 7f59a8ff9700 1 -- 172.21.15.97:0/2169354033 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 1833+0+0 (secure 0 0 0) 0x7f599c08cf40 con 0x7f59ac102c40 2024-03-21T13:23:30.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:30.162+0000 7f59a27fc700 1 --2- 172.21.15.97:0/2169354033 >> v2:172.21.15.97:6800/1943212787 conn(0x7f599806e180 0x7f5998070640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:23:30.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:30.162+0000 7f59a27fc700 1 --2- 172.21.15.97:0/2169354033 >> v2:172.21.15.97:6800/1943212787 conn(0x7f599806e180 0x7f5998070640 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5994009230 tx=0x7f5994009f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:23:30.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:30.166+0000 7f59a8ff9700 1 -- 172.21.15.97:0/2169354033 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f599c014030 con 0x7f59ac102c40 2024-03-21T13:23:30.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:30.370+0000 7f59b12a6700 1 -- 172.21.15.97:0/2169354033 --> v2:172.21.15.97:6800/1943212787 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f59ac02dc60 con 0x7f599806e180 2024-03-21T13:23:30.470 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:30 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T13:23:30.471 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:30 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T13:23:30.471 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:30 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:23:30.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:30 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T13:23:30.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:30 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T13:23:30.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:30 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:23:30.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:30 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T13:23:30.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:30 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T13:23:30.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:30 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:23:31.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:31 smithi097 ceph-mon[39416]: from='client.14274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:23:31.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:31 smithi097 ceph-mon[39416]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T13:23:31.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:31 smithi097 ceph-mon[30040]: from='client.14274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:23:31.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:31 smithi097 ceph-mon[30040]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T13:23:31.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:31 smithi144 ceph-mon[33633]: from='client.14274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:23:31.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:31 smithi144 ceph-mon[33633]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T13:23:33.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:33 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4211613452' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4d2f5c6-ac47-4a1d-a409-2bb884b6772f"}]: dispatch 2024-03-21T13:23:33.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:33 smithi144 ceph-mon[33633]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4d2f5c6-ac47-4a1d-a409-2bb884b6772f"}]: dispatch 2024-03-21T13:23:33.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:33 smithi144 ceph-mon[33633]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f4d2f5c6-ac47-4a1d-a409-2bb884b6772f"}]': finished 2024-03-21T13:23:33.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:33 smithi144 ceph-mon[33633]: osdmap e15: 3 total, 2 up, 3 in 2024-03-21T13:23:33.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:33 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T13:23:33.396 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:33 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4211613452' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4d2f5c6-ac47-4a1d-a409-2bb884b6772f"}]: dispatch 2024-03-21T13:23:33.396 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:33 smithi097 ceph-mon[39416]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4d2f5c6-ac47-4a1d-a409-2bb884b6772f"}]: dispatch 2024-03-21T13:23:33.396 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:33 smithi097 ceph-mon[39416]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f4d2f5c6-ac47-4a1d-a409-2bb884b6772f"}]': finished 2024-03-21T13:23:33.396 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:33 smithi097 ceph-mon[39416]: osdmap e15: 3 total, 2 up, 3 in 2024-03-21T13:23:33.396 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:33 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T13:23:33.396 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:33 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4211613452' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4d2f5c6-ac47-4a1d-a409-2bb884b6772f"}]: dispatch 2024-03-21T13:23:33.397 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:33 smithi097 ceph-mon[30040]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4d2f5c6-ac47-4a1d-a409-2bb884b6772f"}]: dispatch 2024-03-21T13:23:33.397 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:33 smithi097 ceph-mon[30040]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f4d2f5c6-ac47-4a1d-a409-2bb884b6772f"}]': finished 2024-03-21T13:23:33.397 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:33 smithi097 ceph-mon[30040]: osdmap e15: 3 total, 2 up, 3 in 2024-03-21T13:23:33.397 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:33 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T13:23:34.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:34 smithi144 ceph-mon[33633]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T13:23:34.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:34 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1364377767' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T13:23:34.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:34 smithi097 ceph-mon[30040]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T13:23:34.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:34 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1364377767' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T13:23:34.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:34 smithi097 ceph-mon[39416]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T13:23:34.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:34 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1364377767' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T13:23:36.300 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:36 smithi097 ceph-mon[30040]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T13:23:36.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:36 smithi097 ceph-mon[39416]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T13:23:36.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:36 smithi144 ceph-mon[33633]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T13:23:38.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:38 smithi144 ceph-mon[33633]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T13:23:38.401 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:38 smithi097 ceph-mon[30040]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T13:23:38.401 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:38 smithi097 ceph-mon[39416]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T13:23:40.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:40 smithi144 ceph-mon[33633]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T13:23:40.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:40 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-21T13:23:40.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:40 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:23:40.423 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:40 smithi097 ceph-mon[39416]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T13:23:40.423 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:40 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-21T13:23:40.423 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:40 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:23:40.424 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:40 smithi097 ceph-mon[30040]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T13:23:40.424 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:40 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-21T13:23:40.424 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:40 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:23:41.301 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:41 smithi097 ceph-mon[30040]: Deploying daemon osd.2 on smithi097 2024-03-21T13:23:41.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:41 smithi097 ceph-mon[39416]: Deploying daemon osd.2 on smithi097 2024-03-21T13:23:41.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:41 smithi144 ceph-mon[33633]: Deploying daemon osd.2 on smithi097 2024-03-21T13:23:42.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:42 smithi097 ceph-mon[30040]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T13:23:42.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:42 smithi097 ceph-mon[39416]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T13:23:42.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:42 smithi144 ceph-mon[33633]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T13:23:44.375 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:44 smithi097 ceph-mon[30040]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T13:23:44.376 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:44 smithi097 ceph-mon[39416]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T13:23:44.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:44 smithi144 ceph-mon[33633]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T13:23:46.274 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:46 smithi097 ceph-mon[30040]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T13:23:46.274 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:46 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:46.274 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:46 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:46.275 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:46 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:23:46.275 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:46 smithi097 ceph-mon[39416]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T13:23:46.275 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:46 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:46.275 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:46 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:46.275 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:46 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:23:46.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:46 smithi144 ceph-mon[33633]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T13:23:46.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:46 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:46.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:46 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:46.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:46 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:23:48.381 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:48 smithi097 ceph-mon[30040]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T13:23:48.382 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:48 smithi097 ceph-mon[39416]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T13:23:48.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:48 smithi144 ceph-mon[33633]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T13:23:48.724 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 2 on host 'smithi097' 2024-03-21T13:23:48.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:48.724+0000 7f59a8ff9700 1 -- 172.21.15.97:0/2169354033 <== mgr.14152 v2:172.21.15.97:6800/1943212787 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f59ac02dc60 con 0x7f599806e180 2024-03-21T13:23:48.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:48.726+0000 7f59b12a6700 1 -- 172.21.15.97:0/2169354033 >> v2:172.21.15.97:6800/1943212787 conn(0x7f599806e180 msgr2=0x7f5998070640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:23:48.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:48.726+0000 7f59b12a6700 1 --2- 172.21.15.97:0/2169354033 >> v2:172.21.15.97:6800/1943212787 conn(0x7f599806e180 0x7f5998070640 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5994009230 tx=0x7f5994009f50 comp rx=0 tx=0).stop 2024-03-21T13:23:48.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:48.726+0000 7f59b12a6700 1 -- 172.21.15.97:0/2169354033 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59ac102c40 msgr2=0x7f59ac197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:23:48.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:48.726+0000 7f59b12a6700 1 --2- 172.21.15.97:0/2169354033 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59ac102c40 0x7f59ac197a90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f599c00c9e0 tx=0x7f599c00ccf0 comp rx=0 tx=0).stop 2024-03-21T13:23:48.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:48.726+0000 7f59b12a6700 1 -- 172.21.15.97:0/2169354033 shutdown_connections 2024-03-21T13:23:48.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:48.726+0000 7f59b12a6700 1 --2- 172.21.15.97:0/2169354033 >> v2:172.21.15.97:6800/1943212787 conn(0x7f599806e180 0x7f5998070640 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:23:48.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:48.726+0000 7f59b12a6700 1 --2- 172.21.15.97:0/2169354033 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f59ac107ea0 0x7f59ac19f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:23:48.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:48.726+0000 7f59b12a6700 1 --2- 172.21.15.97:0/2169354033 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59ac102c40 0x7f59ac197a90 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:23:48.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:48.726+0000 7f59b12a6700 1 --2- 172.21.15.97:0/2169354033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59ac105570 0x7f59ac197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:23:48.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:48.726+0000 7f59b12a6700 1 -- 172.21.15.97:0/2169354033 >> 172.21.15.97:0/2169354033 conn(0x7f59ac0fc650 msgr2=0x7f59ac103580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:23:48.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:48.727+0000 7f59b12a6700 1 -- 172.21.15.97:0/2169354033 shutdown_connections 2024-03-21T13:23:48.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:48.727+0000 7f59b12a6700 1 -- 172.21.15.97:0/2169354033 wait complete. 2024-03-21T13:23:49.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:49 smithi097 ceph-mon[39416]: from='osd.2 v2:172.21.15.97:6809/2953470250' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-21T13:23:49.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:49 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:49.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:49 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:49.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:49 smithi097 ceph-mon[30040]: from='osd.2 v2:172.21.15.97:6809/2953470250' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-21T13:23:49.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:49 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:49.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:49 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:49.368 DEBUG:teuthology.orchestra.run.smithi097:osd.2> sudo journalctl -f -n 0 -u ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@osd.2.service 2024-03-21T13:23:49.372 INFO:tasks.cephadm:Deploying osd.3 on smithi097 with /dev/vg_nvme/lv_1... 2024-03-21T13:23:49.372 DEBUG:teuthology.orchestra.run.smithi097:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-21T13:23:49.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:49 smithi144 ceph-mon[33633]: from='osd.2 v2:172.21.15.97:6809/2953470250' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-21T13:23:49.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:49 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:49.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:49 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:49.659 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T13:23:50.457 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:50 smithi097 ceph-mon[30040]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T13:23:50.457 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:50 smithi097 ceph-mon[30040]: from='osd.2 v2:172.21.15.97:6809/2953470250' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-21T13:23:50.457 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:50 smithi097 ceph-mon[30040]: osdmap e16: 3 total, 2 up, 3 in 2024-03-21T13:23:50.457 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:50 smithi097 ceph-mon[30040]: from='osd.2 v2:172.21.15.97:6809/2953470250' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-03-21T13:23:50.457 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:50 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T13:23:50.457 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:50 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:50.457 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:50 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:50.458 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:50 smithi097 ceph-mon[39416]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T13:23:50.458 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:50 smithi097 ceph-mon[39416]: from='osd.2 v2:172.21.15.97:6809/2953470250' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-21T13:23:50.458 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:50 smithi097 ceph-mon[39416]: osdmap e16: 3 total, 2 up, 3 in 2024-03-21T13:23:50.458 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:50 smithi097 ceph-mon[39416]: from='osd.2 v2:172.21.15.97:6809/2953470250' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-03-21T13:23:50.458 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:50 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T13:23:50.458 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:50 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:50.458 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:50 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:50.458 INFO:journalctl@ceph.osd.2.smithi097.stdout:Mar 21 13:23:50 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-2[56207]: 2024-03-21T13:23:50.142+0000 7fa8975fc700 -1 osd.2 0 waiting for initial osdmap 2024-03-21T13:23:50.458 INFO:journalctl@ceph.osd.2.smithi097.stdout:Mar 21 13:23:50 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-2[56207]: 2024-03-21T13:23:50.146+0000 7fa8933ef700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T13:23:50.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:50 smithi144 ceph-mon[33633]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T13:23:50.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:50 smithi144 ceph-mon[33633]: from='osd.2 v2:172.21.15.97:6809/2953470250' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-21T13:23:50.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:50 smithi144 ceph-mon[33633]: osdmap e16: 3 total, 2 up, 3 in 2024-03-21T13:23:50.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:50 smithi144 ceph-mon[33633]: from='osd.2 v2:172.21.15.97:6809/2953470250' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-03-21T13:23:50.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:50 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T13:23:50.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:50 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:50.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:50 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:51.433 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:51 smithi097 ceph-mon[39416]: from='osd.2 v2:172.21.15.97:6809/2953470250' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-03-21T13:23:51.433 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:51 smithi097 ceph-mon[39416]: osdmap e17: 3 total, 2 up, 3 in 2024-03-21T13:23:51.434 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:51 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T13:23:51.434 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:51 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T13:23:51.434 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:51 smithi097 ceph-mon[30040]: from='osd.2 v2:172.21.15.97:6809/2953470250' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-03-21T13:23:51.434 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:51 smithi097 ceph-mon[30040]: osdmap e17: 3 total, 2 up, 3 in 2024-03-21T13:23:51.434 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:51 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T13:23:51.434 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:51 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T13:23:51.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:51 smithi144 ceph-mon[33633]: from='osd.2 v2:172.21.15.97:6809/2953470250' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-03-21T13:23:51.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:51 smithi144 ceph-mon[33633]: osdmap e17: 3 total, 2 up, 3 in 2024-03-21T13:23:51.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:51 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T13:23:51.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:51 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T13:23:51.677 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-21T13:23:51.677 INFO:teuthology.orchestra.run.smithi097.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-21T13:23:51.677 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10+0 records in 2024-03-21T13:23:51.677 INFO:teuthology.orchestra.run.smithi097.stderr:10+0 records out 2024-03-21T13:23:51.677 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125582 s, 835 MB/s 2024-03-21T13:23:51.677 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping successful for: 2024-03-21T13:23:52.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:52 smithi097 ceph-mon[39416]: purged_snaps scrub starts 2024-03-21T13:23:52.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:52 smithi097 ceph-mon[39416]: purged_snaps scrub ok 2024-03-21T13:23:52.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:52 smithi097 ceph-mon[39416]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T13:23:52.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:52 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T13:23:52.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:52 smithi097 ceph-mon[39416]: osd.2 v2:172.21.15.97:6809/2953470250 boot 2024-03-21T13:23:52.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:52 smithi097 ceph-mon[39416]: osdmap e18: 3 total, 3 up, 3 in 2024-03-21T13:23:52.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:52 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T13:23:52.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:52 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:23:52.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:52 smithi097 ceph-mon[30040]: purged_snaps scrub starts 2024-03-21T13:23:52.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:52 smithi097 ceph-mon[30040]: purged_snaps scrub ok 2024-03-21T13:23:52.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:52 smithi097 ceph-mon[30040]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T13:23:52.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:52 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T13:23:52.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:52 smithi097 ceph-mon[30040]: osd.2 v2:172.21.15.97:6809/2953470250 boot 2024-03-21T13:23:52.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:52 smithi097 ceph-mon[30040]: osdmap e18: 3 total, 3 up, 3 in 2024-03-21T13:23:52.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:52 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T13:23:52.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:52 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:23:52.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:52 smithi144 ceph-mon[33633]: purged_snaps scrub starts 2024-03-21T13:23:52.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:52 smithi144 ceph-mon[33633]: purged_snaps scrub ok 2024-03-21T13:23:52.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:52 smithi144 ceph-mon[33633]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T13:23:52.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:52 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T13:23:52.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:52 smithi144 ceph-mon[33633]: osd.2 v2:172.21.15.97:6809/2953470250 boot 2024-03-21T13:23:52.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:52 smithi144 ceph-mon[33633]: osdmap e18: 3 total, 3 up, 3 in 2024-03-21T13:23:52.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:52 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T13:23:52.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:52 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:23:52.695 DEBUG:teuthology.orchestra.run.smithi097:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi097:vg_nvme/lv_1 2024-03-21T13:23:53.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:53 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-21T13:23:53.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:53 smithi097 ceph-mon[30040]: osdmap e19: 3 total, 3 up, 3 in 2024-03-21T13:23:53.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:53 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:23:53.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:53 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-21T13:23:53.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:53 smithi097 ceph-mon[39416]: osdmap e19: 3 total, 3 up, 3 in 2024-03-21T13:23:53.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:53 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:23:53.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:53 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-21T13:23:53.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:53 smithi144 ceph-mon[33633]: osdmap e19: 3 total, 3 up, 3 in 2024-03-21T13:23:53.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:53 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:23:53.629 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T13:23:54.170 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 21 13:23:54 smithi097 sudo[58181]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-21T13:23:54.170 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 21 13:23:54 smithi097 sudo[58181]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T13:23:54.170 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 21 13:23:54 smithi097 sudo[58181]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T13:23:54.170 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 21 13:23:54 smithi097 sudo[58181]: pam_unix(sudo:session): session closed for user root 2024-03-21T13:23:54.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:54 smithi097 ceph-mon[39416]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-21T13:23:54.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:54 smithi097 ceph-mon[39416]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:23:54.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:54 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-21T13:23:54.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:54 smithi097 ceph-mon[39416]: osdmap e20: 3 total, 3 up, 3 in 2024-03-21T13:23:54.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:54 smithi097 ceph-mon[30040]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-21T13:23:54.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:54 smithi097 ceph-mon[30040]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:23:54.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:54 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-21T13:23:54.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:54 smithi097 ceph-mon[30040]: osdmap e20: 3 total, 3 up, 3 in 2024-03-21T13:23:54.554 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 21 13:23:54 smithi097 sudo[60143]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-21T13:23:54.554 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 21 13:23:54 smithi097 sudo[60143]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T13:23:54.554 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 21 13:23:54 smithi097 sudo[60143]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T13:23:54.554 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 21 13:23:54 smithi097 sudo[60143]: pam_unix(sudo:session): session closed for user root 2024-03-21T13:23:54.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:54 smithi144 ceph-mon[33633]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-21T13:23:54.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:54 smithi144 ceph-mon[33633]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:23:54.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:54 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-21T13:23:54.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:54 smithi144 ceph-mon[33633]: osdmap e20: 3 total, 3 up, 3 in 2024-03-21T13:23:54.860 INFO:journalctl@ceph.osd.1.smithi097.stdout:Mar 21 13:23:54 smithi097 sudo[60153]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-21T13:23:54.860 INFO:journalctl@ceph.osd.1.smithi097.stdout:Mar 21 13:23:54 smithi097 sudo[60153]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T13:23:54.860 INFO:journalctl@ceph.osd.1.smithi097.stdout:Mar 21 13:23:54 smithi097 sudo[60153]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T13:23:54.860 INFO:journalctl@ceph.osd.1.smithi097.stdout:Mar 21 13:23:54 smithi097 sudo[60153]: pam_unix(sudo:session): session closed for user root 2024-03-21T13:23:55.119 INFO:journalctl@ceph.osd.1.smithi097.stdout:Mar 21 13:23:54 smithi097 sudo[60156]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-21T13:23:55.120 INFO:journalctl@ceph.osd.1.smithi097.stdout:Mar 21 13:23:54 smithi097 sudo[60156]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T13:23:55.120 INFO:journalctl@ceph.osd.1.smithi097.stdout:Mar 21 13:23:54 smithi097 sudo[60156]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T13:23:55.120 INFO:journalctl@ceph.osd.1.smithi097.stdout:Mar 21 13:23:54 smithi097 sudo[60156]: pam_unix(sudo:session): session closed for user root 2024-03-21T13:23:55.378 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:55 smithi097 ceph-mon[30040]: osdmap e21: 3 total, 3 up, 3 in 2024-03-21T13:23:55.378 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:55 smithi097 ceph-mon[30040]: mgrmap e15: y(active, since 2m), standbys: x 2024-03-21T13:23:55.378 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:55 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:55.378 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:55 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:55.378 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:55 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:55.378 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:55 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:23:55.378 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:55 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:23:55.378 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:55 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:23:55.378 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:55 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:55.378 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:55 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:23:55.379 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:55 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:23:55.379 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:55 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:55.379 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:55 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:23:55.379 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:55 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:23:55.379 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:55 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:23:55.379 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:55 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:55.379 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:55 smithi097 ceph-mon[39416]: osdmap e21: 3 total, 3 up, 3 in 2024-03-21T13:23:55.379 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:55 smithi097 ceph-mon[39416]: mgrmap e15: y(active, since 2m), standbys: x 2024-03-21T13:23:55.379 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:55 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:55.379 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:55 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:55.380 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:55 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:55.380 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:55 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:23:55.380 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:55 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:23:55.380 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:55 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:23:55.380 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:55 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:55.380 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:55 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:23:55.380 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:55 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:23:55.380 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:55 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:55.380 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:55 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:23:55.380 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:55 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:23:55.380 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:55 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:23:55.380 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:55 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:55.381 INFO:journalctl@ceph.osd.2.smithi097.stdout:Mar 21 13:23:55 smithi097 sudo[60231]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-21T13:23:55.381 INFO:journalctl@ceph.osd.2.smithi097.stdout:Mar 21 13:23:55 smithi097 sudo[60231]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T13:23:55.381 INFO:journalctl@ceph.osd.2.smithi097.stdout:Mar 21 13:23:55 smithi097 sudo[60231]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T13:23:55.381 INFO:journalctl@ceph.osd.2.smithi097.stdout:Mar 21 13:23:55 smithi097 sudo[60231]: pam_unix(sudo:session): session closed for user root 2024-03-21T13:23:55.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:55 smithi144 ceph-mon[33633]: osdmap e21: 3 total, 3 up, 3 in 2024-03-21T13:23:55.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:55 smithi144 ceph-mon[33633]: mgrmap e15: y(active, since 2m), standbys: x 2024-03-21T13:23:55.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:55 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:55.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:55 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:55.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:55 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:55.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:55 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:23:55.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:55 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:23:55.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:55 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:23:55.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:55 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:55.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:55 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:23:55.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:55 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:23:55.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:55 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:55.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:55 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:23:55.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:55 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:23:55.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:55 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:23:55.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:55 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:55.634 INFO:journalctl@ceph.osd.2.smithi097.stdout:Mar 21 13:23:55 smithi097 sudo[60234]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-21T13:23:55.634 INFO:journalctl@ceph.osd.2.smithi097.stdout:Mar 21 13:23:55 smithi097 sudo[60234]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T13:23:55.634 INFO:journalctl@ceph.osd.2.smithi097.stdout:Mar 21 13:23:55 smithi097 sudo[60234]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T13:23:55.634 INFO:journalctl@ceph.osd.2.smithi097.stdout:Mar 21 13:23:55 smithi097 sudo[60234]: pam_unix(sudo:session): session closed for user root 2024-03-21T13:23:55.895 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:55 smithi097 sudo[60258]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-21T13:23:55.895 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:55 smithi097 sudo[60258]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T13:23:55.895 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:55 smithi097 sudo[60258]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T13:23:55.895 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:55 smithi097 sudo[60258]: pam_unix(sudo:session): session closed for user root 2024-03-21T13:23:55.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:55.922+0000 7fc814d14700 1 -- 172.21.15.97:0/1329738625 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc810102c60 msgr2=0x7fc810109510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:23:55.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:55.922+0000 7fc814d14700 1 --2- 172.21.15.97:0/1329738625 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc810102c60 0x7fc810109510 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc80800b0a0 tx=0x7fc80800b3b0 comp rx=0 tx=0).stop 2024-03-21T13:23:55.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:55.922+0000 7fc814d14700 1 -- 172.21.15.97:0/1329738625 shutdown_connections 2024-03-21T13:23:55.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:55.922+0000 7fc814d14700 1 --2- 172.21.15.97:0/1329738625 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc810102c60 0x7fc810109510 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:23:55.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:55.922+0000 7fc814d14700 1 --2- 172.21.15.97:0/1329738625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8101010c0 0x7fc8101014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:23:55.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:55.922+0000 7fc814d14700 1 --2- 172.21.15.97:0/1329738625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8101022c0 0x7fc810102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:23:55.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:55.922+0000 7fc814d14700 1 -- 172.21.15.97:0/1329738625 >> 172.21.15.97:0/1329738625 conn(0x7fc8100fc680 msgr2=0x7fc8100feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:23:55.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:55.922+0000 7fc814d14700 1 -- 172.21.15.97:0/1329738625 shutdown_connections 2024-03-21T13:23:55.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:55.922+0000 7fc814d14700 1 -- 172.21.15.97:0/1329738625 wait complete. 2024-03-21T13:23:55.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:55.922+0000 7fc814d14700 1 Processor -- start 2024-03-21T13:23:55.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:55.922+0000 7fc814d14700 1 -- start start 2024-03-21T13:23:55.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:55.923+0000 7fc814d14700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc8101010c0 0x7fc81006b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:23:55.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:55.923+0000 7fc814d14700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8101022c0 0x7fc81019c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:23:55.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:55.923+0000 7fc814d14700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc810102c60 0x7fc81019ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:23:55.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:55.923+0000 7fc814d14700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81010c3f0 con 0x7fc810102c60 2024-03-21T13:23:55.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:55.923+0000 7fc814d14700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81010c270 con 0x7fc8101022c0 2024-03-21T13:23:55.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:55.923+0000 7fc814d14700 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc81010c570 con 0x7fc8101010c0 2024-03-21T13:23:55.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:55.923+0000 7fc80e59c700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc8101010c0 0x7fc81006b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:23:55.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:55.923+0000 7fc80ed9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc810102c60 0x7fc81019ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:23:55.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:55.923+0000 7fc80e59c700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc8101010c0 0x7fc81006b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.97:39900/0 (socket says 172.21.15.97:39900) 2024-03-21T13:23:55.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:55.923+0000 7fc80ed9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc810102c60 0x7fc81019ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55608/0 (socket says 172.21.15.97:55608) 2024-03-21T13:23:55.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:55.923+0000 7fc80e59c700 1 -- 172.21.15.97:0/3435610827 learned_addr learned my addr 172.21.15.97:0/3435610827 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:23:55.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:55.923+0000 7fc80dd9b700 1 --2- 172.21.15.97:0/3435610827 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8101022c0 0x7fc81019c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:23:55.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:55.923+0000 7fc80ed9d700 1 -- 172.21.15.97:0/3435610827 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc8101010c0 msgr2=0x7fc81006b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:23:55.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:55.923+0000 7fc80ed9d700 1 --2- 172.21.15.97:0/3435610827 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc8101010c0 0x7fc81006b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:23:55.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:55.923+0000 7fc80ed9d700 1 -- 172.21.15.97:0/3435610827 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8101022c0 msgr2=0x7fc81019c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:23:55.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:55.923+0000 7fc80ed9d700 1 --2- 172.21.15.97:0/3435610827 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8101022c0 0x7fc81019c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:23:55.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:55.923+0000 7fc80ed9d700 1 -- 172.21.15.97:0/3435610827 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc81019f650 con 0x7fc810102c60 2024-03-21T13:23:55.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:55.923+0000 7fc80e59c700 1 --2- 172.21.15.97:0/3435610827 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc8101010c0 0x7fc81006b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T13:23:55.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:55.923+0000 7fc80dd9b700 1 --2- 172.21.15.97:0/3435610827 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8101022c0 0x7fc81019c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T13:23:55.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:55.923+0000 7fc80ed9d700 1 --2- 172.21.15.97:0/3435610827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc810102c60 0x7fc81019ed80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc808009bc0 tx=0x7fc808004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:23:55.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:55.923+0000 7fc7ff7fe700 1 -- 172.21.15.97:0/3435610827 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc808021070 con 0x7fc810102c60 2024-03-21T13:23:55.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:55.923+0000 7fc814d14700 1 -- 172.21.15.97:0/3435610827 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc81019f8e0 con 0x7fc810102c60 2024-03-21T13:23:55.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:55.923+0000 7fc7ff7fe700 1 -- 172.21.15.97:0/3435610827 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fc808007b80 con 0x7fc810102c60 2024-03-21T13:23:55.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:55.923+0000 7fc7ff7fe700 1 -- 172.21.15.97:0/3435610827 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc808004760 con 0x7fc810102c60 2024-03-21T13:23:55.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:55.924+0000 7fc814d14700 1 -- 172.21.15.97:0/3435610827 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc81019fd90 con 0x7fc810102c60 2024-03-21T13:23:55.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:55.924+0000 7fc814d14700 1 -- 172.21.15.97:0/3435610827 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc810068e50 con 0x7fc810102c60 2024-03-21T13:23:55.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:55.925+0000 7fc7ff7fe700 1 -- 172.21.15.97:0/3435610827 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 15) v1 ==== 92095+0+0 (secure 0 0 0) 0x7fc808003a00 con 0x7fc810102c60 2024-03-21T13:23:55.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:55.926+0000 7fc7ff7fe700 1 --2- 172.21.15.97:0/3435610827 >> v2:172.21.15.97:6800/1943212787 conn(0x7fc7f806e210 0x7fc7f80706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:23:55.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:55.926+0000 7fc80e59c700 1 --2- 172.21.15.97:0/3435610827 >> v2:172.21.15.97:6800/1943212787 conn(0x7fc7f806e210 0x7fc7f80706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:23:55.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:55.926+0000 7fc7ff7fe700 1 -- 172.21.15.97:0/3435610827 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 2511+0+0 (secure 0 0 0) 0x7fc80800f020 con 0x7fc810102c60 2024-03-21T13:23:55.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:55.926+0000 7fc80e59c700 1 --2- 172.21.15.97:0/3435610827 >> v2:172.21.15.97:6800/1943212787 conn(0x7fc7f806e210 0x7fc7f80706d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc800005950 tx=0x7fc800009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:23:55.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:55.930+0000 7fc7ff7fe700 1 -- 172.21.15.97:0/3435610827 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc80805d150 con 0x7fc810102c60 2024-03-21T13:23:56.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:23:56.133+0000 7fc814d14700 1 -- 172.21.15.97:0/3435610827 --> v2:172.21.15.97:6800/1943212787 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fc81002dda0 con 0x7fc7f806e210 2024-03-21T13:23:56.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:55 smithi097 sudo[60287]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-21T13:23:56.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:55 smithi097 sudo[60287]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T13:23:56.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:55 smithi097 sudo[60287]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T13:23:56.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:55 smithi097 sudo[60287]: pam_unix(sudo:session): session closed for user root 2024-03-21T13:23:56.393 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:56 smithi144 sudo[36451]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-21T13:23:56.393 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:56 smithi144 sudo[36451]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T13:23:56.394 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:56 smithi144 sudo[36451]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T13:23:56.394 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:56 smithi144 sudo[36451]: pam_unix(sudo:session): session closed for user root 2024-03-21T13:23:56.672 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:56 smithi097 ceph-mon[39416]: Detected new or changed devices on smithi097 2024-03-21T13:23:56.672 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:56 smithi097 ceph-mon[39416]: Adjusting osd_memory_target on smithi097 to 5354M 2024-03-21T13:23:56.672 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:56 smithi097 ceph-mon[39416]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-21T13:23:56.672 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:56 smithi097 ceph-mon[39416]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T13:23:56.672 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:56 smithi097 ceph-mon[39416]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T13:23:56.672 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:56 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T13:23:56.672 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:56 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T13:23:56.672 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:56 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T13:23:56.672 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:56 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T13:23:56.672 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:56 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T13:23:56.673 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:56 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T13:23:56.673 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:56 smithi097 ceph-mon[39416]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-21T13:23:56.673 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:56 smithi097 ceph-mon[39416]: Cluster is now healthy 2024-03-21T13:23:56.673 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:56 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T13:23:56.673 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:56 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T13:23:56.673 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:56 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:23:56.673 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:56 smithi097 sudo[60431]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-21T13:23:56.673 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:56 smithi097 ceph-mon[30040]: Detected new or changed devices on smithi097 2024-03-21T13:23:56.673 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:56 smithi097 ceph-mon[30040]: Adjusting osd_memory_target on smithi097 to 5354M 2024-03-21T13:23:56.673 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:56 smithi097 ceph-mon[30040]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-21T13:23:56.673 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:56 smithi097 ceph-mon[30040]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T13:23:56.673 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:56 smithi097 ceph-mon[30040]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T13:23:56.674 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:56 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T13:23:56.674 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:56 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T13:23:56.674 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:56 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T13:23:56.674 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:56 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T13:23:56.674 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:56 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T13:23:56.674 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:56 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T13:23:56.674 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:56 smithi097 ceph-mon[30040]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-21T13:23:56.674 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:56 smithi097 ceph-mon[30040]: Cluster is now healthy 2024-03-21T13:23:56.674 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:56 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T13:23:56.674 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:56 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T13:23:56.674 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:56 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:23:56.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:56 smithi144 sudo[36454]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-21T13:23:56.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:56 smithi144 sudo[36454]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T13:23:56.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:56 smithi144 sudo[36454]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T13:23:56.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:56 smithi144 sudo[36454]: pam_unix(sudo:session): session closed for user root 2024-03-21T13:23:56.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:56 smithi144 ceph-mon[33633]: Detected new or changed devices on smithi097 2024-03-21T13:23:56.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:56 smithi144 ceph-mon[33633]: Adjusting osd_memory_target on smithi097 to 5354M 2024-03-21T13:23:56.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:56 smithi144 ceph-mon[33633]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-21T13:23:56.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:56 smithi144 ceph-mon[33633]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T13:23:56.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:56 smithi144 ceph-mon[33633]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T13:23:56.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:56 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T13:23:56.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:56 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T13:23:56.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:56 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T13:23:56.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:56 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T13:23:56.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:56 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T13:23:56.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:56 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T13:23:56.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:56 smithi144 ceph-mon[33633]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-21T13:23:56.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:56 smithi144 ceph-mon[33633]: Cluster is now healthy 2024-03-21T13:23:56.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:56 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T13:23:56.873 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:56 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T13:23:56.873 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:56 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:23:56.930 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:56 smithi097 sudo[60431]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T13:23:56.930 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:56 smithi097 sudo[60431]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T13:23:56.931 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:56 smithi097 sudo[60431]: pam_unix(sudo:session): session closed for user root 2024-03-21T13:23:57.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:56 smithi097 sudo[60454]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-21T13:23:57.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:56 smithi097 sudo[60454]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T13:23:57.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:56 smithi097 sudo[60454]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T13:23:57.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:56 smithi097 sudo[60454]: pam_unix(sudo:session): session closed for user root 2024-03-21T13:23:57.771 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:57 smithi097 ceph-mon[39416]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T13:23:57.772 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:57 smithi097 ceph-mon[39416]: from='client.14310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:23:57.772 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:57 smithi097 ceph-mon[39416]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T13:23:57.772 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:57 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:57.772 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:57 smithi097 ceph-mon[39416]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T13:23:57.772 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:57 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T13:23:57.772 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:57 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T13:23:57.772 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:57 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T13:23:57.772 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:57 smithi097 ceph-mon[39416]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T13:23:57.772 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:57 smithi097 ceph-mon[39416]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-21T13:23:57.772 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:57 smithi097 ceph-mon[30040]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T13:23:57.772 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:57 smithi097 ceph-mon[30040]: from='client.14310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:23:57.773 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:57 smithi097 ceph-mon[30040]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T13:23:57.773 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:57 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:57.773 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:57 smithi097 ceph-mon[30040]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T13:23:57.773 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:57 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T13:23:57.773 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:57 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T13:23:57.773 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:57 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T13:23:57.773 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:57 smithi097 ceph-mon[30040]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T13:23:57.773 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:57 smithi097 ceph-mon[30040]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-21T13:23:57.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:57 smithi144 ceph-mon[33633]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T13:23:57.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:57 smithi144 ceph-mon[33633]: from='client.14310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:23:57.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:57 smithi144 ceph-mon[33633]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T13:23:57.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:57 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:23:57.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:57 smithi144 ceph-mon[33633]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T13:23:57.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:57 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T13:23:57.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:57 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T13:23:57.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:57 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T13:23:57.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:57 smithi144 ceph-mon[33633]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T13:23:57.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:57 smithi144 ceph-mon[33633]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-21T13:23:59.279 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:58 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3797248386' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3ecf8a17-0ada-41e4-9740-b9f48ee14c81"}]: dispatch 2024-03-21T13:23:59.280 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:58 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3797248386' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3ecf8a17-0ada-41e4-9740-b9f48ee14c81"}]': finished 2024-03-21T13:23:59.280 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:58 smithi097 ceph-mon[39416]: osdmap e22: 4 total, 3 up, 4 in 2024-03-21T13:23:59.280 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:58 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T13:23:59.280 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:58 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3797248386' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3ecf8a17-0ada-41e4-9740-b9f48ee14c81"}]: dispatch 2024-03-21T13:23:59.280 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:58 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3797248386' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3ecf8a17-0ada-41e4-9740-b9f48ee14c81"}]': finished 2024-03-21T13:23:59.280 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:58 smithi097 ceph-mon[30040]: osdmap e22: 4 total, 3 up, 4 in 2024-03-21T13:23:59.280 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:58 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T13:23:59.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:58 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3797248386' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3ecf8a17-0ada-41e4-9740-b9f48ee14c81"}]: dispatch 2024-03-21T13:23:59.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:58 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3797248386' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3ecf8a17-0ada-41e4-9740-b9f48ee14c81"}]': finished 2024-03-21T13:23:59.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:58 smithi144 ceph-mon[33633]: osdmap e22: 4 total, 3 up, 4 in 2024-03-21T13:23:59.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:58 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T13:24:00.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:59 smithi097 ceph-mon[30040]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-21T13:24:00.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:23:59 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1093488030' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T13:24:00.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:59 smithi097 ceph-mon[39416]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-21T13:24:00.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:23:59 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1093488030' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T13:24:00.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:59 smithi144 ceph-mon[33633]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-21T13:24:00.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:23:59 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1093488030' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T13:24:02.505 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:02 smithi097 ceph-mon[30040]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-21T13:24:02.505 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:02 smithi097 ceph-mon[39416]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-21T13:24:02.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:02 smithi144 ceph-mon[33633]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-21T13:24:04.457 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:04 smithi097 ceph-mon[30040]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-03-21T13:24:04.458 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:04 smithi097 ceph-mon[39416]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-03-21T13:24:04.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:04 smithi144 ceph-mon[33633]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-03-21T13:24:06.488 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:06 smithi097 ceph-mon[30040]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-03-21T13:24:06.489 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:06 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-21T13:24:06.489 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:06 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:24:06.489 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:06 smithi097 ceph-mon[39416]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-03-21T13:24:06.489 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:06 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-21T13:24:06.489 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:06 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:24:06.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:06 smithi144 ceph-mon[33633]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-03-21T13:24:06.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:06 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-21T13:24:06.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:06 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:24:07.225 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:07 smithi097 ceph-mon[30040]: Deploying daemon osd.3 on smithi097 2024-03-21T13:24:07.226 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:07 smithi097 ceph-mon[39416]: Deploying daemon osd.3 on smithi097 2024-03-21T13:24:07.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:07 smithi144 ceph-mon[33633]: Deploying daemon osd.3 on smithi097 2024-03-21T13:24:08.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:08 smithi097 ceph-mon[30040]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-03-21T13:24:08.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:08 smithi097 ceph-mon[39416]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-03-21T13:24:08.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:08 smithi144 ceph-mon[33633]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-03-21T13:24:10.423 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:10 smithi144 ceph-mon[33633]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 77 KiB/s wr, 5 op/s 2024-03-21T13:24:10.440 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:10 smithi097 ceph-mon[30040]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 77 KiB/s wr, 5 op/s 2024-03-21T13:24:10.440 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:10 smithi097 ceph-mon[39416]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 77 KiB/s wr, 5 op/s 2024-03-21T13:24:12.272 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:12 smithi097 ceph-mon[30040]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-03-21T13:24:12.273 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:12 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:12.273 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:12 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:12.273 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:12 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:24:12.273 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:12 smithi097 ceph-mon[39416]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-03-21T13:24:12.273 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:12 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:12.273 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:12 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:12.273 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:12 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:24:12.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:12 smithi144 ceph-mon[33633]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-03-21T13:24:12.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:12 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:12.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:12 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:12.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:12 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:24:14.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:14 smithi097 ceph-mon[30040]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-03-21T13:24:14.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:14 smithi097 ceph-mon[39416]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-03-21T13:24:14.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:14 smithi144 ceph-mon[33633]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-03-21T13:24:15.035 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 3 on host 'smithi097' 2024-03-21T13:24:15.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:24:15.035+0000 7fc7ff7fe700 1 -- 172.21.15.97:0/3435610827 <== mgr.14152 v2:172.21.15.97:6800/1943212787 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc81002dda0 con 0x7fc7f806e210 2024-03-21T13:24:15.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:24:15.037+0000 7fc814d14700 1 -- 172.21.15.97:0/3435610827 >> v2:172.21.15.97:6800/1943212787 conn(0x7fc7f806e210 msgr2=0x7fc7f80706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:24:15.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:24:15.037+0000 7fc814d14700 1 --2- 172.21.15.97:0/3435610827 >> v2:172.21.15.97:6800/1943212787 conn(0x7fc7f806e210 0x7fc7f80706d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc800005950 tx=0x7fc800009260 comp rx=0 tx=0).stop 2024-03-21T13:24:15.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:24:15.037+0000 7fc814d14700 1 -- 172.21.15.97:0/3435610827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc810102c60 msgr2=0x7fc81019ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:24:15.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:24:15.037+0000 7fc814d14700 1 --2- 172.21.15.97:0/3435610827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc810102c60 0x7fc81019ed80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc808009bc0 tx=0x7fc808004ce0 comp rx=0 tx=0).stop 2024-03-21T13:24:15.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:24:15.037+0000 7fc814d14700 1 -- 172.21.15.97:0/3435610827 shutdown_connections 2024-03-21T13:24:15.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:24:15.037+0000 7fc814d14700 1 --2- 172.21.15.97:0/3435610827 >> v2:172.21.15.97:6800/1943212787 conn(0x7fc7f806e210 0x7fc7f80706d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:24:15.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:24:15.037+0000 7fc814d14700 1 --2- 172.21.15.97:0/3435610827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc810102c60 0x7fc81019ed80 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:24:15.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:24:15.037+0000 7fc814d14700 1 --2- 172.21.15.97:0/3435610827 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc8101010c0 0x7fc81006b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:24:15.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:24:15.037+0000 7fc814d14700 1 --2- 172.21.15.97:0/3435610827 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8101022c0 0x7fc81019c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:24:15.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:24:15.037+0000 7fc814d14700 1 -- 172.21.15.97:0/3435610827 >> 172.21.15.97:0/3435610827 conn(0x7fc8100fc680 msgr2=0x7fc8101055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:24:15.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:24:15.037+0000 7fc814d14700 1 -- 172.21.15.97:0/3435610827 shutdown_connections 2024-03-21T13:24:15.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:24:15.037+0000 7fc814d14700 1 -- 172.21.15.97:0/3435610827 wait complete. 2024-03-21T13:24:15.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:15 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:15.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:15 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:15.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:15 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:24:15.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:15 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:24:15.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:15 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:15.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:15 smithi144 ceph-mon[33633]: from='osd.3 v2:172.21.15.97:6813/1678932102' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-21T13:24:15.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:15 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:15.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:15 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:15.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:15 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:24:15.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:15 smithi144 ceph-mon[33633]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-21T13:24:15.715 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:15 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:15.715 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:15 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:15.715 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:15 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:24:15.715 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:15 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:24:15.715 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:15 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:15.715 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:15 smithi097 ceph-mon[30040]: from='osd.3 v2:172.21.15.97:6813/1678932102' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-21T13:24:15.715 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:15 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:15.715 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:15 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:15.715 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:15 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:24:15.715 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:15 smithi097 ceph-mon[30040]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-21T13:24:15.716 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:15 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:15.716 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:15 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:15.716 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:15 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:24:15.716 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:15 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:24:15.716 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:15 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:15.716 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:15 smithi097 ceph-mon[39416]: from='osd.3 v2:172.21.15.97:6813/1678932102' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-21T13:24:15.716 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:15 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:15.716 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:15 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:15.716 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:15 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:24:15.716 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:15 smithi097 ceph-mon[39416]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-21T13:24:15.742 DEBUG:teuthology.orchestra.run.smithi097:osd.3> sudo journalctl -f -n 0 -u ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@osd.3.service 2024-03-21T13:24:15.744 INFO:tasks.cephadm:Deploying osd.4 on smithi144 with /dev/vg_nvme/lv_4... 2024-03-21T13:24:15.745 DEBUG:teuthology.orchestra.run.smithi144:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-21T13:24:15.991 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T13:24:16.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:16 smithi144 ceph-mon[33633]: from='osd.3 v2:172.21.15.97:6813/1678932102' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-21T13:24:16.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:16 smithi144 ceph-mon[33633]: osdmap e23: 4 total, 3 up, 4 in 2024-03-21T13:24:16.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:16 smithi144 ceph-mon[33633]: from='osd.3 v2:172.21.15.97:6813/1678932102' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-03-21T13:24:16.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:16 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T13:24:16.771 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:16 smithi097 ceph-mon[30040]: from='osd.3 v2:172.21.15.97:6813/1678932102' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-21T13:24:16.771 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:16 smithi097 ceph-mon[30040]: osdmap e23: 4 total, 3 up, 4 in 2024-03-21T13:24:16.771 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:16 smithi097 ceph-mon[30040]: from='osd.3 v2:172.21.15.97:6813/1678932102' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-03-21T13:24:16.771 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:16 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T13:24:16.771 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:16 smithi097 ceph-mon[39416]: from='osd.3 v2:172.21.15.97:6813/1678932102' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-21T13:24:16.771 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:16 smithi097 ceph-mon[39416]: osdmap e23: 4 total, 3 up, 4 in 2024-03-21T13:24:16.771 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:16 smithi097 ceph-mon[39416]: from='osd.3 v2:172.21.15.97:6813/1678932102' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-03-21T13:24:16.771 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:16 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T13:24:17.047 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 21 13:24:16 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-3[62397]: 2024-03-21T13:24:16.813+0000 7f7dec793700 -1 osd.3 0 waiting for initial osdmap 2024-03-21T13:24:17.047 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 21 13:24:16 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-3[62397]: 2024-03-21T13:24:16.820+0000 7f7de457e700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T13:24:17.460 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:17 smithi144 ceph-mon[33633]: from='osd.3 v2:172.21.15.97:6813/1678932102' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-03-21T13:24:17.460 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:17 smithi144 ceph-mon[33633]: osdmap e24: 4 total, 3 up, 4 in 2024-03-21T13:24:17.460 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:17 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T13:24:17.461 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:17 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T13:24:17.461 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:17 smithi144 ceph-mon[33633]: from='osd.3 v2:172.21.15.97:6813/1678932102' entity='osd.3' 2024-03-21T13:24:17.461 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:17 smithi144 ceph-mon[33633]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-21T13:24:17.461 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:17 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T13:24:17.565 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:17 smithi097 ceph-mon[39416]: from='osd.3 v2:172.21.15.97:6813/1678932102' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-03-21T13:24:17.565 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:17 smithi097 ceph-mon[39416]: osdmap e24: 4 total, 3 up, 4 in 2024-03-21T13:24:17.565 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:17 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T13:24:17.565 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:17 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T13:24:17.566 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:17 smithi097 ceph-mon[39416]: from='osd.3 v2:172.21.15.97:6813/1678932102' entity='osd.3' 2024-03-21T13:24:17.566 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:17 smithi097 ceph-mon[39416]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-21T13:24:17.566 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:17 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T13:24:17.566 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:17 smithi097 ceph-mon[30040]: from='osd.3 v2:172.21.15.97:6813/1678932102' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-03-21T13:24:17.566 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:17 smithi097 ceph-mon[30040]: osdmap e24: 4 total, 3 up, 4 in 2024-03-21T13:24:17.566 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:17 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T13:24:17.566 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:17 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T13:24:17.566 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:17 smithi097 ceph-mon[30040]: from='osd.3 v2:172.21.15.97:6813/1678932102' entity='osd.3' 2024-03-21T13:24:17.566 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:17 smithi097 ceph-mon[30040]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-21T13:24:17.566 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:17 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T13:24:17.995 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-21T13:24:17.996 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-21T13:24:17.996 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10+0 records in 2024-03-21T13:24:17.996 INFO:teuthology.orchestra.run.smithi144.stderr:10+0 records out 2024-03-21T13:24:17.996 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0128867 s, 814 MB/s 2024-03-21T13:24:17.996 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping successful for: 2024-03-21T13:24:18.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:18 smithi097 ceph-mon[30040]: purged_snaps scrub starts 2024-03-21T13:24:18.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:18 smithi097 ceph-mon[30040]: purged_snaps scrub ok 2024-03-21T13:24:18.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:18 smithi097 ceph-mon[30040]: osd.3 v2:172.21.15.97:6813/1678932102 boot 2024-03-21T13:24:18.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:18 smithi097 ceph-mon[30040]: osdmap e25: 4 total, 4 up, 4 in 2024-03-21T13:24:18.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:18 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T13:24:18.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:18 smithi097 ceph-mon[39416]: purged_snaps scrub starts 2024-03-21T13:24:18.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:18 smithi097 ceph-mon[39416]: purged_snaps scrub ok 2024-03-21T13:24:18.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:18 smithi097 ceph-mon[39416]: osd.3 v2:172.21.15.97:6813/1678932102 boot 2024-03-21T13:24:18.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:18 smithi097 ceph-mon[39416]: osdmap e25: 4 total, 4 up, 4 in 2024-03-21T13:24:18.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:18 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T13:24:18.563 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:18 smithi144 ceph-mon[33633]: purged_snaps scrub starts 2024-03-21T13:24:18.563 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:18 smithi144 ceph-mon[33633]: purged_snaps scrub ok 2024-03-21T13:24:18.563 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:18 smithi144 ceph-mon[33633]: osd.3 v2:172.21.15.97:6813/1678932102 boot 2024-03-21T13:24:18.563 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:18 smithi144 ceph-mon[33633]: osdmap e25: 4 total, 4 up, 4 in 2024-03-21T13:24:18.563 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:18 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T13:24:18.591 DEBUG:teuthology.orchestra.run.smithi144:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi144:vg_nvme/lv_4 2024-03-21T13:24:18.829 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T13:24:20.198 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:19 smithi144 ceph-mon[33633]: Detected new or changed devices on smithi097 2024-03-21T13:24:20.199 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:19 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:20.199 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:19 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:20.199 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:19 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:24:20.199 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:19 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:24:20.199 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:19 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:24:20.199 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:19 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:24:20.199 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:19 smithi144 ceph-mon[33633]: Adjusting osd_memory_target on smithi097 to 4016M 2024-03-21T13:24:20.199 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:19 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:20.199 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:19 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:24:20.199 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:19 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:24:20.199 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:19 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:20.199 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:19 smithi144 ceph-mon[33633]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T13:24:20.199 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:19 smithi144 ceph-mon[33633]: osdmap e26: 4 total, 4 up, 4 in 2024-03-21T13:24:20.223 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:19 smithi097 ceph-mon[30040]: Detected new or changed devices on smithi097 2024-03-21T13:24:20.223 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:19 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:20.223 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:19 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:20.223 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:19 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:24:20.223 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:19 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:24:20.223 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:19 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:24:20.223 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:19 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:24:20.223 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:19 smithi097 ceph-mon[30040]: Adjusting osd_memory_target on smithi097 to 4016M 2024-03-21T13:24:20.223 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:19 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:20.224 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:19 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:24:20.224 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:19 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:24:20.224 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:19 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:20.224 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:19 smithi097 ceph-mon[30040]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T13:24:20.224 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:19 smithi097 ceph-mon[30040]: osdmap e26: 4 total, 4 up, 4 in 2024-03-21T13:24:20.224 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:19 smithi097 ceph-mon[39416]: Detected new or changed devices on smithi097 2024-03-21T13:24:20.224 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:19 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:20.224 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:19 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:20.224 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:19 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:24:20.224 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:19 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:24:20.224 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:19 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:24:20.224 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:19 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:24:20.225 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:19 smithi097 ceph-mon[39416]: Adjusting osd_memory_target on smithi097 to 4016M 2024-03-21T13:24:20.225 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:19 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:20.225 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:19 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:24:20.225 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:19 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:24:20.225 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:19 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:20.225 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:19 smithi097 ceph-mon[39416]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T13:24:20.225 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:19 smithi097 ceph-mon[39416]: osdmap e26: 4 total, 4 up, 4 in 2024-03-21T13:24:20.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:20.440+0000 7fea085db700 1 -- 172.21.15.144:0/1464744689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea001033c0 msgr2=0x7fea00103820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:24:20.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:20.440+0000 7fea085db700 1 --2- 172.21.15.144:0/1464744689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea001033c0 0x7fea00103820 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc0099c0 tx=0x7fe9fc009cd0 comp rx=0 tx=0).stop 2024-03-21T13:24:20.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:20.440+0000 7fea085db700 1 -- 172.21.15.144:0/1464744689 shutdown_connections 2024-03-21T13:24:20.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:20.440+0000 7fea085db700 1 --2- 172.21.15.144:0/1464744689 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fea00103d60 0x7fea0010a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:24:20.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:20.440+0000 7fea085db700 1 --2- 172.21.15.144:0/1464744689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea001021c0 0x7fea001025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:24:20.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:20.440+0000 7fea085db700 1 --2- 172.21.15.144:0/1464744689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea001033c0 0x7fea00103820 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:24:20.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:20.440+0000 7fea085db700 1 -- 172.21.15.144:0/1464744689 >> 172.21.15.144:0/1464744689 conn(0x7fea000fd7a0 msgr2=0x7fea000ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:24:20.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:20.440+0000 7fea085db700 1 -- 172.21.15.144:0/1464744689 shutdown_connections 2024-03-21T13:24:20.444 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:20.440+0000 7fea085db700 1 -- 172.21.15.144:0/1464744689 wait complete. 2024-03-21T13:24:20.444 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:20.441+0000 7fea085db700 1 Processor -- start 2024-03-21T13:24:20.444 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:20.441+0000 7fea085db700 1 -- start start 2024-03-21T13:24:20.444 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:20.441+0000 7fea085db700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea001021c0 0x7fea00197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:24:20.444 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:20.441+0000 7fea085db700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea001033c0 0x7fea00198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:24:20.444 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:20.441+0000 7fea085db700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fea00103d60 0x7fea0019f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:24:20.444 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:20.441+0000 7fea085db700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea0010d530 con 0x7fea001033c0 2024-03-21T13:24:20.444 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:20.441+0000 7fea085db700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea0010d3b0 con 0x7fea001021c0 2024-03-21T13:24:20.444 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:20.441+0000 7fea085db700 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea0010d6b0 con 0x7fea00103d60 2024-03-21T13:24:20.444 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:20.441+0000 7fea06377700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea001021c0 0x7fea00197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:24:20.444 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:20.441+0000 7fea06377700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea001021c0 0x7fea00197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:48294/0 (socket says 172.21.15.144:48294) 2024-03-21T13:24:20.444 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:20.441+0000 7fea06377700 1 -- 172.21.15.144:0/234624451 learned_addr learned my addr 172.21.15.144:0/234624451 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-21T13:24:20.444 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:20.441+0000 7fea05b76700 1 --2- 172.21.15.144:0/234624451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea001033c0 0x7fea00198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:24:20.444 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:20.441+0000 7fea06b78700 1 --2- 172.21.15.144:0/234624451 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fea00103d60 0x7fea0019f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:24:20.445 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:20.442+0000 7fea06377700 1 -- 172.21.15.144:0/234624451 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fea00103d60 msgr2=0x7fea0019f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:24:20.445 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:20.442+0000 7fea06377700 1 --2- 172.21.15.144:0/234624451 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fea00103d60 0x7fea0019f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:24:20.445 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:20.442+0000 7fea06377700 1 -- 172.21.15.144:0/234624451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea001033c0 msgr2=0x7fea00198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:24:20.445 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:20.442+0000 7fea06377700 1 --2- 172.21.15.144:0/234624451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea001033c0 0x7fea00198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:24:20.445 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:20.442+0000 7fea06377700 1 -- 172.21.15.144:0/234624451 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea0019f7a0 con 0x7fea001021c0 2024-03-21T13:24:20.445 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:20.442+0000 7fea06377700 1 --2- 172.21.15.144:0/234624451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea001021c0 0x7fea00197b40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe9f000d980 tx=0x7fe9f000dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:24:20.445 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:20.442+0000 7fe9f77fe700 1 -- 172.21.15.144:0/234624451 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f0017030 con 0x7fea001021c0 2024-03-21T13:24:20.445 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:20.442+0000 7fe9f77fe700 1 -- 172.21.15.144:0/234624451 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fe9f00044d0 con 0x7fea001021c0 2024-03-21T13:24:20.445 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:20.442+0000 7fea085db700 1 -- 172.21.15.144:0/234624451 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea0019fa90 con 0x7fea001021c0 2024-03-21T13:24:20.445 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:20.442+0000 7fe9f77fe700 1 -- 172.21.15.144:0/234624451 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f0003980 con 0x7fea001021c0 2024-03-21T13:24:20.445 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:20.442+0000 7fea085db700 1 -- 172.21.15.144:0/234624451 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea0019ffd0 con 0x7fea001021c0 2024-03-21T13:24:20.445 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:20.443+0000 7fea085db700 1 -- 172.21.15.144:0/234624451 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea00068e50 con 0x7fea001021c0 2024-03-21T13:24:20.446 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:20.443+0000 7fe9f77fe700 1 -- 172.21.15.144:0/234624451 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 15) v1 ==== 92095+0+0 (secure 0 0 0) 0x7fe9f001a070 con 0x7fea001021c0 2024-03-21T13:24:20.446 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:20.444+0000 7fe9f77fe700 1 --2- 172.21.15.144:0/234624451 >> v2:172.21.15.97:6800/1943212787 conn(0x7fe9ec06e210 0x7fe9ec0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:24:20.446 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:20.444+0000 7fe9f77fe700 1 -- 172.21.15.144:0/234624451 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 2803+0+0 (secure 0 0 0) 0x7fe9f008cda0 con 0x7fea001021c0 2024-03-21T13:24:20.446 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:20.444+0000 7fea05b76700 1 --2- 172.21.15.144:0/234624451 >> v2:172.21.15.97:6800/1943212787 conn(0x7fe9ec06e210 0x7fe9ec0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:24:20.447 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:20.445+0000 7fea05b76700 1 --2- 172.21.15.144:0/234624451 >> v2:172.21.15.97:6800/1943212787 conn(0x7fe9ec06e210 0x7fe9ec0706d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc000c00 tx=0x7fe9fc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:24:20.449 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:20.449+0000 7fe9f77fe700 1 -- 172.21.15.144:0/234624451 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe9f005b270 con 0x7fea001021c0 2024-03-21T13:24:20.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:20.652+0000 7fea085db700 1 -- 172.21.15.144:0/234624451 --> v2:172.21.15.97:6800/1943212787 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fea000625d0 con 0x7fe9ec06e210 2024-03-21T13:24:21.248 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:20 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T13:24:21.248 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:20 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T13:24:21.248 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:20 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:24:21.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:20 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T13:24:21.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:20 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T13:24:21.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:20 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:24:21.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:20 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T13:24:21.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:20 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T13:24:21.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:20 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:24:22.197 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:21 smithi144 ceph-mon[33633]: from='client.24196 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:24:22.197 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:21 smithi144 ceph-mon[33633]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T13:24:22.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:21 smithi097 ceph-mon[39416]: from='client.24196 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:24:22.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:21 smithi097 ceph-mon[39416]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T13:24:22.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:21 smithi097 ceph-mon[30040]: from='client.24196 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:24:22.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:21 smithi097 ceph-mon[30040]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T13:24:24.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:24 smithi097 ceph-mon[30040]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T13:24:24.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:24 smithi097 ceph-mon[30040]: from='client.? 172.21.15.144:0/4156821052' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52ca17a1-0407-4fb1-994b-a382d3c46879"}]: dispatch 2024-03-21T13:24:24.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:24 smithi097 ceph-mon[30040]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52ca17a1-0407-4fb1-994b-a382d3c46879"}]: dispatch 2024-03-21T13:24:24.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:24 smithi097 ceph-mon[30040]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "52ca17a1-0407-4fb1-994b-a382d3c46879"}]': finished 2024-03-21T13:24:24.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:24 smithi097 ceph-mon[30040]: osdmap e27: 5 total, 4 up, 5 in 2024-03-21T13:24:24.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:24 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T13:24:24.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:24 smithi097 ceph-mon[30040]: from='client.? 172.21.15.144:0/1147867049' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T13:24:24.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:24 smithi097 ceph-mon[39416]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T13:24:24.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:24 smithi097 ceph-mon[39416]: from='client.? 172.21.15.144:0/4156821052' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52ca17a1-0407-4fb1-994b-a382d3c46879"}]: dispatch 2024-03-21T13:24:24.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:24 smithi097 ceph-mon[39416]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52ca17a1-0407-4fb1-994b-a382d3c46879"}]: dispatch 2024-03-21T13:24:24.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:24 smithi097 ceph-mon[39416]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "52ca17a1-0407-4fb1-994b-a382d3c46879"}]': finished 2024-03-21T13:24:24.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:24 smithi097 ceph-mon[39416]: osdmap e27: 5 total, 4 up, 5 in 2024-03-21T13:24:24.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:24 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T13:24:24.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:24 smithi097 ceph-mon[39416]: from='client.? 172.21.15.144:0/1147867049' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T13:24:24.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:24 smithi144 ceph-mon[33633]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T13:24:24.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:24 smithi144 ceph-mon[33633]: from='client.? 172.21.15.144:0/4156821052' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52ca17a1-0407-4fb1-994b-a382d3c46879"}]: dispatch 2024-03-21T13:24:24.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:24 smithi144 ceph-mon[33633]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52ca17a1-0407-4fb1-994b-a382d3c46879"}]: dispatch 2024-03-21T13:24:24.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:24 smithi144 ceph-mon[33633]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "52ca17a1-0407-4fb1-994b-a382d3c46879"}]': finished 2024-03-21T13:24:24.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:24 smithi144 ceph-mon[33633]: osdmap e27: 5 total, 4 up, 5 in 2024-03-21T13:24:24.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:24 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T13:24:24.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:24 smithi144 ceph-mon[33633]: from='client.? 172.21.15.144:0/1147867049' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T13:24:26.535 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:26 smithi144 ceph-mon[33633]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T13:24:26.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:26 smithi097 ceph-mon[30040]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T13:24:26.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:26 smithi097 ceph-mon[39416]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T13:24:28.300 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:28 smithi144 ceph-mon[33633]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T13:24:28.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:28 smithi097 ceph-mon[30040]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T13:24:28.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:28 smithi097 ceph-mon[39416]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T13:24:30.518 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:30 smithi144 ceph-mon[33633]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T13:24:30.518 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:30 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-21T13:24:30.518 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:30 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:24:30.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:30 smithi097 ceph-mon[30040]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T13:24:30.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:30 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-21T13:24:30.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:30 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:24:30.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:30 smithi097 ceph-mon[39416]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T13:24:30.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:30 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-21T13:24:30.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:30 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:24:31.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:31 smithi144 ceph-mon[33633]: Deploying daemon osd.4 on smithi144 2024-03-21T13:24:31.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:31 smithi097 ceph-mon[30040]: Deploying daemon osd.4 on smithi144 2024-03-21T13:24:31.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:31 smithi097 ceph-mon[39416]: Deploying daemon osd.4 on smithi144 2024-03-21T13:24:32.435 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:32 smithi144 ceph-mon[33633]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T13:24:32.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:32 smithi097 ceph-mon[30040]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T13:24:32.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:32 smithi097 ceph-mon[39416]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T13:24:34.434 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:34 smithi144 ceph-mon[33633]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T13:24:34.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:34 smithi097 ceph-mon[30040]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T13:24:34.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:34 smithi097 ceph-mon[39416]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T13:24:36.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:36 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:36.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:36 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:36.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:36 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:24:36.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:36 smithi097 ceph-mon[30040]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T13:24:36.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:36 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:36.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:36 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:36.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:36 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:24:36.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:36 smithi097 ceph-mon[39416]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T13:24:36.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:36 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:36.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:36 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:36.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:36 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:24:36.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:36 smithi144 ceph-mon[33633]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T13:24:38.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:37 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:38.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:37 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:38.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:37 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:24:38.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:37 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:24:38.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:37 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:38.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:37 smithi097 ceph-mon[30040]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T13:24:38.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:37 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:38.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:37 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:38.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:37 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:24:38.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:37 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:24:38.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:37 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:38.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:37 smithi097 ceph-mon[39416]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T13:24:38.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:37 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:38.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:37 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:38.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:37 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:24:38.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:37 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:24:38.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:37 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:38.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:37 smithi144 ceph-mon[33633]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T13:24:38.712 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:38.712+0000 7fe9f77fe700 1 -- 172.21.15.144:0/234624451 <== mgr.14152 v2:172.21.15.97:6800/1943212787 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fea000625d0 con 0x7fe9ec06e210 2024-03-21T13:24:38.713 INFO:teuthology.orchestra.run.smithi144.stdout:Created osd(s) 4 on host 'smithi144' 2024-03-21T13:24:38.715 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:38.715+0000 7fea085db700 1 -- 172.21.15.144:0/234624451 >> v2:172.21.15.97:6800/1943212787 conn(0x7fe9ec06e210 msgr2=0x7fe9ec0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:24:38.715 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:38.715+0000 7fea085db700 1 --2- 172.21.15.144:0/234624451 >> v2:172.21.15.97:6800/1943212787 conn(0x7fe9ec06e210 0x7fe9ec0706d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc000c00 tx=0x7fe9fc005fb0 comp rx=0 tx=0).stop 2024-03-21T13:24:38.715 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:38.715+0000 7fea085db700 1 -- 172.21.15.144:0/234624451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea001021c0 msgr2=0x7fea00197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:24:38.715 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:38.715+0000 7fea085db700 1 --2- 172.21.15.144:0/234624451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea001021c0 0x7fea00197b40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe9f000d980 tx=0x7fe9f000dc90 comp rx=0 tx=0).stop 2024-03-21T13:24:38.715 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:38.715+0000 7fea085db700 1 -- 172.21.15.144:0/234624451 shutdown_connections 2024-03-21T13:24:38.715 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:38.715+0000 7fea085db700 1 --2- 172.21.15.144:0/234624451 >> v2:172.21.15.97:6800/1943212787 conn(0x7fe9ec06e210 0x7fe9ec0706d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:24:38.715 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:38.715+0000 7fea085db700 1 --2- 172.21.15.144:0/234624451 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fea00103d60 0x7fea0019f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:24:38.715 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:38.715+0000 7fea085db700 1 --2- 172.21.15.144:0/234624451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea001021c0 0x7fea00197b40 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:24:38.716 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:38.715+0000 7fea085db700 1 --2- 172.21.15.144:0/234624451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea001033c0 0x7fea00198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:24:38.716 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:38.715+0000 7fea085db700 1 -- 172.21.15.144:0/234624451 >> 172.21.15.144:0/234624451 conn(0x7fea000fd7a0 msgr2=0x7fea00106740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:24:38.716 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:38.715+0000 7fea085db700 1 -- 172.21.15.144:0/234624451 shutdown_connections 2024-03-21T13:24:38.716 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:38.715+0000 7fea085db700 1 -- 172.21.15.144:0/234624451 wait complete. 2024-03-21T13:24:39.247 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:38 smithi144 ceph-mon[33633]: from='osd.4 v2:172.21.15.144:6800/913882960' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-21T13:24:39.247 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:38 smithi144 ceph-mon[33633]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-21T13:24:39.247 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:38 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:39.247 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:38 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:39.248 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:38 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:24:39.280 DEBUG:teuthology.orchestra.run.smithi144:osd.4> sudo journalctl -f -n 0 -u ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@osd.4.service 2024-03-21T13:24:39.283 INFO:tasks.cephadm:Deploying osd.5 on smithi144 with /dev/vg_nvme/lv_3... 2024-03-21T13:24:39.283 DEBUG:teuthology.orchestra.run.smithi144:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-21T13:24:39.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:38 smithi097 ceph-mon[30040]: from='osd.4 v2:172.21.15.144:6800/913882960' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-21T13:24:39.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:38 smithi097 ceph-mon[30040]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-21T13:24:39.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:38 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:39.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:38 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:39.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:38 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:24:39.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:38 smithi097 ceph-mon[39416]: from='osd.4 v2:172.21.15.144:6800/913882960' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-21T13:24:39.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:38 smithi097 ceph-mon[39416]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-21T13:24:39.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:38 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:39.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:38 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:39.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:38 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:24:39.957 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T13:24:40.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:39 smithi097 ceph-mon[39416]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-21T13:24:40.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:39 smithi097 ceph-mon[39416]: osdmap e28: 5 total, 4 up, 5 in 2024-03-21T13:24:40.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:39 smithi097 ceph-mon[39416]: from='osd.4 v2:172.21.15.144:6800/913882960' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-21T13:24:40.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:39 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T13:24:40.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:39 smithi097 ceph-mon[39416]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-21T13:24:40.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:39 smithi097 ceph-mon[39416]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T13:24:40.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:39 smithi097 ceph-mon[30040]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-21T13:24:40.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:39 smithi097 ceph-mon[30040]: osdmap e28: 5 total, 4 up, 5 in 2024-03-21T13:24:40.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:39 smithi097 ceph-mon[30040]: from='osd.4 v2:172.21.15.144:6800/913882960' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-21T13:24:40.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:39 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T13:24:40.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:39 smithi097 ceph-mon[30040]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-21T13:24:40.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:39 smithi097 ceph-mon[30040]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T13:24:40.321 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:39 smithi144 ceph-mon[33633]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-21T13:24:40.321 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:39 smithi144 ceph-mon[33633]: osdmap e28: 5 total, 4 up, 5 in 2024-03-21T13:24:40.321 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:39 smithi144 ceph-mon[33633]: from='osd.4 v2:172.21.15.144:6800/913882960' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-21T13:24:40.322 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:39 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T13:24:40.322 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:39 smithi144 ceph-mon[33633]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-21T13:24:40.322 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:39 smithi144 ceph-mon[33633]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T13:24:40.621 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 21 13:24:40 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-4[39152]: 2024-03-21T13:24:40.416+0000 7fa4bd244700 -1 osd.4 0 waiting for initial osdmap 2024-03-21T13:24:40.621 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 21 13:24:40 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-4[39152]: 2024-03-21T13:24:40.422+0000 7fa4b502f700 -1 osd.4 29 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T13:24:41.264 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:40 smithi144 ceph-mon[33633]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-21T13:24:41.264 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:40 smithi144 ceph-mon[33633]: osdmap e29: 5 total, 4 up, 5 in 2024-03-21T13:24:41.264 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:40 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T13:24:41.264 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:40 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T13:24:41.264 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:40 smithi144 ceph-mon[33633]: from='osd.4 ' entity='osd.4' 2024-03-21T13:24:41.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:40 smithi097 ceph-mon[30040]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-21T13:24:41.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:40 smithi097 ceph-mon[30040]: osdmap e29: 5 total, 4 up, 5 in 2024-03-21T13:24:41.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:40 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T13:24:41.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:40 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T13:24:41.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:40 smithi097 ceph-mon[30040]: from='osd.4 ' entity='osd.4' 2024-03-21T13:24:41.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:40 smithi097 ceph-mon[39416]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-21T13:24:41.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:40 smithi097 ceph-mon[39416]: osdmap e29: 5 total, 4 up, 5 in 2024-03-21T13:24:41.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:40 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T13:24:41.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:40 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T13:24:41.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:40 smithi097 ceph-mon[39416]: from='osd.4 ' entity='osd.4' 2024-03-21T13:24:42.160 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:41 smithi144 ceph-mon[33633]: purged_snaps scrub starts 2024-03-21T13:24:42.160 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:41 smithi144 ceph-mon[33633]: purged_snaps scrub ok 2024-03-21T13:24:42.160 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:41 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T13:24:42.161 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:41 smithi144 ceph-mon[33633]: osd.4 v2:172.21.15.144:6800/913882960 boot 2024-03-21T13:24:42.161 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:41 smithi144 ceph-mon[33633]: osdmap e30: 5 total, 5 up, 5 in 2024-03-21T13:24:42.161 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:41 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T13:24:42.161 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:41 smithi144 ceph-mon[33633]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T13:24:42.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:41 smithi097 ceph-mon[30040]: purged_snaps scrub starts 2024-03-21T13:24:42.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:41 smithi097 ceph-mon[30040]: purged_snaps scrub ok 2024-03-21T13:24:42.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:41 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T13:24:42.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:41 smithi097 ceph-mon[30040]: osd.4 v2:172.21.15.144:6800/913882960 boot 2024-03-21T13:24:42.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:41 smithi097 ceph-mon[30040]: osdmap e30: 5 total, 5 up, 5 in 2024-03-21T13:24:42.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:41 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T13:24:42.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:41 smithi097 ceph-mon[30040]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T13:24:42.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:41 smithi097 ceph-mon[39416]: purged_snaps scrub starts 2024-03-21T13:24:42.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:41 smithi097 ceph-mon[39416]: purged_snaps scrub ok 2024-03-21T13:24:42.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:41 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T13:24:42.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:41 smithi097 ceph-mon[39416]: osd.4 v2:172.21.15.144:6800/913882960 boot 2024-03-21T13:24:42.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:41 smithi097 ceph-mon[39416]: osdmap e30: 5 total, 5 up, 5 in 2024-03-21T13:24:42.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:41 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T13:24:42.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:41 smithi097 ceph-mon[39416]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T13:24:42.636 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-21T13:24:42.636 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-21T13:24:42.636 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10+0 records in 2024-03-21T13:24:42.636 INFO:teuthology.orchestra.run.smithi144.stderr:10+0 records out 2024-03-21T13:24:42.636 INFO:teuthology.orchestra.run.smithi144.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0129913 s, 807 MB/s 2024-03-21T13:24:42.636 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 2024-03-21T13:24:42.637 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping successful for: 2024-03-21T13:24:43.206 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:43 smithi144 ceph-mon[33633]: osdmap e31: 5 total, 5 up, 5 in 2024-03-21T13:24:43.240 DEBUG:teuthology.orchestra.run.smithi144:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi144:vg_nvme/lv_3 2024-03-21T13:24:43.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:43 smithi097 ceph-mon[39416]: osdmap e31: 5 total, 5 up, 5 in 2024-03-21T13:24:43.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:43 smithi097 ceph-mon[30040]: osdmap e31: 5 total, 5 up, 5 in 2024-03-21T13:24:43.664 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T13:24:44.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:44 smithi097 ceph-mon[30040]: osdmap e32: 5 total, 5 up, 5 in 2024-03-21T13:24:44.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:44 smithi097 ceph-mon[30040]: pgmap v118: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T13:24:44.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:44 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:44.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:44 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:44.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:44 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:24:44.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:44 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:44.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:44 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:24:44.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:44 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:24:44.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:44 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:44.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:44 smithi097 ceph-mon[39416]: osdmap e32: 5 total, 5 up, 5 in 2024-03-21T13:24:44.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:44 smithi097 ceph-mon[39416]: pgmap v118: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T13:24:44.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:44 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:44.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:44 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:44.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:44 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:24:44.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:44 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:44.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:44 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:24:44.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:44 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:24:44.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:44 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:44.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:44 smithi144 ceph-mon[33633]: osdmap e32: 5 total, 5 up, 5 in 2024-03-21T13:24:44.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:44 smithi144 ceph-mon[33633]: pgmap v118: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T13:24:44.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:44 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:44.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:44 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:44.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:44 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:24:44.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:44 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:44.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:44 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:24:44.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:44 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:24:44.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:44 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:24:45.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:45 smithi144 ceph-mon[33633]: Detected new or changed devices on smithi144 2024-03-21T13:24:45.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:45 smithi144 ceph-mon[33633]: Adjusting osd_memory_target on smithi144 to 17088M 2024-03-21T13:24:45.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:45 smithi144 ceph-mon[33633]: osdmap e33: 5 total, 5 up, 5 in 2024-03-21T13:24:45.217 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:45.205+0000 7fb74dd9c700 1 -- 172.21.15.144:0/2993440386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb748102cd0 msgr2=0x7fb748109580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:24:45.218 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:45.205+0000 7fb74dd9c700 1 --2- 172.21.15.144:0/2993440386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb748102cd0 0x7fb748109580 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb74000b0d0 tx=0x7fb74000b3e0 comp rx=0 tx=0).stop 2024-03-21T13:24:45.218 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:45.205+0000 7fb74dd9c700 1 -- 172.21.15.144:0/2993440386 shutdown_connections 2024-03-21T13:24:45.218 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:45.205+0000 7fb74dd9c700 1 --2- 172.21.15.144:0/2993440386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb748102cd0 0x7fb748109580 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:24:45.218 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:45.205+0000 7fb74dd9c700 1 --2- 172.21.15.144:0/2993440386 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb748101130 0x7fb748101550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:24:45.218 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:45.205+0000 7fb74dd9c700 1 --2- 172.21.15.144:0/2993440386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb748102330 0x7fb748102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:24:45.218 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:45.205+0000 7fb74dd9c700 1 -- 172.21.15.144:0/2993440386 >> 172.21.15.144:0/2993440386 conn(0x7fb7480fc6f0 msgr2=0x7fb7480feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:24:45.218 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:45.206+0000 7fb74dd9c700 1 -- 172.21.15.144:0/2993440386 shutdown_connections 2024-03-21T13:24:45.218 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:45.206+0000 7fb74dd9c700 1 -- 172.21.15.144:0/2993440386 wait complete. 2024-03-21T13:24:45.218 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:45.206+0000 7fb74dd9c700 1 Processor -- start 2024-03-21T13:24:45.218 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:45.206+0000 7fb74dd9c700 1 -- start start 2024-03-21T13:24:45.218 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:45.206+0000 7fb74dd9c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb748101130 0x7fb748197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:24:45.218 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:45.206+0000 7fb74dd9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb748102330 0x7fb748197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:24:45.219 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:45.206+0000 7fb74dd9c700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb748102cd0 0x7fb74819f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:24:45.219 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:45.206+0000 7fb74dd9c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb74810c430 con 0x7fb748102330 2024-03-21T13:24:45.219 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:45.206+0000 7fb74dd9c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb74810c2b0 con 0x7fb748101130 2024-03-21T13:24:45.219 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:45.206+0000 7fb74dd9c700 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb74810c5b0 con 0x7fb748102cd0 2024-03-21T13:24:45.219 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:45.207+0000 7fb7477fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb748101130 0x7fb748197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:24:45.219 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:45.207+0000 7fb7477fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb748101130 0x7fb748197a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:57874/0 (socket says 172.21.15.144:57874) 2024-03-21T13:24:45.219 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:45.207+0000 7fb7477fe700 1 -- 172.21.15.144:0/146756170 learned_addr learned my addr 172.21.15.144:0/146756170 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-21T13:24:45.219 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:45.207+0000 7fb746ffd700 1 --2- 172.21.15.144:0/146756170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb748102330 0x7fb748197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:24:45.219 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:45.207+0000 7fb747fff700 1 --2- 172.21.15.144:0/146756170 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb748102cd0 0x7fb74819f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:24:45.219 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:45.207+0000 7fb7477fe700 1 -- 172.21.15.144:0/146756170 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb748102cd0 msgr2=0x7fb74819f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:24:45.219 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:45.207+0000 7fb7477fe700 1 --2- 172.21.15.144:0/146756170 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb748102cd0 0x7fb74819f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:24:45.219 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:45.207+0000 7fb7477fe700 1 -- 172.21.15.144:0/146756170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb748102330 msgr2=0x7fb748197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:24:45.219 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:45.207+0000 7fb7477fe700 1 --2- 172.21.15.144:0/146756170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb748102330 0x7fb748197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:24:45.219 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:45.207+0000 7fb7477fe700 1 -- 172.21.15.144:0/146756170 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb74819f6a0 con 0x7fb748101130 2024-03-21T13:24:45.220 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:45.207+0000 7fb746ffd700 1 --2- 172.21.15.144:0/146756170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb748102330 0x7fb748197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T13:24:45.220 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:45.207+0000 7fb7477fe700 1 --2- 172.21.15.144:0/146756170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb748101130 0x7fb748197a40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb73800d980 tx=0x7fb73800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:24:45.220 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:45.207+0000 7fb744ff9700 1 -- 172.21.15.144:0/146756170 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb738017070 con 0x7fb748101130 2024-03-21T13:24:45.220 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:45.207+0000 7fb744ff9700 1 -- 172.21.15.144:0/146756170 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fb738009de0 con 0x7fb748101130 2024-03-21T13:24:45.220 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:45.207+0000 7fb744ff9700 1 -- 172.21.15.144:0/146756170 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73800f3c0 con 0x7fb748101130 2024-03-21T13:24:45.220 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:45.207+0000 7fb74dd9c700 1 -- 172.21.15.144:0/146756170 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb74819f990 con 0x7fb748101130 2024-03-21T13:24:45.220 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:45.207+0000 7fb74dd9c700 1 -- 172.21.15.144:0/146756170 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb74819fe80 con 0x7fb748101130 2024-03-21T13:24:45.220 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:45.208+0000 7fb74dd9c700 1 -- 172.21.15.144:0/146756170 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb748068e50 con 0x7fb748101130 2024-03-21T13:24:45.220 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:45.209+0000 7fb744ff9700 1 -- 172.21.15.144:0/146756170 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 15) v1 ==== 92095+0+0 (secure 0 0 0) 0x7fb738009930 con 0x7fb748101130 2024-03-21T13:24:45.220 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:45.209+0000 7fb744ff9700 1 --2- 172.21.15.144:0/146756170 >> v2:172.21.15.97:6800/1943212787 conn(0x7fb73006e210 0x7fb7300706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:24:45.220 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:45.209+0000 7fb744ff9700 1 -- 172.21.15.144:0/146756170 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 3205+0+0 (secure 0 0 0) 0x7fb73808d210 con 0x7fb748101130 2024-03-21T13:24:45.220 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:45.209+0000 7fb746ffd700 1 --2- 172.21.15.144:0/146756170 >> v2:172.21.15.97:6800/1943212787 conn(0x7fb73006e210 0x7fb7300706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:24:45.220 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:45.210+0000 7fb746ffd700 1 --2- 172.21.15.144:0/146756170 >> v2:172.21.15.97:6800/1943212787 conn(0x7fb73006e210 0x7fb7300706d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb73c005950 tx=0x7fb73c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:24:45.220 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:45.214+0000 7fb744ff9700 1 -- 172.21.15.144:0/146756170 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb73805b550 con 0x7fb748101130 2024-03-21T13:24:45.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:45 smithi097 ceph-mon[30040]: Detected new or changed devices on smithi144 2024-03-21T13:24:45.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:45 smithi097 ceph-mon[30040]: Adjusting osd_memory_target on smithi144 to 17088M 2024-03-21T13:24:45.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:45 smithi097 ceph-mon[30040]: osdmap e33: 5 total, 5 up, 5 in 2024-03-21T13:24:45.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:45 smithi097 ceph-mon[39416]: Detected new or changed devices on smithi144 2024-03-21T13:24:45.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:45 smithi097 ceph-mon[39416]: Adjusting osd_memory_target on smithi144 to 17088M 2024-03-21T13:24:45.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:45 smithi097 ceph-mon[39416]: osdmap e33: 5 total, 5 up, 5 in 2024-03-21T13:24:45.418 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:24:45.418+0000 7fb74dd9c700 1 -- 172.21.15.144:0/146756170 --> v2:172.21.15.97:6800/1943212787 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fb7481a0110 con 0x7fb73006e210 2024-03-21T13:24:46.276 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:46 smithi144 ceph-mon[33633]: osdmap e34: 5 total, 5 up, 5 in 2024-03-21T13:24:46.276 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:46 smithi144 ceph-mon[33633]: pgmap v121: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T13:24:46.276 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:46 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T13:24:46.276 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:46 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T13:24:46.276 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:46 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:24:46.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:46 smithi097 ceph-mon[39416]: osdmap e34: 5 total, 5 up, 5 in 2024-03-21T13:24:46.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:46 smithi097 ceph-mon[39416]: pgmap v121: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T13:24:46.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:46 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T13:24:46.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:46 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T13:24:46.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:46 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:24:46.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:46 smithi097 ceph-mon[30040]: osdmap e34: 5 total, 5 up, 5 in 2024-03-21T13:24:46.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:46 smithi097 ceph-mon[30040]: pgmap v121: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T13:24:46.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:46 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T13:24:46.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:46 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T13:24:46.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:46 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:24:47.255 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:47 smithi144 ceph-mon[33633]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:24:47.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:47 smithi097 ceph-mon[30040]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:24:47.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:47 smithi097 ceph-mon[39416]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:24:48.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:48 smithi097 ceph-mon[30040]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-03-21T13:24:48.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:48 smithi097 ceph-mon[39416]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-03-21T13:24:48.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:48 smithi144 ceph-mon[33633]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-03-21T13:24:49.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:49 smithi097 ceph-mon[39416]: from='client.? 172.21.15.144:0/3656580433' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3f7f8a2-d40b-4866-8dad-310dca09941c"}]: dispatch 2024-03-21T13:24:49.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:49 smithi097 ceph-mon[39416]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3f7f8a2-d40b-4866-8dad-310dca09941c"}]: dispatch 2024-03-21T13:24:49.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:49 smithi097 ceph-mon[39416]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f3f7f8a2-d40b-4866-8dad-310dca09941c"}]': finished 2024-03-21T13:24:49.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:49 smithi097 ceph-mon[39416]: osdmap e35: 6 total, 5 up, 6 in 2024-03-21T13:24:49.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:49 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T13:24:49.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:49 smithi097 ceph-mon[39416]: from='client.? 172.21.15.144:0/8506624' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T13:24:49.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:49 smithi097 ceph-mon[30040]: from='client.? 172.21.15.144:0/3656580433' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3f7f8a2-d40b-4866-8dad-310dca09941c"}]: dispatch 2024-03-21T13:24:49.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:49 smithi097 ceph-mon[30040]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3f7f8a2-d40b-4866-8dad-310dca09941c"}]: dispatch 2024-03-21T13:24:49.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:49 smithi097 ceph-mon[30040]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f3f7f8a2-d40b-4866-8dad-310dca09941c"}]': finished 2024-03-21T13:24:49.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:49 smithi097 ceph-mon[30040]: osdmap e35: 6 total, 5 up, 6 in 2024-03-21T13:24:49.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:49 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T13:24:49.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:49 smithi097 ceph-mon[30040]: from='client.? 172.21.15.144:0/8506624' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T13:24:49.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:49 smithi144 ceph-mon[33633]: from='client.? 172.21.15.144:0/3656580433' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3f7f8a2-d40b-4866-8dad-310dca09941c"}]: dispatch 2024-03-21T13:24:49.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:49 smithi144 ceph-mon[33633]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3f7f8a2-d40b-4866-8dad-310dca09941c"}]: dispatch 2024-03-21T13:24:49.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:49 smithi144 ceph-mon[33633]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f3f7f8a2-d40b-4866-8dad-310dca09941c"}]': finished 2024-03-21T13:24:49.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:49 smithi144 ceph-mon[33633]: osdmap e35: 6 total, 5 up, 6 in 2024-03-21T13:24:49.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:49 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T13:24:49.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:49 smithi144 ceph-mon[33633]: from='client.? 172.21.15.144:0/8506624' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T13:24:50.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:50 smithi097 ceph-mon[30040]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-21T13:24:50.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:50 smithi097 ceph-mon[39416]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-21T13:24:50.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:50 smithi144 ceph-mon[33633]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-21T13:24:52.518 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:52 smithi144 ceph-mon[33633]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-03-21T13:24:52.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:52 smithi097 ceph-mon[30040]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-03-21T13:24:52.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:52 smithi097 ceph-mon[39416]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-03-21T13:24:54.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:54 smithi144 ceph-mon[33633]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-03-21T13:24:54.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:54 smithi097 ceph-mon[30040]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-03-21T13:24:54.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:54 smithi097 ceph-mon[39416]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-03-21T13:24:55.422 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:55 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-21T13:24:55.422 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:55 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:24:55.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:55 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-21T13:24:55.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:55 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:24:55.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:55 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-21T13:24:55.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:55 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:24:56.292 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:56 smithi144 ceph-mon[33633]: Deploying daemon osd.5 on smithi144 2024-03-21T13:24:56.292 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:56 smithi144 ceph-mon[33633]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-21T13:24:56.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:56 smithi097 ceph-mon[30040]: Deploying daemon osd.5 on smithi144 2024-03-21T13:24:56.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:56 smithi097 ceph-mon[30040]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-21T13:24:56.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:56 smithi097 ceph-mon[39416]: Deploying daemon osd.5 on smithi144 2024-03-21T13:24:56.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:56 smithi097 ceph-mon[39416]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-21T13:24:58.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:24:58 smithi144 ceph-mon[33633]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T13:24:58.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:24:58 smithi097 ceph-mon[30040]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T13:24:58.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:24:58 smithi097 ceph-mon[39416]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T13:25:00.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:00 smithi144 ceph-mon[33633]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T13:25:00.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:00 smithi097 ceph-mon[30040]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T13:25:00.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:00 smithi097 ceph-mon[39416]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T13:25:01.263 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:01 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:25:01.263 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:01 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:01.263 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:01 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:01.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:01 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:25:01.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:01 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:01.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:01 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:01.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:01 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:25:01.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:01 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:01.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:01 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:02.524 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:02 smithi144 ceph-mon[33633]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T13:25:02.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:02 smithi097 ceph-mon[30040]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T13:25:02.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:02 smithi097 ceph-mon[39416]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T13:25:03.882 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:03.882+0000 7fb744ff9700 1 -- 172.21.15.144:0/146756170 <== mgr.14152 v2:172.21.15.97:6800/1943212787 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb7481a0110 con 0x7fb73006e210 2024-03-21T13:25:03.882 INFO:teuthology.orchestra.run.smithi144.stdout:Created osd(s) 5 on host 'smithi144' 2024-03-21T13:25:03.884 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:03.884+0000 7fb74dd9c700 1 -- 172.21.15.144:0/146756170 >> v2:172.21.15.97:6800/1943212787 conn(0x7fb73006e210 msgr2=0x7fb7300706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:25:03.884 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:03.884+0000 7fb74dd9c700 1 --2- 172.21.15.144:0/146756170 >> v2:172.21.15.97:6800/1943212787 conn(0x7fb73006e210 0x7fb7300706d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb73c005950 tx=0x7fb73c0058e0 comp rx=0 tx=0).stop 2024-03-21T13:25:03.884 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:03.884+0000 7fb74dd9c700 1 -- 172.21.15.144:0/146756170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb748101130 msgr2=0x7fb748197a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:25:03.884 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:03.884+0000 7fb74dd9c700 1 --2- 172.21.15.144:0/146756170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb748101130 0x7fb748197a40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb73800d980 tx=0x7fb73800dc90 comp rx=0 tx=0).stop 2024-03-21T13:25:03.884 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:03.884+0000 7fb74dd9c700 1 -- 172.21.15.144:0/146756170 shutdown_connections 2024-03-21T13:25:03.884 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:03.884+0000 7fb74dd9c700 1 --2- 172.21.15.144:0/146756170 >> v2:172.21.15.97:6800/1943212787 conn(0x7fb73006e210 0x7fb7300706d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:25:03.884 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:03.884+0000 7fb74dd9c700 1 --2- 172.21.15.144:0/146756170 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb748102cd0 0x7fb74819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:25:03.885 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:03.884+0000 7fb74dd9c700 1 --2- 172.21.15.144:0/146756170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb748101130 0x7fb748197a40 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:25:03.885 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:03.884+0000 7fb74dd9c700 1 --2- 172.21.15.144:0/146756170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb748102330 0x7fb748197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:25:03.885 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:03.884+0000 7fb74dd9c700 1 -- 172.21.15.144:0/146756170 >> 172.21.15.144:0/146756170 conn(0x7fb7480fc6f0 msgr2=0x7fb7480fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:25:03.885 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:03.884+0000 7fb74dd9c700 1 -- 172.21.15.144:0/146756170 shutdown_connections 2024-03-21T13:25:03.885 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:03.884+0000 7fb74dd9c700 1 -- 172.21.15.144:0/146756170 wait complete. 2024-03-21T13:25:03.895 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:03 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:03.896 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:03 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:03.896 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:03 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:25:03.896 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:03 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:25:03.896 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:03 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:03.896 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:03 smithi144 ceph-mon[33633]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T13:25:03.896 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:03 smithi144 ceph-mon[33633]: from='osd.5 v2:172.21.15.144:6804/2553466741' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-21T13:25:03.896 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:03 smithi144 ceph-mon[33633]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-21T13:25:04.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:03 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:04.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:03 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:04.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:03 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:25:04.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:03 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:25:04.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:03 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:04.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:03 smithi097 ceph-mon[30040]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T13:25:04.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:03 smithi097 ceph-mon[30040]: from='osd.5 v2:172.21.15.144:6804/2553466741' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-21T13:25:04.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:03 smithi097 ceph-mon[30040]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-21T13:25:04.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:03 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:04.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:03 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:04.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:03 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:25:04.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:03 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:25:04.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:03 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:04.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:03 smithi097 ceph-mon[39416]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T13:25:04.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:03 smithi097 ceph-mon[39416]: from='osd.5 v2:172.21.15.144:6804/2553466741' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-21T13:25:04.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:03 smithi097 ceph-mon[39416]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-21T13:25:04.484 DEBUG:teuthology.orchestra.run.smithi144:osd.5> sudo journalctl -f -n 0 -u ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@osd.5.service 2024-03-21T13:25:04.486 INFO:tasks.cephadm:Deploying osd.6 on smithi144 with /dev/vg_nvme/lv_2... 2024-03-21T13:25:04.487 DEBUG:teuthology.orchestra.run.smithi144:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-21T13:25:04.749 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:04 smithi144 ceph-mon[33633]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-21T13:25:04.750 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:04 smithi144 ceph-mon[33633]: from='osd.5 v2:172.21.15.144:6804/2553466741' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-21T13:25:04.750 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:04 smithi144 ceph-mon[33633]: osdmap e36: 6 total, 5 up, 6 in 2024-03-21T13:25:04.750 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:04 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T13:25:04.750 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:04 smithi144 ceph-mon[33633]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-21T13:25:04.750 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:04 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:04.750 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:04 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:04.750 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:04 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:25:05.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:04 smithi097 ceph-mon[30040]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-21T13:25:05.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:04 smithi097 ceph-mon[30040]: from='osd.5 v2:172.21.15.144:6804/2553466741' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-21T13:25:05.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:04 smithi097 ceph-mon[30040]: osdmap e36: 6 total, 5 up, 6 in 2024-03-21T13:25:05.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:04 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T13:25:05.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:04 smithi097 ceph-mon[30040]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-21T13:25:05.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:04 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:05.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:04 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:05.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:04 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:25:05.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:04 smithi097 ceph-mon[39416]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-21T13:25:05.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:04 smithi097 ceph-mon[39416]: from='osd.5 v2:172.21.15.144:6804/2553466741' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-21T13:25:05.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:04 smithi097 ceph-mon[39416]: osdmap e36: 6 total, 5 up, 6 in 2024-03-21T13:25:05.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:04 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T13:25:05.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:04 smithi097 ceph-mon[39416]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-21T13:25:05.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:04 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:05.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:04 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:05.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:04 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:25:05.493 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 21 13:25:05 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-5[45082]: 2024-03-21T13:25:05.147+0000 7f975961d700 -1 osd.5 0 waiting for initial osdmap 2024-03-21T13:25:05.494 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 21 13:25:05 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-5[45082]: 2024-03-21T13:25:05.151+0000 7f9751408700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T13:25:05.768 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:05 smithi144 ceph-mon[33633]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-21T13:25:05.768 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:05 smithi144 ceph-mon[33633]: osdmap e37: 6 total, 5 up, 6 in 2024-03-21T13:25:05.768 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:05 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T13:25:05.768 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:05 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T13:25:05.768 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:05 smithi144 ceph-mon[33633]: pgmap v134: 1 pgs: 1 unknown; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T13:25:05.768 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:05 smithi144 ceph-mon[33633]: from='osd.5 ' entity='osd.5' 2024-03-21T13:25:06.002 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T13:25:06.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:05 smithi097 ceph-mon[30040]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-21T13:25:06.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:05 smithi097 ceph-mon[30040]: osdmap e37: 6 total, 5 up, 6 in 2024-03-21T13:25:06.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:05 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T13:25:06.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:05 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T13:25:06.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:05 smithi097 ceph-mon[30040]: pgmap v134: 1 pgs: 1 unknown; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T13:25:06.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:05 smithi097 ceph-mon[30040]: from='osd.5 ' entity='osd.5' 2024-03-21T13:25:06.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:05 smithi097 ceph-mon[39416]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-21T13:25:06.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:05 smithi097 ceph-mon[39416]: osdmap e37: 6 total, 5 up, 6 in 2024-03-21T13:25:06.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:05 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T13:25:06.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:05 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T13:25:06.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:05 smithi097 ceph-mon[39416]: pgmap v134: 1 pgs: 1 unknown; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T13:25:06.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:05 smithi097 ceph-mon[39416]: from='osd.5 ' entity='osd.5' 2024-03-21T13:25:06.956 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:06 smithi097 ceph-mon[30040]: purged_snaps scrub starts 2024-03-21T13:25:06.956 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:06 smithi097 ceph-mon[30040]: purged_snaps scrub ok 2024-03-21T13:25:06.956 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:06 smithi097 ceph-mon[30040]: osd.5 v2:172.21.15.144:6804/2553466741 boot 2024-03-21T13:25:06.957 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:06 smithi097 ceph-mon[30040]: osdmap e38: 6 total, 6 up, 6 in 2024-03-21T13:25:06.957 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:06 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T13:25:06.957 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:06 smithi097 ceph-mon[30040]: osdmap e39: 6 total, 6 up, 6 in 2024-03-21T13:25:06.957 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:06 smithi097 ceph-mon[39416]: purged_snaps scrub starts 2024-03-21T13:25:06.957 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:06 smithi097 ceph-mon[39416]: purged_snaps scrub ok 2024-03-21T13:25:06.957 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:06 smithi097 ceph-mon[39416]: osd.5 v2:172.21.15.144:6804/2553466741 boot 2024-03-21T13:25:06.957 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:06 smithi097 ceph-mon[39416]: osdmap e38: 6 total, 6 up, 6 in 2024-03-21T13:25:06.957 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:06 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T13:25:06.957 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:06 smithi097 ceph-mon[39416]: osdmap e39: 6 total, 6 up, 6 in 2024-03-21T13:25:06.962 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:06 smithi144 ceph-mon[33633]: purged_snaps scrub starts 2024-03-21T13:25:06.963 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:06 smithi144 ceph-mon[33633]: purged_snaps scrub ok 2024-03-21T13:25:06.963 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:06 smithi144 ceph-mon[33633]: osd.5 v2:172.21.15.144:6804/2553466741 boot 2024-03-21T13:25:06.963 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:06 smithi144 ceph-mon[33633]: osdmap e38: 6 total, 6 up, 6 in 2024-03-21T13:25:06.963 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:06 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T13:25:06.963 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:06 smithi144 ceph-mon[33633]: osdmap e39: 6 total, 6 up, 6 in 2024-03-21T13:25:08.484 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-21T13:25:08.484 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-21T13:25:08.484 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10+0 records in 2024-03-21T13:25:08.484 INFO:teuthology.orchestra.run.smithi144.stderr:10+0 records out 2024-03-21T13:25:08.484 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114283 s, 918 MB/s 2024-03-21T13:25:08.484 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping successful for: 2024-03-21T13:25:08.509 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:08 smithi144 ceph-mon[33633]: pgmap v137: 1 pgs: 1 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T13:25:08.509 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:08 smithi144 ceph-mon[33633]: Detected new or changed devices on smithi144 2024-03-21T13:25:08.509 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:08 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:08.509 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:08 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:08.509 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:08 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:25:08.510 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:08 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:25:08.510 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:08 smithi144 ceph-mon[33633]: Adjusting osd_memory_target on smithi144 to 8544M 2024-03-21T13:25:08.510 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:08 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:08.510 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:08 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:25:08.510 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:08 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:25:08.510 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:08 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:08.510 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:08 smithi144 ceph-mon[33633]: osdmap e40: 6 total, 6 up, 6 in 2024-03-21T13:25:08.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:08 smithi097 ceph-mon[30040]: pgmap v137: 1 pgs: 1 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T13:25:08.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:08 smithi097 ceph-mon[30040]: Detected new or changed devices on smithi144 2024-03-21T13:25:08.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:08 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:08.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:08 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:08.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:08 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:25:08.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:08 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:25:08.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:08 smithi097 ceph-mon[30040]: Adjusting osd_memory_target on smithi144 to 8544M 2024-03-21T13:25:08.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:08 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:08.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:08 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:25:08.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:08 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:25:08.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:08 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:08.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:08 smithi097 ceph-mon[30040]: osdmap e40: 6 total, 6 up, 6 in 2024-03-21T13:25:08.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:08 smithi097 ceph-mon[39416]: pgmap v137: 1 pgs: 1 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T13:25:08.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:08 smithi097 ceph-mon[39416]: Detected new or changed devices on smithi144 2024-03-21T13:25:08.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:08 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:08.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:08 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:08.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:08 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:25:08.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:08 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:25:08.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:08 smithi097 ceph-mon[39416]: Adjusting osd_memory_target on smithi144 to 8544M 2024-03-21T13:25:08.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:08 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:08.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:08 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:25:08.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:08 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:25:08.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:08 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:08.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:08 smithi097 ceph-mon[39416]: osdmap e40: 6 total, 6 up, 6 in 2024-03-21T13:25:09.109 DEBUG:teuthology.orchestra.run.smithi144:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi144:vg_nvme/lv_2 2024-03-21T13:25:09.347 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T13:25:10.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:10 smithi144 ceph-mon[33633]: pgmap v139: 1 pgs: 1 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T13:25:10.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:10 smithi144 ceph-mon[33633]: osdmap e41: 6 total, 6 up, 6 in 2024-03-21T13:25:10.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:10 smithi097 ceph-mon[30040]: pgmap v139: 1 pgs: 1 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T13:25:10.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:10 smithi097 ceph-mon[30040]: osdmap e41: 6 total, 6 up, 6 in 2024-03-21T13:25:10.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:10 smithi097 ceph-mon[39416]: pgmap v139: 1 pgs: 1 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T13:25:10.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:10 smithi097 ceph-mon[39416]: osdmap e41: 6 total, 6 up, 6 in 2024-03-21T13:25:10.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:10.903+0000 7f8b0c131700 1 -- 172.21.15.144:0/235774629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b041010b0 msgr2=0x7f8b041014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:25:10.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:10.903+0000 7f8b0c131700 1 --2- 172.21.15.144:0/235774629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b041010b0 0x7f8b041014d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8b0000b0d0 tx=0x7f8b0000b3e0 comp rx=0 tx=0).stop 2024-03-21T13:25:10.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:10.903+0000 7f8b0c131700 1 -- 172.21.15.144:0/235774629 shutdown_connections 2024-03-21T13:25:10.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:10.903+0000 7f8b0c131700 1 --2- 172.21.15.144:0/235774629 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8b04102c50 0x7f8b04109500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:25:10.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:10.903+0000 7f8b0c131700 1 --2- 172.21.15.144:0/235774629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b041010b0 0x7f8b041014d0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:25:10.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:10.903+0000 7f8b0c131700 1 --2- 172.21.15.144:0/235774629 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b041022b0 0x7f8b04102710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:25:10.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:10.903+0000 7f8b0c131700 1 -- 172.21.15.144:0/235774629 >> 172.21.15.144:0/235774629 conn(0x7f8b040fc650 msgr2=0x7f8b040fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:25:10.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:10.903+0000 7f8b0c131700 1 -- 172.21.15.144:0/235774629 shutdown_connections 2024-03-21T13:25:10.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:10.903+0000 7f8b0c131700 1 -- 172.21.15.144:0/235774629 wait complete. 2024-03-21T13:25:10.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:10.904+0000 7f8b0c131700 1 Processor -- start 2024-03-21T13:25:10.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:10.904+0000 7f8b0c131700 1 -- start start 2024-03-21T13:25:10.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:10.904+0000 7f8b0c131700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8b041010b0 0x7f8b04105040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:25:10.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:10.904+0000 7f8b0c131700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b041022b0 0x7f8b04103690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:25:10.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:10.904+0000 7f8b0c131700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b04102c50 0x7f8b04103bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:25:10.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:10.904+0000 7f8b0c131700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0410c420 con 0x7f8b041022b0 2024-03-21T13:25:10.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:10.904+0000 7f8b0c131700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0410c2a0 con 0x7f8b04102c50 2024-03-21T13:25:10.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:10.904+0000 7f8b0c131700 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0410c5a0 con 0x7f8b041010b0 2024-03-21T13:25:10.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:10.904+0000 7f8b0a6ce700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b04102c50 0x7f8b04103bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:25:10.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:10.904+0000 7f8b0a6ce700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b04102c50 0x7f8b04103bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:59618/0 (socket says 172.21.15.144:59618) 2024-03-21T13:25:10.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:10.904+0000 7f8b0a6ce700 1 -- 172.21.15.144:0/613037422 learned_addr learned my addr 172.21.15.144:0/613037422 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-21T13:25:10.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:10.904+0000 7f8b09ecd700 1 --2- 172.21.15.144:0/613037422 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8b041010b0 0x7f8b04105040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:25:10.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:10.904+0000 7f8b096cc700 1 --2- 172.21.15.144:0/613037422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b041022b0 0x7f8b04103690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:25:10.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:10.904+0000 7f8b0a6ce700 1 -- 172.21.15.144:0/613037422 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8b041010b0 msgr2=0x7f8b04105040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:25:10.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:10.904+0000 7f8b0a6ce700 1 --2- 172.21.15.144:0/613037422 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8b041010b0 0x7f8b04105040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:25:10.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:10.904+0000 7f8b0a6ce700 1 -- 172.21.15.144:0/613037422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b041022b0 msgr2=0x7f8b04103690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:25:10.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:10.904+0000 7f8b0a6ce700 1 --2- 172.21.15.144:0/613037422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b041022b0 0x7f8b04103690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:25:10.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:10.904+0000 7f8b0a6ce700 1 -- 172.21.15.144:0/613037422 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b04104480 con 0x7f8b04102c50 2024-03-21T13:25:10.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:10.904+0000 7f8b096cc700 1 --2- 172.21.15.144:0/613037422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b041022b0 0x7f8b04103690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T13:25:10.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:10.904+0000 7f8b0a6ce700 1 --2- 172.21.15.144:0/613037422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b04102c50 0x7f8b04103bd0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8afc00d980 tx=0x7f8afc00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:25:10.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:10.905+0000 7f8afaffd700 1 -- 172.21.15.144:0/613037422 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8afc017070 con 0x7f8b04102c50 2024-03-21T13:25:10.908 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:10.905+0000 7f8afaffd700 1 -- 172.21.15.144:0/613037422 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f8afc005020 con 0x7f8b04102c50 2024-03-21T13:25:10.908 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:10.905+0000 7f8afaffd700 1 -- 172.21.15.144:0/613037422 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8afc009990 con 0x7f8b04102c50 2024-03-21T13:25:10.908 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:10.905+0000 7f8b0c131700 1 -- 172.21.15.144:0/613037422 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b041a1c80 con 0x7f8b04102c50 2024-03-21T13:25:10.908 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:10.905+0000 7f8b0c131700 1 -- 172.21.15.144:0/613037422 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b041a21c0 con 0x7f8b04102c50 2024-03-21T13:25:10.909 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:10.906+0000 7f8b0c131700 1 -- 172.21.15.144:0/613037422 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b04068e50 con 0x7f8b04102c50 2024-03-21T13:25:10.909 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:10.906+0000 7f8afaffd700 1 -- 172.21.15.144:0/613037422 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 15) v1 ==== 92095+0+0 (secure 0 0 0) 0x7f8afc009b30 con 0x7f8b04102c50 2024-03-21T13:25:10.909 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:10.906+0000 7f8afaffd700 1 --2- 172.21.15.144:0/613037422 >> v2:172.21.15.97:6800/1943212787 conn(0x7f8af006e210 0x7f8af00706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:25:10.909 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:10.907+0000 7f8afaffd700 1 -- 172.21.15.144:0/613037422 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 3497+0+0 (secure 0 0 0) 0x7f8afc08d560 con 0x7f8b04102c50 2024-03-21T13:25:10.909 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:10.907+0000 7f8b09ecd700 1 --2- 172.21.15.144:0/613037422 >> v2:172.21.15.97:6800/1943212787 conn(0x7f8af006e210 0x7f8af00706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:25:10.909 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:10.907+0000 7f8b09ecd700 1 --2- 172.21.15.144:0/613037422 >> v2:172.21.15.97:6800/1943212787 conn(0x7f8af006e210 0x7f8af00706d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8b00009fd0 tx=0x7f8b0000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:25:10.911 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:10.911+0000 7f8afaffd700 1 -- 172.21.15.144:0/613037422 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8afc020020 con 0x7f8b04102c50 2024-03-21T13:25:11.116 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:11.116+0000 7f8b0c131700 1 -- 172.21.15.144:0/613037422 --> v2:172.21.15.97:6800/1943212787 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f8b040fe7b0 con 0x7f8af006e210 2024-03-21T13:25:11.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:11 smithi144 ceph-mon[33633]: osdmap e42: 6 total, 6 up, 6 in 2024-03-21T13:25:11.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:11 smithi144 ceph-mon[33633]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-03-21T13:25:11.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:11 smithi144 ceph-mon[33633]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:25:11.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:11 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T13:25:11.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:11 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T13:25:11.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:11 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:25:11.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:11 smithi097 ceph-mon[30040]: osdmap e42: 6 total, 6 up, 6 in 2024-03-21T13:25:11.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:11 smithi097 ceph-mon[30040]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-03-21T13:25:11.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:11 smithi097 ceph-mon[30040]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:25:11.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:11 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T13:25:11.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:11 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T13:25:11.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:11 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:25:11.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:11 smithi097 ceph-mon[39416]: osdmap e42: 6 total, 6 up, 6 in 2024-03-21T13:25:11.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:11 smithi097 ceph-mon[39416]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-03-21T13:25:11.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:11 smithi097 ceph-mon[39416]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:25:11.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:11 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T13:25:11.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:11 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T13:25:11.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:11 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:25:14.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:14 smithi097 ceph-mon[30040]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-21T13:25:14.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:14 smithi097 ceph-mon[30040]: from='client.? 172.21.15.144:0/3301589820' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cfca8ffd-b02e-44cd-bd48-55fdb4bd6ae5"}]: dispatch 2024-03-21T13:25:14.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:14 smithi097 ceph-mon[30040]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cfca8ffd-b02e-44cd-bd48-55fdb4bd6ae5"}]: dispatch 2024-03-21T13:25:14.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:14 smithi097 ceph-mon[30040]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cfca8ffd-b02e-44cd-bd48-55fdb4bd6ae5"}]': finished 2024-03-21T13:25:14.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:14 smithi097 ceph-mon[30040]: osdmap e43: 7 total, 6 up, 7 in 2024-03-21T13:25:14.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:14 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T13:25:14.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:14 smithi097 ceph-mon[39416]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-21T13:25:14.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:14 smithi097 ceph-mon[39416]: from='client.? 172.21.15.144:0/3301589820' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cfca8ffd-b02e-44cd-bd48-55fdb4bd6ae5"}]: dispatch 2024-03-21T13:25:14.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:14 smithi097 ceph-mon[39416]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cfca8ffd-b02e-44cd-bd48-55fdb4bd6ae5"}]: dispatch 2024-03-21T13:25:14.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:14 smithi097 ceph-mon[39416]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cfca8ffd-b02e-44cd-bd48-55fdb4bd6ae5"}]': finished 2024-03-21T13:25:14.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:14 smithi097 ceph-mon[39416]: osdmap e43: 7 total, 6 up, 7 in 2024-03-21T13:25:14.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:14 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T13:25:14.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:14 smithi144 ceph-mon[33633]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-21T13:25:14.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:14 smithi144 ceph-mon[33633]: from='client.? 172.21.15.144:0/3301589820' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cfca8ffd-b02e-44cd-bd48-55fdb4bd6ae5"}]: dispatch 2024-03-21T13:25:14.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:14 smithi144 ceph-mon[33633]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cfca8ffd-b02e-44cd-bd48-55fdb4bd6ae5"}]: dispatch 2024-03-21T13:25:14.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:14 smithi144 ceph-mon[33633]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cfca8ffd-b02e-44cd-bd48-55fdb4bd6ae5"}]': finished 2024-03-21T13:25:14.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:14 smithi144 ceph-mon[33633]: osdmap e43: 7 total, 6 up, 7 in 2024-03-21T13:25:14.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:14 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T13:25:15.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:15 smithi097 ceph-mon[30040]: from='client.? 172.21.15.144:0/2374102315' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T13:25:15.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:15 smithi097 ceph-mon[39416]: from='client.? 172.21.15.144:0/2374102315' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T13:25:15.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:15 smithi144 ceph-mon[33633]: from='client.? 172.21.15.144:0/2374102315' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T13:25:16.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:16 smithi097 ceph-mon[30040]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-21T13:25:16.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:16 smithi097 ceph-mon[39416]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-21T13:25:16.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:16 smithi144 ceph-mon[33633]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-21T13:25:18.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:18 smithi144 ceph-mon[33633]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-03-21T13:25:18.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:18 smithi097 ceph-mon[30040]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-03-21T13:25:18.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:18 smithi097 ceph-mon[39416]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-03-21T13:25:20.543 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:20 smithi144 ceph-mon[33633]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-03-21T13:25:20.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:20 smithi097 ceph-mon[30040]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-03-21T13:25:20.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:20 smithi097 ceph-mon[39416]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-03-21T13:25:21.437 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:21 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-21T13:25:21.437 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:21 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:25:21.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:21 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-21T13:25:21.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:21 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:25:21.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:21 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-21T13:25:21.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:21 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:25:22.493 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:22 smithi144 ceph-mon[33633]: Deploying daemon osd.6 on smithi144 2024-03-21T13:25:22.493 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:22 smithi144 ceph-mon[33633]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T13:25:22.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:22 smithi097 ceph-mon[39416]: Deploying daemon osd.6 on smithi144 2024-03-21T13:25:22.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:22 smithi097 ceph-mon[39416]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T13:25:22.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:22 smithi097 ceph-mon[30040]: Deploying daemon osd.6 on smithi144 2024-03-21T13:25:22.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:22 smithi097 ceph-mon[30040]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T13:25:24.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:24 smithi097 ceph-mon[30040]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T13:25:24.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:24 smithi097 ceph-mon[39416]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T13:25:24.593 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:24 smithi144 ceph-mon[33633]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T13:25:26.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:26 smithi097 ceph-mon[39416]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T13:25:26.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:26 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:26.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:26 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:26.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:26 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:25:26.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:26 smithi097 ceph-mon[30040]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T13:25:26.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:26 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:26.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:26 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:26.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:26 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:25:26.576 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:26 smithi144 ceph-mon[33633]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T13:25:26.576 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:26 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:26.577 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:26 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:26.577 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:26 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:25:28.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:28 smithi144 ceph-mon[33633]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T13:25:28.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:28 smithi097 ceph-mon[39416]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T13:25:28.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:28 smithi097 ceph-mon[30040]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T13:25:29.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:29.780+0000 7f8afaffd700 1 -- 172.21.15.144:0/613037422 <== mgr.14152 v2:172.21.15.97:6800/1943212787 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f8b040fe7b0 con 0x7f8af006e210 2024-03-21T13:25:29.781 INFO:teuthology.orchestra.run.smithi144.stdout:Created osd(s) 6 on host 'smithi144' 2024-03-21T13:25:29.783 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:29.782+0000 7f8b0c131700 1 -- 172.21.15.144:0/613037422 >> v2:172.21.15.97:6800/1943212787 conn(0x7f8af006e210 msgr2=0x7f8af00706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:25:29.783 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:29.782+0000 7f8b0c131700 1 --2- 172.21.15.144:0/613037422 >> v2:172.21.15.97:6800/1943212787 conn(0x7f8af006e210 0x7f8af00706d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8b00009fd0 tx=0x7f8b0000b040 comp rx=0 tx=0).stop 2024-03-21T13:25:29.783 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:29.782+0000 7f8b0c131700 1 -- 172.21.15.144:0/613037422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b04102c50 msgr2=0x7f8b04103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:25:29.783 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:29.782+0000 7f8b0c131700 1 --2- 172.21.15.144:0/613037422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b04102c50 0x7f8b04103bd0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8afc00d980 tx=0x7f8afc00dc90 comp rx=0 tx=0).stop 2024-03-21T13:25:29.783 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:29.783+0000 7f8b0c131700 1 -- 172.21.15.144:0/613037422 shutdown_connections 2024-03-21T13:25:29.783 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:29.783+0000 7f8b0c131700 1 --2- 172.21.15.144:0/613037422 >> v2:172.21.15.97:6800/1943212787 conn(0x7f8af006e210 0x7f8af00706d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:25:29.783 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:29.783+0000 7f8b0c131700 1 --2- 172.21.15.144:0/613037422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b04102c50 0x7f8b04103bd0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:25:29.783 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:29.783+0000 7f8b0c131700 1 --2- 172.21.15.144:0/613037422 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8b041010b0 0x7f8b04105040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:25:29.783 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:29.783+0000 7f8b0c131700 1 --2- 172.21.15.144:0/613037422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b041022b0 0x7f8b04103690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:25:29.783 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:29.783+0000 7f8b0c131700 1 -- 172.21.15.144:0/613037422 >> 172.21.15.144:0/613037422 conn(0x7f8b040fc650 msgr2=0x7f8b040fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:25:29.783 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:29.783+0000 7f8b0c131700 1 -- 172.21.15.144:0/613037422 shutdown_connections 2024-03-21T13:25:29.784 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:29.783+0000 7f8b0c131700 1 -- 172.21.15.144:0/613037422 wait complete. 2024-03-21T13:25:30.044 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:29 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:30.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:29 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:30.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:29 smithi144 ceph-mon[33633]: from='osd.6 v2:172.21.15.144:6808/4170414757' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-21T13:25:30.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:29 smithi144 ceph-mon[33633]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-21T13:25:30.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:29 smithi144 ceph-mon[33633]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T13:25:30.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:29 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:30.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:29 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:30.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:29 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:30.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:29 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:30.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:29 smithi097 ceph-mon[30040]: from='osd.6 v2:172.21.15.144:6808/4170414757' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-21T13:25:30.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:29 smithi097 ceph-mon[30040]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-21T13:25:30.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:29 smithi097 ceph-mon[30040]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T13:25:30.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:29 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:30.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:29 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:30.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:29 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:30.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:29 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:30.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:29 smithi097 ceph-mon[39416]: from='osd.6 v2:172.21.15.144:6808/4170414757' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-21T13:25:30.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:29 smithi097 ceph-mon[39416]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-21T13:25:30.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:29 smithi097 ceph-mon[39416]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T13:25:30.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:29 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:30.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:29 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:30.352 DEBUG:teuthology.orchestra.run.smithi144:osd.6> sudo journalctl -f -n 0 -u ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@osd.6.service 2024-03-21T13:25:30.354 INFO:tasks.cephadm:Deploying osd.7 on smithi144 with /dev/vg_nvme/lv_1... 2024-03-21T13:25:30.354 DEBUG:teuthology.orchestra.run.smithi144:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-21T13:25:30.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:30 smithi144 ceph-mon[33633]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-21T13:25:30.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:30 smithi144 ceph-mon[33633]: osdmap e44: 7 total, 6 up, 7 in 2024-03-21T13:25:30.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:30 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T13:25:30.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:30 smithi144 ceph-mon[33633]: from='osd.6 v2:172.21.15.144:6808/4170414757' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-21T13:25:30.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:30 smithi144 ceph-mon[33633]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-21T13:25:31.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:30 smithi097 ceph-mon[30040]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-21T13:25:31.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:30 smithi097 ceph-mon[30040]: osdmap e44: 7 total, 6 up, 7 in 2024-03-21T13:25:31.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:30 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T13:25:31.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:30 smithi097 ceph-mon[30040]: from='osd.6 v2:172.21.15.144:6808/4170414757' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-21T13:25:31.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:30 smithi097 ceph-mon[30040]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-21T13:25:31.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:30 smithi097 ceph-mon[39416]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-21T13:25:31.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:30 smithi097 ceph-mon[39416]: osdmap e44: 7 total, 6 up, 7 in 2024-03-21T13:25:31.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:30 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T13:25:31.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:30 smithi097 ceph-mon[39416]: from='osd.6 v2:172.21.15.144:6808/4170414757' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-21T13:25:31.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:30 smithi097 ceph-mon[39416]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-21T13:25:31.452 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 13:25:31 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T13:25:31.247+0000 7f88fe7e9700 -1 osd.6 0 waiting for initial osdmap 2024-03-21T13:25:31.452 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 13:25:31 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T13:25:31.253+0000 7f88f65d4700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T13:25:31.461 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T13:25:31.968 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:31 smithi144 ceph-mon[33633]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-21T13:25:31.968 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:31 smithi144 ceph-mon[33633]: osdmap e45: 7 total, 6 up, 7 in 2024-03-21T13:25:31.969 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:31 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T13:25:31.969 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:31 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T13:25:31.969 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:31 smithi144 ceph-mon[33633]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T13:25:31.969 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:31 smithi144 ceph-mon[33633]: from='osd.6 ' entity='osd.6' 2024-03-21T13:25:31.969 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:31 smithi144 ceph-mon[33633]: osd.6 v2:172.21.15.144:6808/4170414757 boot 2024-03-21T13:25:31.969 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:31 smithi144 ceph-mon[33633]: osdmap e46: 7 total, 7 up, 7 in 2024-03-21T13:25:31.969 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:31 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T13:25:32.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:31 smithi097 ceph-mon[30040]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-21T13:25:32.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:31 smithi097 ceph-mon[30040]: osdmap e45: 7 total, 6 up, 7 in 2024-03-21T13:25:32.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:31 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T13:25:32.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:31 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T13:25:32.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:31 smithi097 ceph-mon[30040]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T13:25:32.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:31 smithi097 ceph-mon[30040]: from='osd.6 ' entity='osd.6' 2024-03-21T13:25:32.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:31 smithi097 ceph-mon[30040]: osd.6 v2:172.21.15.144:6808/4170414757 boot 2024-03-21T13:25:32.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:31 smithi097 ceph-mon[30040]: osdmap e46: 7 total, 7 up, 7 in 2024-03-21T13:25:32.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:31 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T13:25:32.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:31 smithi097 ceph-mon[39416]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-21T13:25:32.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:31 smithi097 ceph-mon[39416]: osdmap e45: 7 total, 6 up, 7 in 2024-03-21T13:25:32.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:31 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T13:25:32.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:31 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T13:25:32.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:31 smithi097 ceph-mon[39416]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T13:25:32.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:31 smithi097 ceph-mon[39416]: from='osd.6 ' entity='osd.6' 2024-03-21T13:25:32.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:31 smithi097 ceph-mon[39416]: osd.6 v2:172.21.15.144:6808/4170414757 boot 2024-03-21T13:25:32.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:31 smithi097 ceph-mon[39416]: osdmap e46: 7 total, 7 up, 7 in 2024-03-21T13:25:32.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:31 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T13:25:33.094 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:32 smithi144 ceph-mon[33633]: purged_snaps scrub starts 2024-03-21T13:25:33.094 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:32 smithi144 ceph-mon[33633]: purged_snaps scrub ok 2024-03-21T13:25:33.094 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:32 smithi144 ceph-mon[33633]: osdmap e47: 7 total, 7 up, 7 in 2024-03-21T13:25:33.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:32 smithi097 ceph-mon[30040]: purged_snaps scrub starts 2024-03-21T13:25:33.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:32 smithi097 ceph-mon[30040]: purged_snaps scrub ok 2024-03-21T13:25:33.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:32 smithi097 ceph-mon[30040]: osdmap e47: 7 total, 7 up, 7 in 2024-03-21T13:25:33.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:32 smithi097 ceph-mon[39416]: purged_snaps scrub starts 2024-03-21T13:25:33.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:32 smithi097 ceph-mon[39416]: purged_snaps scrub ok 2024-03-21T13:25:33.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:32 smithi097 ceph-mon[39416]: osdmap e47: 7 total, 7 up, 7 in 2024-03-21T13:25:34.340 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-21T13:25:34.340 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-21T13:25:34.340 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10+0 records in 2024-03-21T13:25:34.340 INFO:teuthology.orchestra.run.smithi144.stderr:10+0 records out 2024-03-21T13:25:34.340 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011604 s, 904 MB/s 2024-03-21T13:25:34.340 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping successful for: 2024-03-21T13:25:34.340 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:34 smithi144 ceph-mon[33633]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 461 MiB used, 625 GiB / 626 GiB avail 2024-03-21T13:25:34.340 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:34 smithi144 ceph-mon[33633]: osdmap e48: 7 total, 7 up, 7 in 2024-03-21T13:25:34.341 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:34 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:34.341 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:34 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:34.341 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:34 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:25:34.341 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:34 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:25:34.341 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:34 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:25:34.341 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:34 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:34.341 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:34 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:25:34.341 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:34 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:25:34.341 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:34 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:34.341 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:34 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:25:34.341 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:34 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:25:34.341 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:34 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:25:34.341 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:34 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:34.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:34 smithi097 ceph-mon[39416]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 461 MiB used, 625 GiB / 626 GiB avail 2024-03-21T13:25:34.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:34 smithi097 ceph-mon[39416]: osdmap e48: 7 total, 7 up, 7 in 2024-03-21T13:25:34.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:34 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:34.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:34 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:34.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:34 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:25:34.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:34 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:25:34.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:34 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:25:34.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:34 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:34.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:34 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:25:34.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:34 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:25:34.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:34 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:34.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:34 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:25:34.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:34 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:25:34.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:34 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:25:34.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:34 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:34.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:34 smithi097 ceph-mon[30040]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 461 MiB used, 625 GiB / 626 GiB avail 2024-03-21T13:25:34.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:34 smithi097 ceph-mon[30040]: osdmap e48: 7 total, 7 up, 7 in 2024-03-21T13:25:34.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:34 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:34.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:34 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:34.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:34 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:25:34.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:34 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:25:34.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:34 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:25:34.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:34 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:34.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:34 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:25:34.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:34 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:25:34.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:34 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:34.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:34 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:25:34.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:34 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:25:34.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:34 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:25:34.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:34 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:34.942 DEBUG:teuthology.orchestra.run.smithi144:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi144:vg_nvme/lv_1 2024-03-21T13:25:35.170 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:35 smithi144 ceph-mon[33633]: Detected new or changed devices on smithi144 2024-03-21T13:25:35.170 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:35 smithi144 ceph-mon[33633]: Adjusting osd_memory_target on smithi144 to 5696M 2024-03-21T13:25:35.170 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:35 smithi144 ceph-mon[33633]: osdmap e49: 7 total, 7 up, 7 in 2024-03-21T13:25:35.181 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T13:25:35.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:35 smithi097 ceph-mon[30040]: Detected new or changed devices on smithi144 2024-03-21T13:25:35.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:35 smithi097 ceph-mon[30040]: Adjusting osd_memory_target on smithi144 to 5696M 2024-03-21T13:25:35.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:35 smithi097 ceph-mon[30040]: osdmap e49: 7 total, 7 up, 7 in 2024-03-21T13:25:35.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:35 smithi097 ceph-mon[39416]: Detected new or changed devices on smithi144 2024-03-21T13:25:35.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:35 smithi097 ceph-mon[39416]: Adjusting osd_memory_target on smithi144 to 5696M 2024-03-21T13:25:35.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:35 smithi097 ceph-mon[39416]: osdmap e49: 7 total, 7 up, 7 in 2024-03-21T13:25:36.454 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:36 smithi097 ceph-mon[39416]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 461 MiB used, 625 GiB / 626 GiB avail 2024-03-21T13:25:36.455 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:36 smithi097 ceph-mon[39416]: osdmap e50: 7 total, 7 up, 7 in 2024-03-21T13:25:36.455 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:36 smithi097 ceph-mon[30040]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 461 MiB used, 625 GiB / 626 GiB avail 2024-03-21T13:25:36.455 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:36 smithi097 ceph-mon[30040]: osdmap e50: 7 total, 7 up, 7 in 2024-03-21T13:25:36.475 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:36 smithi144 ceph-mon[33633]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 461 MiB used, 625 GiB / 626 GiB avail 2024-03-21T13:25:36.475 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:36 smithi144 ceph-mon[33633]: osdmap e50: 7 total, 7 up, 7 in 2024-03-21T13:25:36.698 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:36.694+0000 7f89e86fa700 1 -- 172.21.15.144:0/85964002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89e01021b0 msgr2=0x7f89e01025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:25:36.698 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:36.694+0000 7f89e86fa700 1 --2- 172.21.15.144:0/85964002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89e01021b0 0x7f89e01025d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f89d00099c0 tx=0x7f89d0009cd0 comp rx=0 tx=0).stop 2024-03-21T13:25:36.698 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:36.694+0000 7f89e86fa700 1 -- 172.21.15.144:0/85964002 shutdown_connections 2024-03-21T13:25:36.698 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:36.694+0000 7f89e86fa700 1 --2- 172.21.15.144:0/85964002 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f89e0103d50 0x7f89e010a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:25:36.698 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:36.694+0000 7f89e86fa700 1 --2- 172.21.15.144:0/85964002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89e01021b0 0x7f89e01025d0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:25:36.698 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:36.694+0000 7f89e86fa700 1 --2- 172.21.15.144:0/85964002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89e01033b0 0x7f89e0103810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:25:36.698 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:36.694+0000 7f89e86fa700 1 -- 172.21.15.144:0/85964002 >> 172.21.15.144:0/85964002 conn(0x7f89e00fd770 msgr2=0x7f89e00ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:25:36.699 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:36.694+0000 7f89e86fa700 1 -- 172.21.15.144:0/85964002 shutdown_connections 2024-03-21T13:25:36.699 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:36.694+0000 7f89e86fa700 1 -- 172.21.15.144:0/85964002 wait complete. 2024-03-21T13:25:36.699 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:36.695+0000 7f89e86fa700 1 Processor -- start 2024-03-21T13:25:36.699 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:36.695+0000 7f89e86fa700 1 -- start start 2024-03-21T13:25:36.699 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:36.695+0000 7f89e86fa700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89e01021b0 0x7f89e0197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:25:36.699 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:36.695+0000 7f89e86fa700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89e01033b0 0x7f89e0197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:25:36.699 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:36.695+0000 7f89e86fa700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f89e0103d50 0x7f89e019f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:25:36.699 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:36.695+0000 7f89e86fa700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e010d4c0 con 0x7f89e01033b0 2024-03-21T13:25:36.699 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:36.695+0000 7f89e86fa700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e010d340 con 0x7f89e01021b0 2024-03-21T13:25:36.699 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:36.695+0000 7f89e86fa700 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e010d640 con 0x7f89e0103d50 2024-03-21T13:25:36.699 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:36.695+0000 7f89e6496700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89e01021b0 0x7f89e0197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:25:36.699 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:36.695+0000 7f89e6496700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89e01021b0 0x7f89e0197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:56500/0 (socket says 172.21.15.144:56500) 2024-03-21T13:25:36.699 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:36.695+0000 7f89e6496700 1 -- 172.21.15.144:0/2011032765 learned_addr learned my addr 172.21.15.144:0/2011032765 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-21T13:25:36.699 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:36.695+0000 7f89e6c97700 1 --2- 172.21.15.144:0/2011032765 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f89e0103d50 0x7f89e019f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:25:36.700 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:36.695+0000 7f89e5c95700 1 --2- 172.21.15.144:0/2011032765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89e01033b0 0x7f89e0197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:25:36.700 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:36.696+0000 7f89e6496700 1 -- 172.21.15.144:0/2011032765 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f89e0103d50 msgr2=0x7f89e019f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:25:36.700 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:36.696+0000 7f89e6496700 1 --2- 172.21.15.144:0/2011032765 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f89e0103d50 0x7f89e019f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:25:36.700 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:36.696+0000 7f89e6496700 1 -- 172.21.15.144:0/2011032765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89e01033b0 msgr2=0x7f89e0197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:25:36.700 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:36.696+0000 7f89e6496700 1 --2- 172.21.15.144:0/2011032765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89e01033b0 0x7f89e0197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:25:36.700 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:36.696+0000 7f89e6496700 1 -- 172.21.15.144:0/2011032765 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e019f700 con 0x7f89e01021b0 2024-03-21T13:25:36.700 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:36.696+0000 7f89e6c97700 1 --2- 172.21.15.144:0/2011032765 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f89e0103d50 0x7f89e019f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T13:25:36.700 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:36.696+0000 7f89e5c95700 1 --2- 172.21.15.144:0/2011032765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89e01033b0 0x7f89e0197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T13:25:36.700 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:36.696+0000 7f89e6496700 1 --2- 172.21.15.144:0/2011032765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89e01021b0 0x7f89e0197aa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f89d000c010 tx=0x7f89d00047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:25:36.700 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:36.696+0000 7f89d77fe700 1 -- 172.21.15.144:0/2011032765 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d0027070 con 0x7f89e01021b0 2024-03-21T13:25:36.700 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:36.696+0000 7f89d77fe700 1 -- 172.21.15.144:0/2011032765 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f89d0004100 con 0x7f89e01021b0 2024-03-21T13:25:36.700 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:36.696+0000 7f89e86fa700 1 -- 172.21.15.144:0/2011032765 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89e019f990 con 0x7f89e01021b0 2024-03-21T13:25:36.700 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:36.696+0000 7f89d77fe700 1 -- 172.21.15.144:0/2011032765 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d0022dc0 con 0x7f89e01021b0 2024-03-21T13:25:36.700 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:36.696+0000 7f89e86fa700 1 -- 172.21.15.144:0/2011032765 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89e019fde0 con 0x7f89e01021b0 2024-03-21T13:25:36.701 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:36.697+0000 7f89e86fa700 1 -- 172.21.15.144:0/2011032765 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89e0068e50 con 0x7f89e01021b0 2024-03-21T13:25:36.701 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:36.697+0000 7f89d77fe700 1 -- 172.21.15.144:0/2011032765 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 15) v1 ==== 92095+0+0 (secure 0 0 0) 0x7f89d002a070 con 0x7f89e01021b0 2024-03-21T13:25:36.701 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:36.698+0000 7f89d77fe700 1 --2- 172.21.15.144:0/2011032765 >> v2:172.21.15.97:6800/1943212787 conn(0x7f89cc06e210 0x7f89cc0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:25:36.702 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:36.698+0000 7f89d77fe700 1 -- 172.21.15.144:0/2011032765 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 3789+0+0 (secure 0 0 0) 0x7f89d008e8f0 con 0x7f89e01021b0 2024-03-21T13:25:36.702 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:36.698+0000 7f89e5c95700 1 --2- 172.21.15.144:0/2011032765 >> v2:172.21.15.97:6800/1943212787 conn(0x7f89cc06e210 0x7f89cc0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:25:36.702 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:36.698+0000 7f89e5c95700 1 --2- 172.21.15.144:0/2011032765 >> v2:172.21.15.97:6800/1943212787 conn(0x7f89cc06e210 0x7f89cc0706d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f89dc005950 tx=0x7f89dc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:25:36.704 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:36.703+0000 7f89d77fe700 1 -- 172.21.15.144:0/2011032765 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f89d0013070 con 0x7f89e01021b0 2024-03-21T13:25:36.909 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:36.907+0000 7f89e86fa700 1 -- 172.21.15.144:0/2011032765 --> v2:172.21.15.97:6800/1943212787 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f89e00625d0 con 0x7f89cc06e210 2024-03-21T13:25:37.517 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:37 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T13:25:37.517 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:37 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T13:25:37.517 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:37 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:25:37.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:37 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T13:25:37.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:37 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T13:25:37.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:37 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:25:37.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:37 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T13:25:37.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:37 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T13:25:37.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:37 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:25:38.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:38 smithi144 ceph-mon[33633]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:25:38.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:38 smithi144 ceph-mon[33633]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-03-21T13:25:38.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:38 smithi097 ceph-mon[30040]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:25:38.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:38 smithi097 ceph-mon[30040]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-03-21T13:25:38.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:38 smithi097 ceph-mon[39416]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:25:38.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:38 smithi097 ceph-mon[39416]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-03-21T13:25:40.468 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:40 smithi144 ceph-mon[33633]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-21T13:25:40.469 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:40 smithi144 ceph-mon[33633]: from='client.? 172.21.15.144:0/3056385221' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "997e5de8-48df-4348-ab04-22c7c143cf20"}]: dispatch 2024-03-21T13:25:40.469 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:40 smithi144 ceph-mon[33633]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "997e5de8-48df-4348-ab04-22c7c143cf20"}]: dispatch 2024-03-21T13:25:40.469 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:40 smithi144 ceph-mon[33633]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "997e5de8-48df-4348-ab04-22c7c143cf20"}]': finished 2024-03-21T13:25:40.469 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:40 smithi144 ceph-mon[33633]: osdmap e51: 8 total, 7 up, 8 in 2024-03-21T13:25:40.469 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:40 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T13:25:40.469 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:40 smithi144 ceph-mon[33633]: from='client.? 172.21.15.144:0/2255653084' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T13:25:40.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:40 smithi097 ceph-mon[39416]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-21T13:25:40.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:40 smithi097 ceph-mon[39416]: from='client.? 172.21.15.144:0/3056385221' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "997e5de8-48df-4348-ab04-22c7c143cf20"}]: dispatch 2024-03-21T13:25:40.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:40 smithi097 ceph-mon[39416]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "997e5de8-48df-4348-ab04-22c7c143cf20"}]: dispatch 2024-03-21T13:25:40.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:40 smithi097 ceph-mon[39416]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "997e5de8-48df-4348-ab04-22c7c143cf20"}]': finished 2024-03-21T13:25:40.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:40 smithi097 ceph-mon[39416]: osdmap e51: 8 total, 7 up, 8 in 2024-03-21T13:25:40.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:40 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T13:25:40.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:40 smithi097 ceph-mon[39416]: from='client.? 172.21.15.144:0/2255653084' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T13:25:40.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:40 smithi097 ceph-mon[30040]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-21T13:25:40.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:40 smithi097 ceph-mon[30040]: from='client.? 172.21.15.144:0/3056385221' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "997e5de8-48df-4348-ab04-22c7c143cf20"}]: dispatch 2024-03-21T13:25:40.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:40 smithi097 ceph-mon[30040]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "997e5de8-48df-4348-ab04-22c7c143cf20"}]: dispatch 2024-03-21T13:25:40.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:40 smithi097 ceph-mon[30040]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "997e5de8-48df-4348-ab04-22c7c143cf20"}]': finished 2024-03-21T13:25:40.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:40 smithi097 ceph-mon[30040]: osdmap e51: 8 total, 7 up, 8 in 2024-03-21T13:25:40.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:40 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T13:25:40.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:40 smithi097 ceph-mon[30040]: from='client.? 172.21.15.144:0/2255653084' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T13:25:42.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:42 smithi097 ceph-mon[30040]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-03-21T13:25:42.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:42 smithi097 ceph-mon[39416]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-03-21T13:25:42.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:42 smithi144 ceph-mon[33633]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-03-21T13:25:44.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:44 smithi097 ceph-mon[30040]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-21T13:25:44.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:44 smithi097 ceph-mon[39416]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-21T13:25:44.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:44 smithi144 ceph-mon[33633]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-21T13:25:46.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:46 smithi097 ceph-mon[30040]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-21T13:25:46.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:46 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-21T13:25:46.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:46 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:25:46.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:46 smithi097 ceph-mon[39416]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-21T13:25:46.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:46 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-21T13:25:46.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:46 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:25:46.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:46 smithi144 ceph-mon[33633]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-21T13:25:46.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:46 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-21T13:25:46.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:46 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:25:47.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:47 smithi144 ceph-mon[33633]: Deploying daemon osd.7 on smithi144 2024-03-21T13:25:47.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:47 smithi097 ceph-mon[30040]: Deploying daemon osd.7 on smithi144 2024-03-21T13:25:47.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:47 smithi097 ceph-mon[39416]: Deploying daemon osd.7 on smithi144 2024-03-21T13:25:48.306 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:48 smithi144 ceph-mon[33633]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-03-21T13:25:48.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:48 smithi097 ceph-mon[30040]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-03-21T13:25:48.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:48 smithi097 ceph-mon[39416]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-03-21T13:25:50.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:50 smithi144 ceph-mon[33633]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-03-21T13:25:50.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:50 smithi097 ceph-mon[30040]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-03-21T13:25:50.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:50 smithi097 ceph-mon[39416]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-03-21T13:25:52.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:52 smithi144 ceph-mon[33633]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-03-21T13:25:52.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:52 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:52.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:52 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:52.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:52 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:25:52.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:52 smithi097 ceph-mon[30040]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-03-21T13:25:52.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:52 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:52.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:52 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:52.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:52 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:25:52.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:52 smithi097 ceph-mon[39416]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-03-21T13:25:52.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:52 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:52.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:52 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:52.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:52 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:25:54.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:54 smithi144 ceph-mon[33633]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-03-21T13:25:54.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:54 smithi097 ceph-mon[30040]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-03-21T13:25:54.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:54 smithi097 ceph-mon[39416]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-03-21T13:25:55.372 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:55.372+0000 7f89d77fe700 1 -- 172.21.15.144:0/2011032765 <== mgr.14152 v2:172.21.15.97:6800/1943212787 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f89e00625d0 con 0x7f89cc06e210 2024-03-21T13:25:55.372 INFO:teuthology.orchestra.run.smithi144.stdout:Created osd(s) 7 on host 'smithi144' 2024-03-21T13:25:55.374 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:55.374+0000 7f89e86fa700 1 -- 172.21.15.144:0/2011032765 >> v2:172.21.15.97:6800/1943212787 conn(0x7f89cc06e210 msgr2=0x7f89cc0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:25:55.374 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:55.374+0000 7f89e86fa700 1 --2- 172.21.15.144:0/2011032765 >> v2:172.21.15.97:6800/1943212787 conn(0x7f89cc06e210 0x7f89cc0706d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f89dc005950 tx=0x7f89dc0058e0 comp rx=0 tx=0).stop 2024-03-21T13:25:55.374 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:55.374+0000 7f89e86fa700 1 -- 172.21.15.144:0/2011032765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89e01021b0 msgr2=0x7f89e0197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:25:55.374 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:55.374+0000 7f89e86fa700 1 --2- 172.21.15.144:0/2011032765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89e01021b0 0x7f89e0197aa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f89d000c010 tx=0x7f89d00047b0 comp rx=0 tx=0).stop 2024-03-21T13:25:55.375 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:55.374+0000 7f89e86fa700 1 -- 172.21.15.144:0/2011032765 shutdown_connections 2024-03-21T13:25:55.375 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:55.374+0000 7f89e86fa700 1 --2- 172.21.15.144:0/2011032765 >> v2:172.21.15.97:6800/1943212787 conn(0x7f89cc06e210 0x7f89cc0706d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:25:55.375 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:55.374+0000 7f89e86fa700 1 --2- 172.21.15.144:0/2011032765 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f89e0103d50 0x7f89e019f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:25:55.375 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:55.374+0000 7f89e86fa700 1 --2- 172.21.15.144:0/2011032765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89e01021b0 0x7f89e0197aa0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:25:55.375 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:55.374+0000 7f89e86fa700 1 --2- 172.21.15.144:0/2011032765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89e01033b0 0x7f89e0197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:25:55.375 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:55.374+0000 7f89e86fa700 1 -- 172.21.15.144:0/2011032765 >> 172.21.15.144:0/2011032765 conn(0x7f89e00fd770 msgr2=0x7f89e0106720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:25:55.375 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:55.374+0000 7f89e86fa700 1 -- 172.21.15.144:0/2011032765 shutdown_connections 2024-03-21T13:25:55.375 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:25:55.374+0000 7f89e86fa700 1 -- 172.21.15.144:0/2011032765 wait complete. 2024-03-21T13:25:55.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:55 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:55.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:55 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:55.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:55 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:25:55.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:55 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:25:55.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:55 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:55.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:55 smithi144 ceph-mon[33633]: from='osd.7 v2:172.21.15.144:6812/3269572470' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-21T13:25:55.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:55 smithi144 ceph-mon[33633]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-21T13:25:55.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:55 smithi144 ceph-mon[33633]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-03-21T13:25:55.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:55 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:55.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:55 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:55.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:55 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:25:55.968 DEBUG:teuthology.orchestra.run.smithi144:osd.7> sudo journalctl -f -n 0 -u ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@osd.7.service 2024-03-21T13:25:55.970 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-03-21T13:25:55.971 DEBUG:teuthology.orchestra.run.smithi097:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph osd stat -f json 2024-03-21T13:25:56.000 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:55 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:56.000 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:55 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:56.000 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:55 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:25:56.001 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:55 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:25:56.001 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:55 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:56.001 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:55 smithi097 ceph-mon[30040]: from='osd.7 v2:172.21.15.144:6812/3269572470' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-21T13:25:56.001 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:55 smithi097 ceph-mon[30040]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-21T13:25:56.001 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:55 smithi097 ceph-mon[30040]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-03-21T13:25:56.001 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:55 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:56.001 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:55 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:56.001 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:55 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:25:56.001 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:55 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:56.001 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:55 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:56.001 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:55 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:25:56.001 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:55 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:25:56.002 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:55 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:56.002 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:55 smithi097 ceph-mon[39416]: from='osd.7 v2:172.21.15.144:6812/3269572470' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-21T13:25:56.002 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:55 smithi097 ceph-mon[39416]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-21T13:25:56.002 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:55 smithi097 ceph-mon[39416]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-03-21T13:25:56.002 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:55 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:56.002 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:55 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:25:56.002 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:55 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:25:56.222 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T13:25:56.613 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:56 smithi097 ceph-mon[30040]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-21T13:25:56.613 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:56 smithi097 ceph-mon[30040]: osdmap e52: 8 total, 7 up, 8 in 2024-03-21T13:25:56.613 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:56 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T13:25:56.613 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:56 smithi097 ceph-mon[30040]: from='osd.7 v2:172.21.15.144:6812/3269572470' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-21T13:25:56.613 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:56 smithi097 ceph-mon[30040]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-21T13:25:56.613 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:56 smithi097 ceph-mon[39416]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-21T13:25:56.613 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:56 smithi097 ceph-mon[39416]: osdmap e52: 8 total, 7 up, 8 in 2024-03-21T13:25:56.614 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:56 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T13:25:56.614 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:56 smithi097 ceph-mon[39416]: from='osd.7 v2:172.21.15.144:6812/3269572470' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-21T13:25:56.614 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:56 smithi097 ceph-mon[39416]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-21T13:25:56.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:56 smithi144 ceph-mon[33633]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-21T13:25:56.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:56 smithi144 ceph-mon[33633]: osdmap e52: 8 total, 7 up, 8 in 2024-03-21T13:25:56.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:56 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T13:25:56.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:56 smithi144 ceph-mon[33633]: from='osd.7 v2:172.21.15.144:6812/3269572470' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-21T13:25:56.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:56 smithi144 ceph-mon[33633]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-21T13:25:57.141 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 13:25:57 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T13:25:57.022+0000 7fc7718a4700 -1 osd.7 0 waiting for initial osdmap 2024-03-21T13:25:57.141 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 13:25:57 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T13:25:57.029+0000 7fc76968f700 -1 osd.7 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T13:25:57.782 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:57 smithi097 ceph-mon[39416]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-21T13:25:57.783 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:57 smithi097 ceph-mon[39416]: osdmap e53: 8 total, 7 up, 8 in 2024-03-21T13:25:57.783 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:57 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T13:25:57.783 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:57 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T13:25:57.783 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:57 smithi097 ceph-mon[39416]: from='osd.7 ' entity='osd.7' 2024-03-21T13:25:57.783 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:57 smithi097 ceph-mon[39416]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-03-21T13:25:57.783 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:57 smithi097 ceph-mon[39416]: osd.7 v2:172.21.15.144:6812/3269572470 boot 2024-03-21T13:25:57.783 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:57 smithi097 ceph-mon[39416]: osdmap e54: 8 total, 8 up, 8 in 2024-03-21T13:25:57.783 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:57 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T13:25:57.783 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:57 smithi097 ceph-mon[30040]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-21T13:25:57.783 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:57 smithi097 ceph-mon[30040]: osdmap e53: 8 total, 7 up, 8 in 2024-03-21T13:25:57.784 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:57 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T13:25:57.784 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:57 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T13:25:57.784 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:57 smithi097 ceph-mon[30040]: from='osd.7 ' entity='osd.7' 2024-03-21T13:25:57.784 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:57 smithi097 ceph-mon[30040]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-03-21T13:25:57.784 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:57 smithi097 ceph-mon[30040]: osd.7 v2:172.21.15.144:6812/3269572470 boot 2024-03-21T13:25:57.784 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:57 smithi097 ceph-mon[30040]: osdmap e54: 8 total, 8 up, 8 in 2024-03-21T13:25:57.784 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:57 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T13:25:57.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:57 smithi144 ceph-mon[33633]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-21T13:25:57.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:57 smithi144 ceph-mon[33633]: osdmap e53: 8 total, 7 up, 8 in 2024-03-21T13:25:57.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:57 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T13:25:57.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:57 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T13:25:57.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:57 smithi144 ceph-mon[33633]: from='osd.7 ' entity='osd.7' 2024-03-21T13:25:57.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:57 smithi144 ceph-mon[33633]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-03-21T13:25:57.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:57 smithi144 ceph-mon[33633]: osd.7 v2:172.21.15.144:6812/3269572470 boot 2024-03-21T13:25:57.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:57 smithi144 ceph-mon[33633]: osdmap e54: 8 total, 8 up, 8 in 2024-03-21T13:25:57.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:57 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T13:25:58.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.005+0000 7f7c54c54700 1 -- 172.21.15.97:0/2333023850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c501034a0 msgr2=0x7f7c50103900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:25:58.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.005+0000 7f7c54c54700 1 --2- 172.21.15.97:0/2333023850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c501034a0 0x7f7c50103900 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7c44009990 tx=0x7f7c44009ca0 comp rx=0 tx=0).stop 2024-03-21T13:25:58.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.005+0000 7f7c54c54700 1 -- 172.21.15.97:0/2333023850 shutdown_connections 2024-03-21T13:25:58.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.005+0000 7f7c54c54700 1 --2- 172.21.15.97:0/2333023850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c50103e40 0x7f7c5010a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:25:58.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.005+0000 7f7c54c54700 1 --2- 172.21.15.97:0/2333023850 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7c501004c0 0x7f7c501008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:25:58.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.005+0000 7f7c54c54700 1 --2- 172.21.15.97:0/2333023850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c501034a0 0x7f7c50103900 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:25:58.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.005+0000 7f7c54c54700 1 -- 172.21.15.97:0/2333023850 >> 172.21.15.97:0/2333023850 conn(0x7f7c500fbe20 msgr2=0x7f7c500fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:25:58.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.005+0000 7f7c54c54700 1 -- 172.21.15.97:0/2333023850 shutdown_connections 2024-03-21T13:25:58.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.005+0000 7f7c54c54700 1 -- 172.21.15.97:0/2333023850 wait complete. 2024-03-21T13:25:58.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.006+0000 7f7c54c54700 1 Processor -- start 2024-03-21T13:25:58.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.006+0000 7f7c54c54700 1 -- start start 2024-03-21T13:25:58.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.006+0000 7f7c54c54700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c501004c0 0x7f7c50197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:25:58.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.006+0000 7f7c54c54700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c501034a0 0x7f7c50197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:25:58.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.006+0000 7f7c54c54700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7c50103e40 0x7f7c5019f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:25:58.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.006+0000 7f7c54c54700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5010d520 con 0x7f7c501004c0 2024-03-21T13:25:58.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.006+0000 7f7c54c54700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5010d3a0 con 0x7f7c501034a0 2024-03-21T13:25:58.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.006+0000 7f7c54c54700 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5010d6a0 con 0x7f7c50103e40 2024-03-21T13:25:58.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.006+0000 7f7c4e59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c501004c0 0x7f7c50197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:25:58.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.006+0000 7f7c4e59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c501004c0 0x7f7c50197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36940/0 (socket says 172.21.15.97:36940) 2024-03-21T13:25:58.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.006+0000 7f7c4e59c700 1 -- 172.21.15.97:0/621637173 learned_addr learned my addr 172.21.15.97:0/621637173 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:25:58.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.006+0000 7f7c4ed9d700 1 --2- 172.21.15.97:0/621637173 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7c50103e40 0x7f7c5019f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:25:58.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.006+0000 7f7c4dd9b700 1 --2- 172.21.15.97:0/621637173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c501034a0 0x7f7c50197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:25:58.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.007+0000 7f7c4e59c700 1 -- 172.21.15.97:0/621637173 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7c50103e40 msgr2=0x7f7c5019f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:25:58.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.007+0000 7f7c4e59c700 1 --2- 172.21.15.97:0/621637173 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7c50103e40 0x7f7c5019f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:25:58.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.007+0000 7f7c4e59c700 1 -- 172.21.15.97:0/621637173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c501034a0 msgr2=0x7f7c50197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:25:58.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.007+0000 7f7c4e59c700 1 --2- 172.21.15.97:0/621637173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c501034a0 0x7f7c50197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:25:58.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.007+0000 7f7c4e59c700 1 -- 172.21.15.97:0/621637173 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c5019f6d0 con 0x7f7c501004c0 2024-03-21T13:25:58.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.007+0000 7f7c4e59c700 1 --2- 172.21.15.97:0/621637173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c501004c0 0x7f7c50197a70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7c48009f40 tx=0x7f7c4800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:25:58.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.007+0000 7f7c3f7fe700 1 -- 172.21.15.97:0/621637173 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c480092e0 con 0x7f7c501004c0 2024-03-21T13:25:58.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.007+0000 7f7c54c54700 1 -- 172.21.15.97:0/621637173 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c5019f960 con 0x7f7c501004c0 2024-03-21T13:25:58.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.007+0000 7f7c3f7fe700 1 -- 172.21.15.97:0/621637173 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f7c4800c9c0 con 0x7f7c501004c0 2024-03-21T13:25:58.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.007+0000 7f7c3f7fe700 1 -- 172.21.15.97:0/621637173 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c48004430 con 0x7f7c501004c0 2024-03-21T13:25:58.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.007+0000 7f7c54c54700 1 -- 172.21.15.97:0/621637173 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c5019fea0 con 0x7f7c501004c0 2024-03-21T13:25:58.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.008+0000 7f7c54c54700 1 -- 172.21.15.97:0/621637173 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c50068e50 con 0x7f7c501004c0 2024-03-21T13:25:58.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.008+0000 7f7c3f7fe700 1 -- 172.21.15.97:0/621637173 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 15) v1 ==== 92095+0+0 (secure 0 0 0) 0x7f7c4801b030 con 0x7f7c501004c0 2024-03-21T13:25:58.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.009+0000 7f7c3f7fe700 1 --2- 172.21.15.97:0/621637173 >> v2:172.21.15.97:6800/1943212787 conn(0x7f7c3806e210 0x7f7c380706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:25:58.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.009+0000 7f7c4dd9b700 1 --2- 172.21.15.97:0/621637173 >> v2:172.21.15.97:6800/1943212787 conn(0x7f7c3806e210 0x7f7c380706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:25:58.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.009+0000 7f7c3f7fe700 1 -- 172.21.15.97:0/621637173 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 4114+0+0 (secure 0 0 0) 0x7f7c4808d960 con 0x7f7c501004c0 2024-03-21T13:25:58.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.009+0000 7f7c4dd9b700 1 --2- 172.21.15.97:0/621637173 >> v2:172.21.15.97:6800/1943212787 conn(0x7f7c3806e210 0x7f7c380706d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7c4400b7c0 tx=0x7f7c44005ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:25:58.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.014+0000 7f7c3f7fe700 1 -- 172.21.15.97:0/621637173 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7c48022020 con 0x7f7c501004c0 2024-03-21T13:25:58.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.215+0000 7f7c54c54700 1 -- 172.21.15.97:0/621637173 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f7c5002da30 con 0x7f7c501004c0 2024-03-21T13:25:58.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.215+0000 7f7c3f7fe700 1 -- 172.21.15.97:0/621637173 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v55) v1 ==== 74+0+130 (secure 0 0 0) 0x7f7c48010040 con 0x7f7c501004c0 2024-03-21T13:25:58.217 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:25:58.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.218+0000 7f7c54c54700 1 -- 172.21.15.97:0/621637173 >> v2:172.21.15.97:6800/1943212787 conn(0x7f7c3806e210 msgr2=0x7f7c380706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:25:58.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.218+0000 7f7c54c54700 1 --2- 172.21.15.97:0/621637173 >> v2:172.21.15.97:6800/1943212787 conn(0x7f7c3806e210 0x7f7c380706d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7c4400b7c0 tx=0x7f7c44005ad0 comp rx=0 tx=0).stop 2024-03-21T13:25:58.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.218+0000 7f7c54c54700 1 -- 172.21.15.97:0/621637173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c501004c0 msgr2=0x7f7c50197a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:25:58.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.218+0000 7f7c54c54700 1 --2- 172.21.15.97:0/621637173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c501004c0 0x7f7c50197a70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7c48009f40 tx=0x7f7c4800c3a0 comp rx=0 tx=0).stop 2024-03-21T13:25:58.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.218+0000 7f7c54c54700 1 -- 172.21.15.97:0/621637173 shutdown_connections 2024-03-21T13:25:58.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.218+0000 7f7c54c54700 1 --2- 172.21.15.97:0/621637173 >> v2:172.21.15.97:6800/1943212787 conn(0x7f7c3806e210 0x7f7c380706d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:25:58.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.218+0000 7f7c54c54700 1 --2- 172.21.15.97:0/621637173 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7c50103e40 0x7f7c5019f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:25:58.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.218+0000 7f7c54c54700 1 --2- 172.21.15.97:0/621637173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c501004c0 0x7f7c50197a70 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:25:58.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.218+0000 7f7c54c54700 1 --2- 172.21.15.97:0/621637173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c501034a0 0x7f7c50197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:25:58.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.218+0000 7f7c54c54700 1 -- 172.21.15.97:0/621637173 >> 172.21.15.97:0/621637173 conn(0x7f7c500fbe20 msgr2=0x7f7c501067a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:25:58.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.218+0000 7f7c54c54700 1 -- 172.21.15.97:0/621637173 shutdown_connections 2024-03-21T13:25:58.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:25:58.218+0000 7f7c54c54700 1 -- 172.21.15.97:0/621637173 wait complete. 2024-03-21T13:25:58.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:58 smithi144 ceph-mon[33633]: purged_snaps scrub starts 2024-03-21T13:25:58.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:58 smithi144 ceph-mon[33633]: purged_snaps scrub ok 2024-03-21T13:25:58.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:58 smithi144 ceph-mon[33633]: osdmap e55: 8 total, 8 up, 8 in 2024-03-21T13:25:58.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:58 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/621637173' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-21T13:25:58.927 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:58 smithi097 ceph-mon[30040]: purged_snaps scrub starts 2024-03-21T13:25:58.927 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:58 smithi097 ceph-mon[30040]: purged_snaps scrub ok 2024-03-21T13:25:58.927 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:58 smithi097 ceph-mon[30040]: osdmap e55: 8 total, 8 up, 8 in 2024-03-21T13:25:58.927 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:58 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/621637173' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-21T13:25:58.927 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:58 smithi097 ceph-mon[39416]: purged_snaps scrub starts 2024-03-21T13:25:58.927 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:58 smithi097 ceph-mon[39416]: purged_snaps scrub ok 2024-03-21T13:25:58.928 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:58 smithi097 ceph-mon[39416]: osdmap e55: 8 total, 8 up, 8 in 2024-03-21T13:25:58.928 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:58 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/621637173' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-21T13:25:58.954 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":55,"num_osds":8,"num_up_osds":8,"osd_up_since":1711027557,"num_in_osds":8,"osd_in_since":1711027539,"num_remapped_pgs":1} 2024-03-21T13:25:58.957 DEBUG:teuthology.orchestra.run.smithi097:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-21T13:25:59.233 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T13:26:00.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:59 smithi097 ceph-mon[30040]: Detected new or changed devices on smithi144 2024-03-21T13:26:00.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:59 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:00.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:59 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:00.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:59 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:26:00.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:59 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:26:00.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:59 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:26:00.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:59 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:26:00.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:59 smithi097 ceph-mon[30040]: Adjusting osd_memory_target on smithi144 to 4272M 2024-03-21T13:26:00.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:59 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:00.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:59 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:26:00.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:59 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:26:00.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:59 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:00.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:59 smithi097 ceph-mon[30040]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 484 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:26:00.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:25:59 smithi097 ceph-mon[30040]: osdmap e56: 8 total, 8 up, 8 in 2024-03-21T13:26:00.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:59 smithi097 ceph-mon[39416]: Detected new or changed devices on smithi144 2024-03-21T13:26:00.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:59 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:00.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:59 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:00.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:59 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:26:00.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:59 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:26:00.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:59 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:26:00.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:59 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:26:00.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:59 smithi097 ceph-mon[39416]: Adjusting osd_memory_target on smithi144 to 4272M 2024-03-21T13:26:00.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:59 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:00.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:59 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:26:00.306 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:59 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:26:00.306 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:59 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:00.306 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:59 smithi097 ceph-mon[39416]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 484 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:26:00.306 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:25:59 smithi097 ceph-mon[39416]: osdmap e56: 8 total, 8 up, 8 in 2024-03-21T13:26:00.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:59 smithi144 ceph-mon[33633]: Detected new or changed devices on smithi144 2024-03-21T13:26:00.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:59 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:00.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:59 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:00.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:59 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:26:00.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:59 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:26:00.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:59 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:26:00.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:59 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:26:00.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:59 smithi144 ceph-mon[33633]: Adjusting osd_memory_target on smithi144 to 4272M 2024-03-21T13:26:00.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:59 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:00.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:59 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:26:00.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:59 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:26:00.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:59 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:00.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:59 smithi144 ceph-mon[33633]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 484 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:26:00.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:25:59 smithi144 ceph-mon[33633]: osdmap e56: 8 total, 8 up, 8 in 2024-03-21T13:26:00.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:00.901+0000 7f9f0e7f2700 1 -- 172.21.15.97:0/352839806 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9f081070d0 msgr2=0x7f9f081074b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:00.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:00.901+0000 7f9f0e7f2700 1 --2- 172.21.15.97:0/352839806 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9f081070d0 0x7f9f081074b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9efc009990 tx=0x7f9efc009ca0 comp rx=0 tx=0).stop 2024-03-21T13:26:00.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:00.901+0000 7f9f0e7f2700 1 -- 172.21.15.97:0/352839806 shutdown_connections 2024-03-21T13:26:00.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:00.901+0000 7f9f0e7f2700 1 --2- 172.21.15.97:0/352839806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f08101a30 0x7f9f0810df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:00.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:00.901+0000 7f9f0e7f2700 1 --2- 172.21.15.97:0/352839806 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9f081070d0 0x7f9f081074b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:00.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:00.901+0000 7f9f0e7f2700 1 --2- 172.21.15.97:0/352839806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f08101090 0x7f9f081014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:00.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:00.901+0000 7f9f0e7f2700 1 -- 172.21.15.97:0/352839806 >> 172.21.15.97:0/352839806 conn(0x7f9f08077fc0 msgr2=0x7f9f080783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:26:00.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:00.901+0000 7f9f0e7f2700 1 -- 172.21.15.97:0/352839806 shutdown_connections 2024-03-21T13:26:00.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:00.901+0000 7f9f0e7f2700 1 -- 172.21.15.97:0/352839806 wait complete. 2024-03-21T13:26:00.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:00.902+0000 7f9f0e7f2700 1 Processor -- start 2024-03-21T13:26:00.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:00.902+0000 7f9f0e7f2700 1 -- start start 2024-03-21T13:26:00.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:00.902+0000 7f9f0e7f2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f08101090 0x7f9f08198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:00.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:00.902+0000 7f9f0e7f2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f08101a30 0x7f9f081985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:00.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:00.902+0000 7f9f0e7f2700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9f081070d0 0x7f9f0819c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:00.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:00.902+0000 7f9f0e7f2700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f08110a40 con 0x7f9f08101090 2024-03-21T13:26:00.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:00.902+0000 7f9f0e7f2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f081108c0 con 0x7f9f08101a30 2024-03-21T13:26:00.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:00.902+0000 7f9f0e7f2700 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f08110bc0 con 0x7f9f081070d0 2024-03-21T13:26:00.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:00.902+0000 7f9f0cd8f700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9f081070d0 0x7f9f0819c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:00.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:00.902+0000 7f9f07fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f08101090 0x7f9f08198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:00.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:00.902+0000 7f9f0cd8f700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9f081070d0 0x7f9f0819c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.97:60854/0 (socket says 172.21.15.97:60854) 2024-03-21T13:26:00.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:00.902+0000 7f9f0cd8f700 1 -- 172.21.15.97:0/1015746528 learned_addr learned my addr 172.21.15.97:0/1015746528 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:26:00.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:00.902+0000 7f9f07fff700 1 --2- 172.21.15.97:0/1015746528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f08101090 0x7f9f08198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42272/0 (socket says 172.21.15.97:42272) 2024-03-21T13:26:00.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:00.902+0000 7f9f077fe700 1 --2- 172.21.15.97:0/1015746528 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f08101a30 0x7f9f081985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:00.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:00.902+0000 7f9f0cd8f700 1 -- 172.21.15.97:0/1015746528 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f08101a30 msgr2=0x7f9f081985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:00.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:00.902+0000 7f9f0cd8f700 1 --2- 172.21.15.97:0/1015746528 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f08101a30 0x7f9f081985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:00.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:00.902+0000 7f9f0cd8f700 1 -- 172.21.15.97:0/1015746528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f08101090 msgr2=0x7f9f08198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:00.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:00.902+0000 7f9f0cd8f700 1 --2- 172.21.15.97:0/1015746528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f08101090 0x7f9f08198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:00.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:00.902+0000 7f9f0cd8f700 1 -- 172.21.15.97:0/1015746528 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f0819d0a0 con 0x7f9f081070d0 2024-03-21T13:26:00.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:00.902+0000 7f9f077fe700 1 --2- 172.21.15.97:0/1015746528 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f08101a30 0x7f9f081985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T13:26:00.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:00.902+0000 7f9f07fff700 1 --2- 172.21.15.97:0/1015746528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f08101090 0x7f9f08198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T13:26:00.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:00.903+0000 7f9f0cd8f700 1 --2- 172.21.15.97:0/1015746528 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9f081070d0 0x7f9f0819c960 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9f0000c370 tx=0x7f9f0000c680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:26:00.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:00.903+0000 7f9f057fa700 1 -- 172.21.15.97:0/1015746528 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0000ce20 con 0x7f9f081070d0 2024-03-21T13:26:00.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:00.903+0000 7f9f057fa700 1 -- 172.21.15.97:0/1015746528 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f9f000090d0 con 0x7f9f081070d0 2024-03-21T13:26:00.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:00.903+0000 7f9f0e7f2700 1 -- 172.21.15.97:0/1015746528 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f0819d330 con 0x7f9f081070d0 2024-03-21T13:26:00.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:00.903+0000 7f9f057fa700 1 -- 172.21.15.97:0/1015746528 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f00004780 con 0x7f9f081070d0 2024-03-21T13:26:00.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:00.903+0000 7f9f0e7f2700 1 -- 172.21.15.97:0/1015746528 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f081a4c10 con 0x7f9f081070d0 2024-03-21T13:26:00.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:00.904+0000 7f9f0e7f2700 1 -- 172.21.15.97:0/1015746528 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f08068e50 con 0x7f9f081070d0 2024-03-21T13:26:00.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:00.904+0000 7f9f057fa700 1 -- 172.21.15.97:0/1015746528 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 15) v1 ==== 92095+0+0 (secure 0 0 0) 0x7f9f0001a020 con 0x7f9f081070d0 2024-03-21T13:26:00.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:00.905+0000 7f9f057fa700 1 --2- 172.21.15.97:0/1015746528 >> v2:172.21.15.97:6800/1943212787 conn(0x7f9ef006e210 0x7f9ef00706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:00.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:00.905+0000 7f9f07fff700 1 --2- 172.21.15.97:0/1015746528 >> v2:172.21.15.97:6800/1943212787 conn(0x7f9ef006e210 0x7f9ef00706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:00.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:00.905+0000 7f9f057fa700 1 -- 172.21.15.97:0/1015746528 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 4081+0+0 (secure 0 0 0) 0x7f9f0008d960 con 0x7f9f081070d0 2024-03-21T13:26:00.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:00.905+0000 7f9f07fff700 1 --2- 172.21.15.97:0/1015746528 >> v2:172.21.15.97:6800/1943212787 conn(0x7f9ef006e210 0x7f9ef00706d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9efc0098e0 tx=0x7f9efc005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:26:00.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:00.909+0000 7f9f057fa700 1 -- 172.21.15.97:0/1015746528 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9f00010040 con 0x7f9f081070d0 2024-03-21T13:26:01.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:01.106+0000 7f9f0e7f2700 1 -- 172.21.15.97:0/1015746528 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9f081a4de0 con 0x7f9f081070d0 2024-03-21T13:26:01.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:01.107+0000 7f9f057fa700 1 -- 172.21.15.97:0/1015746528 <== mon.2 v2:172.21.15.97:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+11493 (secure 0 0 0) 0x7f9f0005ba90 con 0x7f9f081070d0 2024-03-21T13:26:01.107 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:26:01.107 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":57,"fsid":"a5dd1f32-e785-11ee-95cd-87774f69a715","created":"2024-03-21T13:20:36.782185+0000","modified":"2024-03-21T13:26:00.164076+0000","last_up_change":"2024-03-21T13:25:57.160251+0000","last_in_change":"2024-03-21T13:25:39.485525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T13:23:51.415792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da68a71b-2182-4471-a6f8-4aa689009b20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6801","nonce":2691001097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2691001097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2691001097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2691001097}]},"public_addr":"172.21.15.97:6801/2691001097","cluster_addr":"172.21.15.97:6802/2691001097","heartbeat_back_addr":"172.21.15.97:6804/2691001097","heartbeat_front_addr":"172.21.15.97:6803/2691001097","state":["exists","up"]},{"osd":1,"uuid":"19b99056-a65c-47ec-9392-cc0d40b3b399","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":287967678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":287967678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":287967678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6807","nonce":287967678}]},"public_addr":"172.21.15.97:6805/287967678","cluster_addr":"172.21.15.97:6806/287967678","heartbeat_back_addr":"172.21.15.97:6808/287967678","heartbeat_front_addr":"172.21.15.97:6807/287967678","state":["exists","up"]},{"osd":2,"uuid":"f4d2f5c6-ac47-4a1d-a409-2bb884b6772f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":2953470250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2953470250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2953470250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2953470250}]},"public_addr":"172.21.15.97:6809/2953470250","cluster_addr":"172.21.15.97:6810/2953470250","heartbeat_back_addr":"172.21.15.97:6812/2953470250","heartbeat_front_addr":"172.21.15.97:6811/2953470250","state":["exists","up"]},{"osd":3,"uuid":"3ecf8a17-0ada-41e4-9740-b9f48ee14c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1678932102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":1678932102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1678932102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":1678932102}]},"public_addr":"172.21.15.97:6813/1678932102","cluster_addr":"172.21.15.97:6814/1678932102","heartbeat_back_addr":"172.21.15.97:6816/1678932102","heartbeat_front_addr":"172.21.15.97:6815/1678932102","state":["exists","up"]},{"osd":4,"uuid":"52ca17a1-0407-4fb1-994b-a382d3c46879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":913882960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6801","nonce":913882960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6803","nonce":913882960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":913882960}]},"public_addr":"172.21.15.144:6800/913882960","cluster_addr":"172.21.15.144:6801/913882960","heartbeat_back_addr":"172.21.15.144:6803/913882960","heartbeat_front_addr":"172.21.15.144:6802/913882960","state":["exists","up"]},{"osd":5,"uuid":"f3f7f8a2-d40b-4866-8dad-310dca09941c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2553466741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6805","nonce":2553466741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6807","nonce":2553466741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2553466741}]},"public_addr":"172.21.15.144:6804/2553466741","cluster_addr":"172.21.15.144:6805/2553466741","heartbeat_back_addr":"172.21.15.144:6807/2553466741","heartbeat_front_addr":"172.21.15.144:6806/2553466741","state":["exists","up"]},{"osd":6,"uuid":"cfca8ffd-b02e-44cd-bd48-55fdb4bd6ae5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":4170414757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4170414757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6811","nonce":4170414757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":4170414757}]},"public_addr":"172.21.15.144:6808/4170414757","cluster_addr":"172.21.15.144:6809/4170414757","heartbeat_back_addr":"172.21.15.144:6811/4170414757","heartbeat_front_addr":"172.21.15.144:6810/4170414757","state":["exists","up"]},{"osd":7,"uuid":"997e5de8-48df-4348-ab04-22c7c143cf20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3269572470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":3269572470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6815","nonce":3269572470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3269572470}]},"public_addr":"172.21.15.144:6812/3269572470","cluster_addr":"172.21.15.144:6813/3269572470","heartbeat_back_addr":"172.21.15.144:6815/3269572470","heartbeat_front_addr":"172.21.15.144:6814/3269572470","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T13:22:56.859160+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T13:23:23.500177+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T13:23:49.429654+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T13:24:15.416871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T13:24:39.114777+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T13:25:04.388302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T13:25:30.020027+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T13:25:55.809198+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/938326700":"2024-03-22T13:21:31.070127+0000","172.21.15.97:0/930977130":"2024-03-22T13:21:31.070127+0000","172.21.15.97:0/2037905570":"2024-03-22T13:21:04.170875+0000","172.21.15.97:0/865729378":"2024-03-22T13:21:04.170875+0000","172.21.15.97:6800/2447843320":"2024-03-22T13:21:04.170875+0000","172.21.15.97:0/1523063805":"2024-03-22T13:21:04.170875+0000","172.21.15.97:6800/4140723216":"2024-03-22T13:21:31.070127+0000","172.21.15.97:0/652843111":"2024-03-22T13:21:31.070127+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T13:26:01.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:01.109+0000 7f9f0e7f2700 1 -- 172.21.15.97:0/1015746528 >> v2:172.21.15.97:6800/1943212787 conn(0x7f9ef006e210 msgr2=0x7f9ef00706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:01.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:01.109+0000 7f9f0e7f2700 1 --2- 172.21.15.97:0/1015746528 >> v2:172.21.15.97:6800/1943212787 conn(0x7f9ef006e210 0x7f9ef00706d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9efc0098e0 tx=0x7f9efc005960 comp rx=0 tx=0).stop 2024-03-21T13:26:01.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:01.109+0000 7f9f0e7f2700 1 -- 172.21.15.97:0/1015746528 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9f081070d0 msgr2=0x7f9f0819c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:01.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:01.109+0000 7f9f0e7f2700 1 --2- 172.21.15.97:0/1015746528 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9f081070d0 0x7f9f0819c960 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9f0000c370 tx=0x7f9f0000c680 comp rx=0 tx=0).stop 2024-03-21T13:26:01.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:01.109+0000 7f9f0e7f2700 1 -- 172.21.15.97:0/1015746528 shutdown_connections 2024-03-21T13:26:01.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:01.109+0000 7f9f0e7f2700 1 --2- 172.21.15.97:0/1015746528 >> v2:172.21.15.97:6800/1943212787 conn(0x7f9ef006e210 0x7f9ef00706d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:01.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:01.109+0000 7f9f0e7f2700 1 --2- 172.21.15.97:0/1015746528 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9f081070d0 0x7f9f0819c960 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:01.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:01.109+0000 7f9f0e7f2700 1 --2- 172.21.15.97:0/1015746528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f08101090 0x7f9f08198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:01.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:01.109+0000 7f9f0e7f2700 1 --2- 172.21.15.97:0/1015746528 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f08101a30 0x7f9f081985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:01.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:01.109+0000 7f9f0e7f2700 1 -- 172.21.15.97:0/1015746528 >> 172.21.15.97:0/1015746528 conn(0x7f9f08077fc0 msgr2=0x7f9f080ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:26:01.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:01.109+0000 7f9f0e7f2700 1 -- 172.21.15.97:0/1015746528 shutdown_connections 2024-03-21T13:26:01.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:01.109+0000 7f9f0e7f2700 1 -- 172.21.15.97:0/1015746528 wait complete. 2024-03-21T13:26:01.418 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:01 smithi097 ceph-mon[39416]: osdmap e57: 8 total, 8 up, 8 in 2024-03-21T13:26:01.418 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:01 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1015746528' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T13:26:01.418 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:01 smithi097 ceph-mon[30040]: osdmap e57: 8 total, 8 up, 8 in 2024-03-21T13:26:01.418 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:01 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1015746528' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T13:26:01.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:01 smithi144 ceph-mon[33633]: osdmap e57: 8 total, 8 up, 8 in 2024-03-21T13:26:01.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:01 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1015746528' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T13:26:01.772 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T13:23:51.415792+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-21T13:26:01.772 DEBUG:teuthology.orchestra.run.smithi097:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-21T13:26:02.011 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T13:26:02.384 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:02 smithi097 ceph-mon[39416]: pgmap v182: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 484 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-21T13:26:02.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:02 smithi097 ceph-mon[39416]: osdmap e58: 8 total, 8 up, 8 in 2024-03-21T13:26:02.385 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:02 smithi097 ceph-mon[30040]: pgmap v182: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 484 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-21T13:26:02.385 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:02 smithi097 ceph-mon[30040]: osdmap e58: 8 total, 8 up, 8 in 2024-03-21T13:26:02.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:02 smithi144 ceph-mon[33633]: pgmap v182: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 484 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-21T13:26:02.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:02 smithi144 ceph-mon[33633]: osdmap e58: 8 total, 8 up, 8 in 2024-03-21T13:26:03.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.667+0000 7fa0e400b700 1 -- 172.21.15.97:0/1858083213 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa0dc1021e0 msgr2=0x7fa0dc102660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:03.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.667+0000 7fa0e400b700 1 --2- 172.21.15.97:0/1858083213 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa0dc1021e0 0x7fa0dc102660 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc009990 tx=0x7fa0cc009ca0 comp rx=0 tx=0).stop 2024-03-21T13:26:03.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.667+0000 7fa0e400b700 1 -- 172.21.15.97:0/1858083213 shutdown_connections 2024-03-21T13:26:03.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.667+0000 7fa0e400b700 1 --2- 172.21.15.97:0/1858083213 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0dc102ba0 0x7fa0dc10efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:03.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.667+0000 7fa0e400b700 1 --2- 172.21.15.97:0/1858083213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa0dc1081c0 0x7fa0dc1085a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:03.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.667+0000 7fa0e400b700 1 --2- 172.21.15.97:0/1858083213 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa0dc1021e0 0x7fa0dc102660 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:03.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.667+0000 7fa0e400b700 1 -- 172.21.15.97:0/1858083213 >> 172.21.15.97:0/1858083213 conn(0x7fa0dc0792c0 msgr2=0x7fa0dc0796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:26:03.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.667+0000 7fa0e400b700 1 -- 172.21.15.97:0/1858083213 shutdown_connections 2024-03-21T13:26:03.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.667+0000 7fa0e400b700 1 -- 172.21.15.97:0/1858083213 wait complete. 2024-03-21T13:26:03.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.668+0000 7fa0e400b700 1 Processor -- start 2024-03-21T13:26:03.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.668+0000 7fa0e400b700 1 -- start start 2024-03-21T13:26:03.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.668+0000 7fa0e400b700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa0dc1021e0 0x7fa0dc074490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:03.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.668+0000 7fa0e400b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0dc102ba0 0x7fa0dc0749d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:03.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.668+0000 7fa0e400b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa0dc1081c0 0x7fa0dc074f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:03.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.668+0000 7fa0e400b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0dc111bc0 con 0x7fa0dc1081c0 2024-03-21T13:26:03.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.668+0000 7fa0e400b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0dc111a40 con 0x7fa0dc102ba0 2024-03-21T13:26:03.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.668+0000 7fa0e400b700 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0dc111d40 con 0x7fa0dc1021e0 2024-03-21T13:26:03.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.668+0000 7fa0e1da7700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa0dc1021e0 0x7fa0dc074490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:03.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.668+0000 7fa0e25a8700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa0dc1081c0 0x7fa0dc074f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:03.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.669+0000 7fa0e1da7700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa0dc1021e0 0x7fa0dc074490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.97:60878/0 (socket says 172.21.15.97:60878) 2024-03-21T13:26:03.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.669+0000 7fa0e1da7700 1 -- 172.21.15.97:0/3344129365 learned_addr learned my addr 172.21.15.97:0/3344129365 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:26:03.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.669+0000 7fa0e15a6700 1 --2- 172.21.15.97:0/3344129365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0dc102ba0 0x7fa0dc0749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:03.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.669+0000 7fa0e1da7700 1 -- 172.21.15.97:0/3344129365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0dc102ba0 msgr2=0x7fa0dc0749d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:03.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.669+0000 7fa0e1da7700 1 --2- 172.21.15.97:0/3344129365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0dc102ba0 0x7fa0dc0749d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:03.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.669+0000 7fa0e1da7700 1 -- 172.21.15.97:0/3344129365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa0dc1081c0 msgr2=0x7fa0dc074f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:03.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.669+0000 7fa0e1da7700 1 --2- 172.21.15.97:0/3344129365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa0dc1081c0 0x7fa0dc074f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:03.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.669+0000 7fa0e1da7700 1 -- 172.21.15.97:0/3344129365 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0dc1a6580 con 0x7fa0dc1021e0 2024-03-21T13:26:03.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.669+0000 7fa0e25a8700 1 --2- 172.21.15.97:0/3344129365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa0dc1081c0 0x7fa0dc074f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T13:26:03.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.669+0000 7fa0e1da7700 1 --2- 172.21.15.97:0/3344129365 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa0dc1021e0 0x7fa0dc074490 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa0d800eb30 tx=0x7fa0d800ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:26:03.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.669+0000 7fa0d2ffd700 1 -- 172.21.15.97:0/3344129365 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d800cbd0 con 0x7fa0dc1021e0 2024-03-21T13:26:03.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.669+0000 7fa0d2ffd700 1 -- 172.21.15.97:0/3344129365 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fa0d80044d0 con 0x7fa0dc1021e0 2024-03-21T13:26:03.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.669+0000 7fa0e400b700 1 -- 172.21.15.97:0/3344129365 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0dc1a6870 con 0x7fa0dc1021e0 2024-03-21T13:26:03.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.669+0000 7fa0d2ffd700 1 -- 172.21.15.97:0/3344129365 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d8003e00 con 0x7fa0dc1021e0 2024-03-21T13:26:03.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.669+0000 7fa0e400b700 1 -- 172.21.15.97:0/3344129365 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0dc1a6e00 con 0x7fa0dc1021e0 2024-03-21T13:26:03.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.670+0000 7fa0e400b700 1 -- 172.21.15.97:0/3344129365 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0dc068e50 con 0x7fa0dc1021e0 2024-03-21T13:26:03.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.671+0000 7fa0d2ffd700 1 -- 172.21.15.97:0/3344129365 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 15) v1 ==== 92095+0+0 (secure 0 0 0) 0x7fa0d8004000 con 0x7fa0dc1021e0 2024-03-21T13:26:03.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.671+0000 7fa0d2ffd700 1 --2- 172.21.15.97:0/3344129365 >> v2:172.21.15.97:6800/1943212787 conn(0x7fa0c806e210 0x7fa0c80706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:03.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.671+0000 7fa0e15a6700 1 --2- 172.21.15.97:0/3344129365 >> v2:172.21.15.97:6800/1943212787 conn(0x7fa0c806e210 0x7fa0c80706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:03.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.671+0000 7fa0d2ffd700 1 -- 172.21.15.97:0/3344129365 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4081+0+0 (secure 0 0 0) 0x7fa0d808dcb0 con 0x7fa0dc1021e0 2024-03-21T13:26:03.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.672+0000 7fa0e15a6700 1 --2- 172.21.15.97:0/3344129365 >> v2:172.21.15.97:6800/1943212787 conn(0x7fa0c806e210 0x7fa0c80706d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc1125f0 tx=0x7fa0cc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:26:03.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.676+0000 7fa0d2ffd700 1 -- 172.21.15.97:0/3344129365 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa0d80042f0 con 0x7fa0dc1021e0 2024-03-21T13:26:03.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.871+0000 7fa0e400b700 1 -- 172.21.15.97:0/3344129365 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa0dc1a7030 con 0x7fa0dc1021e0 2024-03-21T13:26:03.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.871+0000 7fa0d2ffd700 1 -- 172.21.15.97:0/3344129365 <== mon.2 v2:172.21.15.97:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa0d805bd00 con 0x7fa0dc1021e0 2024-03-21T13:26:03.872 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-03-21T13:26:03.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.873+0000 7fa0e400b700 1 -- 172.21.15.97:0/3344129365 >> v2:172.21.15.97:6800/1943212787 conn(0x7fa0c806e210 msgr2=0x7fa0c80706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:03.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.873+0000 7fa0e400b700 1 --2- 172.21.15.97:0/3344129365 >> v2:172.21.15.97:6800/1943212787 conn(0x7fa0c806e210 0x7fa0c80706d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc1125f0 tx=0x7fa0cc005fd0 comp rx=0 tx=0).stop 2024-03-21T13:26:03.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.873+0000 7fa0e400b700 1 -- 172.21.15.97:0/3344129365 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa0dc1021e0 msgr2=0x7fa0dc074490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:03.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.873+0000 7fa0e400b700 1 --2- 172.21.15.97:0/3344129365 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa0dc1021e0 0x7fa0dc074490 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa0d800eb30 tx=0x7fa0d800ee40 comp rx=0 tx=0).stop 2024-03-21T13:26:03.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.874+0000 7fa0e400b700 1 -- 172.21.15.97:0/3344129365 shutdown_connections 2024-03-21T13:26:03.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.874+0000 7fa0e400b700 1 --2- 172.21.15.97:0/3344129365 >> v2:172.21.15.97:6800/1943212787 conn(0x7fa0c806e210 0x7fa0c80706d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:03.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.874+0000 7fa0e400b700 1 --2- 172.21.15.97:0/3344129365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa0dc1081c0 0x7fa0dc074f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:03.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.874+0000 7fa0e400b700 1 --2- 172.21.15.97:0/3344129365 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa0dc1021e0 0x7fa0dc074490 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:03.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.874+0000 7fa0e400b700 1 --2- 172.21.15.97:0/3344129365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0dc102ba0 0x7fa0dc0749d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:03.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.874+0000 7fa0e400b700 1 -- 172.21.15.97:0/3344129365 >> 172.21.15.97:0/3344129365 conn(0x7fa0dc0792c0 msgr2=0x7fa0dc105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:26:03.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.874+0000 7fa0e400b700 1 -- 172.21.15.97:0/3344129365 shutdown_connections 2024-03-21T13:26:03.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:03.874+0000 7fa0e400b700 1 -- 172.21.15.97:0/3344129365 wait complete. 2024-03-21T13:26:04.465 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:04 smithi097 ceph-mon[39416]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-03-21T13:26:04.465 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:04 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3344129365' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T13:26:04.465 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:04 smithi097 ceph-mon[30040]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-03-21T13:26:04.465 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:04 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3344129365' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T13:26:04.491 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi097 2024-03-21T13:26:04.491 DEBUG:teuthology.orchestra.run.smithi144:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph orch apply rgw foo.a --placement '1;smithi097=foo.a' 2024-03-21T13:26:04.523 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:04 smithi144 ceph-mon[33633]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-03-21T13:26:04.523 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:04 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3344129365' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T13:26:04.744 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T13:26:06.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.335+0000 7f50132ec700 1 -- 172.21.15.144:0/343693536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f500c1010f0 msgr2=0x7f500c101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:06.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.335+0000 7f50132ec700 1 --2- 172.21.15.144:0/343693536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f500c1010f0 0x7f500c101510 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f50080099c0 tx=0x7f5008009cd0 comp rx=0 tx=0).stop 2024-03-21T13:26:06.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.335+0000 7f50132ec700 1 -- 172.21.15.144:0/343693536 shutdown_connections 2024-03-21T13:26:06.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.335+0000 7f50132ec700 1 --2- 172.21.15.144:0/343693536 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f500c102c90 0x7f500c109540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:06.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.335+0000 7f50132ec700 1 --2- 172.21.15.144:0/343693536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f500c1010f0 0x7f500c101510 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:06.341 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.335+0000 7f50132ec700 1 --2- 172.21.15.144:0/343693536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f500c1022f0 0x7f500c102750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:06.341 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.335+0000 7f50132ec700 1 -- 172.21.15.144:0/343693536 >> 172.21.15.144:0/343693536 conn(0x7f500c0fc6b0 msgr2=0x7f500c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:26:06.341 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.335+0000 7f50132ec700 1 -- 172.21.15.144:0/343693536 shutdown_connections 2024-03-21T13:26:06.341 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.335+0000 7f50132ec700 1 -- 172.21.15.144:0/343693536 wait complete. 2024-03-21T13:26:06.341 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.336+0000 7f50132ec700 1 Processor -- start 2024-03-21T13:26:06.341 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.336+0000 7f50132ec700 1 -- start start 2024-03-21T13:26:06.341 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.336+0000 7f50132ec700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f500c1010f0 0x7f500c06b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:06.342 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.336+0000 7f50132ec700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f500c1022f0 0x7f500c19c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:06.342 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.336+0000 7f50132ec700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f500c102c90 0x7f500c19ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:06.342 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.336+0000 7f50132ec700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f500c10c460 con 0x7f500c102c90 2024-03-21T13:26:06.342 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.336+0000 7f50132ec700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f500c10c2e0 con 0x7f500c1022f0 2024-03-21T13:26:06.342 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.336+0000 7f50132ec700 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f500c10c5e0 con 0x7f500c1010f0 2024-03-21T13:26:06.342 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.336+0000 7f5010887700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f500c1022f0 0x7f500c19c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:06.343 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.336+0000 7f5010887700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f500c1022f0 0x7f500c19c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:60288/0 (socket says 172.21.15.144:60288) 2024-03-21T13:26:06.343 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.336+0000 7f5010887700 1 -- 172.21.15.144:0/2588109399 learned_addr learned my addr 172.21.15.144:0/2588109399 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-21T13:26:06.343 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.336+0000 7f5011088700 1 --2- 172.21.15.144:0/2588109399 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f500c1010f0 0x7f500c06b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:06.343 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.336+0000 7f5011889700 1 --2- 172.21.15.144:0/2588109399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f500c102c90 0x7f500c19ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:06.343 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.336+0000 7f5010887700 1 -- 172.21.15.144:0/2588109399 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f500c1010f0 msgr2=0x7f500c06b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:06.343 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.336+0000 7f5010887700 1 --2- 172.21.15.144:0/2588109399 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f500c1010f0 0x7f500c06b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:06.343 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.336+0000 7f5010887700 1 -- 172.21.15.144:0/2588109399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f500c102c90 msgr2=0x7f500c19ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:06.344 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.336+0000 7f5010887700 1 --2- 172.21.15.144:0/2588109399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f500c102c90 0x7f500c19ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:06.344 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.336+0000 7f5010887700 1 -- 172.21.15.144:0/2588109399 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f500c19f720 con 0x7f500c1022f0 2024-03-21T13:26:06.344 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.336+0000 7f5011889700 1 --2- 172.21.15.144:0/2588109399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f500c102c90 0x7f500c19ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T13:26:06.344 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.336+0000 7f5010887700 1 --2- 172.21.15.144:0/2588109399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f500c1022f0 0x7f500c19c800 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc00c920 tx=0x7f4ffc00cce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:26:06.344 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.337+0000 7f50027fc700 1 -- 172.21.15.144:0/2588109399 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc007b00 con 0x7f500c1022f0 2024-03-21T13:26:06.344 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.337+0000 7f50027fc700 1 -- 172.21.15.144:0/2588109399 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f4ffc004ce0 con 0x7f500c1022f0 2024-03-21T13:26:06.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.337+0000 7f50132ec700 1 -- 172.21.15.144:0/2588109399 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f500c19fa10 con 0x7f500c1022f0 2024-03-21T13:26:06.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.337+0000 7f50027fc700 1 -- 172.21.15.144:0/2588109399 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc005630 con 0x7f500c1022f0 2024-03-21T13:26:06.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.337+0000 7f50132ec700 1 -- 172.21.15.144:0/2588109399 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f500c19ffa0 con 0x7f500c1022f0 2024-03-21T13:26:06.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.338+0000 7f50132ec700 1 -- 172.21.15.144:0/2588109399 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f500c068e50 con 0x7f500c1022f0 2024-03-21T13:26:06.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.338+0000 7f50027fc700 1 -- 172.21.15.144:0/2588109399 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 15) v1 ==== 92095+0+0 (secure 0 0 0) 0x7f4ffc004750 con 0x7f500c1022f0 2024-03-21T13:26:06.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.338+0000 7f50027fc700 1 --2- 172.21.15.144:0/2588109399 >> v2:172.21.15.97:6800/1943212787 conn(0x7f4ff806e210 0x7f4ff80706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:06.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.339+0000 7f50027fc700 1 -- 172.21.15.144:0/2588109399 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4081+0+0 (secure 0 0 0) 0x7f4ffc08d960 con 0x7f500c1022f0 2024-03-21T13:26:06.346 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.339+0000 7f5011088700 1 --2- 172.21.15.144:0/2588109399 >> v2:172.21.15.97:6800/1943212787 conn(0x7f4ff806e210 0x7f4ff80706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:06.346 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.339+0000 7f5011088700 1 --2- 172.21.15.144:0/2588109399 >> v2:172.21.15.97:6800/1943212787 conn(0x7f4ff806e210 0x7f4ff80706d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5008000c00 tx=0x7f5008005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:26:06.346 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.343+0000 7f50027fc700 1 -- 172.21.15.144:0/2588109399 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4ffc014030 con 0x7f500c1022f0 2024-03-21T13:26:06.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:06 smithi144 ceph-mon[33633]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-21T13:26:06.548 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.547+0000 7f50132ec700 1 -- 172.21.15.144:0/2588109399 --> v2:172.21.15.97:6800/1943212787 -- mgr_command(tid 0: {"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi097=foo.a", "target": ["mon-mgr", ""]}) v1 -- 0x7f500c02de70 con 0x7f4ff806e210 2024-03-21T13:26:06.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:06 smithi097 ceph-mon[39416]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-21T13:26:06.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:06 smithi097 ceph-mon[30040]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-21T13:26:06.555 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.555+0000 7f50027fc700 1 -- 172.21.15.144:0/2588109399 <== mgr.14152 v2:172.21.15.97:6800/1943212787 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+30 (secure 0 0 0) 0x7f500c02de70 con 0x7f4ff806e210 2024-03-21T13:26:06.555 INFO:teuthology.orchestra.run.smithi144.stdout:Scheduled rgw.foo.a update... 2024-03-21T13:26:06.558 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.557+0000 7f50132ec700 1 -- 172.21.15.144:0/2588109399 >> v2:172.21.15.97:6800/1943212787 conn(0x7f4ff806e210 msgr2=0x7f4ff80706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:06.558 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.557+0000 7f50132ec700 1 --2- 172.21.15.144:0/2588109399 >> v2:172.21.15.97:6800/1943212787 conn(0x7f4ff806e210 0x7f4ff80706d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5008000c00 tx=0x7f5008005fd0 comp rx=0 tx=0).stop 2024-03-21T13:26:06.558 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.557+0000 7f50132ec700 1 -- 172.21.15.144:0/2588109399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f500c1022f0 msgr2=0x7f500c19c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:06.558 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.557+0000 7f50132ec700 1 --2- 172.21.15.144:0/2588109399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f500c1022f0 0x7f500c19c800 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc00c920 tx=0x7f4ffc00cce0 comp rx=0 tx=0).stop 2024-03-21T13:26:06.558 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.557+0000 7f50132ec700 1 -- 172.21.15.144:0/2588109399 shutdown_connections 2024-03-21T13:26:06.558 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.557+0000 7f50132ec700 1 --2- 172.21.15.144:0/2588109399 >> v2:172.21.15.97:6800/1943212787 conn(0x7f4ff806e210 0x7f4ff80706d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:06.558 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.557+0000 7f50132ec700 1 --2- 172.21.15.144:0/2588109399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f500c102c90 0x7f500c19ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:06.558 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.557+0000 7f50132ec700 1 --2- 172.21.15.144:0/2588109399 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f500c1010f0 0x7f500c06b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:06.558 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.557+0000 7f50132ec700 1 --2- 172.21.15.144:0/2588109399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f500c1022f0 0x7f500c19c800 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:06.558 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.557+0000 7f50132ec700 1 -- 172.21.15.144:0/2588109399 >> 172.21.15.144:0/2588109399 conn(0x7f500c0fc6b0 msgr2=0x7f500c1055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:26:06.559 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.558+0000 7f50132ec700 1 -- 172.21.15.144:0/2588109399 shutdown_connections 2024-03-21T13:26:06.559 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:06.558+0000 7f50132ec700 1 -- 172.21.15.144:0/2588109399 wait complete. 2024-03-21T13:26:07.174 DEBUG:teuthology.orchestra.run.smithi097:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@rgw.foo.a.service 2024-03-21T13:26:07.177 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi144 2024-03-21T13:26:07.177 DEBUG:teuthology.orchestra.run.smithi144:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph osd pool create datapool 3 3 replicated 2024-03-21T13:26:07.456 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T13:26:07.829 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:07 smithi097 ceph-mon[39416]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi097=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:26:07.829 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:07 smithi097 ceph-mon[39416]: Saving service rgw.foo.a spec with placement smithi097=foo.a;count:1 2024-03-21T13:26:07.829 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:07 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:07.829 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:07 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:26:07.829 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:07 smithi097 ceph-mon[39416]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-21T13:26:07.829 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:07 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:26:07.829 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:07 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:26:07.829 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:07 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:07.829 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:07 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-03-21T13:26:07.830 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:07 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-03-21T13:26:07.830 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:07 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:07.830 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:07 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:26:07.830 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:07 smithi097 ceph-mon[39416]: Deploying daemon rgw.foo.a on smithi097 2024-03-21T13:26:07.830 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:07 smithi097 ceph-mon[30040]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi097=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:26:07.830 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:07 smithi097 ceph-mon[30040]: Saving service rgw.foo.a spec with placement smithi097=foo.a;count:1 2024-03-21T13:26:07.830 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:07 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:07.830 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:07 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:26:07.830 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:07 smithi097 ceph-mon[30040]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-21T13:26:07.830 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:07 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:26:07.831 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:07 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:26:07.831 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:07 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:07.831 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:07 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-03-21T13:26:07.831 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:07 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-03-21T13:26:07.831 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:07 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:07.831 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:07 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:26:07.831 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:07 smithi097 ceph-mon[30040]: Deploying daemon rgw.foo.a on smithi097 2024-03-21T13:26:07.831 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:07 smithi144 ceph-mon[33633]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi097=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:26:07.831 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:07 smithi144 ceph-mon[33633]: Saving service rgw.foo.a spec with placement smithi097=foo.a;count:1 2024-03-21T13:26:07.832 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:07 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:07.832 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:07 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:26:07.832 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:07 smithi144 ceph-mon[33633]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-21T13:26:07.832 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:07 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:26:07.832 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:07 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:26:07.832 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:07 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:07.832 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:07 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-03-21T13:26:07.832 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:07 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-03-21T13:26:07.832 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:07 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:07.832 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:07 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:26:07.832 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:07 smithi144 ceph-mon[33633]: Deploying daemon rgw.foo.a on smithi097 2024-03-21T13:26:09.058 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:09.056+0000 7f569784b700 1 -- 172.21.15.144:0/2127353029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56901022c0 msgr2=0x7f5690102720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:09.059 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:09.057+0000 7f569784b700 1 --2- 172.21.15.144:0/2127353029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56901022c0 0x7f5690102720 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f56800099c0 tx=0x7f5680009cd0 comp rx=0 tx=0).stop 2024-03-21T13:26:09.059 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:09.057+0000 7f569784b700 1 -- 172.21.15.144:0/2127353029 shutdown_connections 2024-03-21T13:26:09.059 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:09.057+0000 7f569784b700 1 --2- 172.21.15.144:0/2127353029 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5690102c60 0x7f5690109510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:09.059 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:09.057+0000 7f569784b700 1 --2- 172.21.15.144:0/2127353029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56901010c0 0x7f56901014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:09.059 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:09.057+0000 7f569784b700 1 --2- 172.21.15.144:0/2127353029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56901022c0 0x7f5690102720 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:09.059 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:09.057+0000 7f569784b700 1 -- 172.21.15.144:0/2127353029 >> 172.21.15.144:0/2127353029 conn(0x7f56900fc680 msgr2=0x7f56900feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:26:09.059 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:09.057+0000 7f569784b700 1 -- 172.21.15.144:0/2127353029 shutdown_connections 2024-03-21T13:26:09.059 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:09.057+0000 7f569784b700 1 -- 172.21.15.144:0/2127353029 wait complete. 2024-03-21T13:26:09.059 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:09.057+0000 7f569784b700 1 Processor -- start 2024-03-21T13:26:09.059 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:09.057+0000 7f569784b700 1 -- start start 2024-03-21T13:26:09.059 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:09.057+0000 7f569784b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56901010c0 0x7f569006b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:09.059 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:09.057+0000 7f569784b700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f56901022c0 0x7f569019c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:09.059 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:09.057+0000 7f569784b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5690102c60 0x7f569019ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:09.060 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:09.057+0000 7f569784b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569010c3f0 con 0x7f56901010c0 2024-03-21T13:26:09.060 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:09.057+0000 7f569784b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569010c270 con 0x7f5690102c60 2024-03-21T13:26:09.060 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:09.057+0000 7f569784b700 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f569010c570 con 0x7f56901022c0 2024-03-21T13:26:09.060 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:09.058+0000 7f5695de8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5690102c60 0x7f569019ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:09.060 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:09.058+0000 7f56955e7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56901010c0 0x7f569006b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:09.060 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:09.058+0000 7f5695de8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5690102c60 0x7f569019ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:60308/0 (socket says 172.21.15.144:60308) 2024-03-21T13:26:09.060 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:09.058+0000 7f56955e7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56901010c0 0x7f569006b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.144:43230/0 (socket says 172.21.15.144:43230) 2024-03-21T13:26:09.060 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:09.058+0000 7f5694de6700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f56901022c0 0x7f569019c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:09.060 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:09.058+0000 7f5695de8700 1 -- 172.21.15.144:0/2725654439 learned_addr learned my addr 172.21.15.144:0/2725654439 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-21T13:26:09.060 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:09.058+0000 7f5695de8700 1 -- 172.21.15.144:0/2725654439 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f56901022c0 msgr2=0x7f569019c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:09.060 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:09.058+0000 7f5695de8700 1 --2- 172.21.15.144:0/2725654439 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f56901022c0 0x7f569019c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:09.060 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:09.058+0000 7f5695de8700 1 -- 172.21.15.144:0/2725654439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56901010c0 msgr2=0x7f569006b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:09.060 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:09.058+0000 7f5695de8700 1 --2- 172.21.15.144:0/2725654439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56901010c0 0x7f569006b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:09.060 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:09.058+0000 7f5695de8700 1 -- 172.21.15.144:0/2725654439 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f569019f700 con 0x7f5690102c60 2024-03-21T13:26:09.060 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:09.058+0000 7f56955e7700 1 --2- 172.21.15.144:0/2725654439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56901010c0 0x7f569006b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T13:26:09.061 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:09.058+0000 7f5695de8700 1 --2- 172.21.15.144:0/2725654439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5690102c60 0x7f569019ee30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5688007ee0 tx=0x7f568800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:26:09.061 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:09.058+0000 7f56867fc700 1 -- 172.21.15.144:0/2725654439 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5688004bb0 con 0x7f5690102c60 2024-03-21T13:26:09.061 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:09.058+0000 7f56867fc700 1 -- 172.21.15.144:0/2725654439 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f5688004d50 con 0x7f5690102c60 2024-03-21T13:26:09.061 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:09.058+0000 7f569784b700 1 -- 172.21.15.144:0/2725654439 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f569019f9f0 con 0x7f5690102c60 2024-03-21T13:26:09.061 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:09.058+0000 7f56867fc700 1 -- 172.21.15.144:0/2725654439 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56880058a0 con 0x7f5690102c60 2024-03-21T13:26:09.061 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:09.058+0000 7f569784b700 1 -- 172.21.15.144:0/2725654439 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f569019fee0 con 0x7f5690102c60 2024-03-21T13:26:09.062 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:09.059+0000 7f569784b700 1 -- 172.21.15.144:0/2725654439 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5690068e50 con 0x7f5690102c60 2024-03-21T13:26:09.062 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:09.060+0000 7f56867fc700 1 -- 172.21.15.144:0/2725654439 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 15) v1 ==== 92095+0+0 (secure 0 0 0) 0x7f568801a020 con 0x7f5690102c60 2024-03-21T13:26:09.062 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:09.060+0000 7f56867fc700 1 --2- 172.21.15.144:0/2725654439 >> v2:172.21.15.97:6800/1943212787 conn(0x7f567c06e210 0x7f567c0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:09.062 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:09.060+0000 7f56867fc700 1 -- 172.21.15.144:0/2725654439 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4081+0+0 (secure 0 0 0) 0x7f568808d0e0 con 0x7f5690102c60 2024-03-21T13:26:09.062 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:09.060+0000 7f56955e7700 1 --2- 172.21.15.144:0/2725654439 >> v2:172.21.15.97:6800/1943212787 conn(0x7f567c06e210 0x7f567c0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:09.062 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:09.061+0000 7f56955e7700 1 --2- 172.21.15.144:0/2725654439 >> v2:172.21.15.97:6800/1943212787 conn(0x7f567c06e210 0x7f567c0706d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5690102120 tx=0x7f568c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:26:09.064 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:09.065+0000 7f56867fc700 1 -- 172.21.15.144:0/2725654439 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5688010070 con 0x7f5690102c60 2024-03-21T13:26:09.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:09.258+0000 7f569784b700 1 -- 172.21.15.144:0/2725654439 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} v 0) v1 -- 0x7f569002de60 con 0x7f5690102c60 2024-03-21T13:26:09.548 INFO:journalctl@ceph.rgw.foo.a.smithi097.stdout:Mar 21 13:26:09 smithi097 systemd[1]: Starting Ceph rgw.foo.a for a5dd1f32-e785-11ee-95cd-87774f69a715... 2024-03-21T13:26:09.800 INFO:journalctl@ceph.rgw.foo.a.smithi097.stdout:Mar 21 13:26:09 smithi097 podman[67788]: 2024-03-21 13:26:09.451362702 +0000 UTC m=+0.018469782 image pull 19e12293512ff7978c8adb5f72a829060dd47f07bfd6903b6b3178d2ebc29e4e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93 2024-03-21T13:26:09.800 INFO:journalctl@ceph.rgw.foo.a.smithi097.stdout:Mar 21 13:26:09 smithi097 podman[67788]: 2024-03-21 13:26:09.579680008 +0000 UTC m=+0.146787086 container create 71d31bf20f2cc96ab4f9586c9dfe1dfff3e0815f758cc381757454753be55fdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-rgw-foo-a, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, ceph=True, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe) 2024-03-21T13:26:09.800 INFO:journalctl@ceph.rgw.foo.a.smithi097.stdout:Mar 21 13:26:09 smithi097 podman[67788]: 2024-03-21 13:26:09.794776227 +0000 UTC m=+0.361883319 container init 71d31bf20f2cc96ab4f9586c9dfe1dfff3e0815f758cc381757454753be55fdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-rgw-foo-a, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-03-21T13:26:09.800 INFO:journalctl@ceph.rgw.foo.a.smithi097.stdout:Mar 21 13:26:09 smithi097 podman[67788]: 2024-03-21 13:26:09.800310492 +0000 UTC m=+0.367417572 container start 71d31bf20f2cc96ab4f9586c9dfe1dfff3e0815f758cc381757454753be55fdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-rgw-foo-a, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_BRANCH=HEAD) 2024-03-21T13:26:10.053 INFO:journalctl@ceph.rgw.foo.a.smithi097.stdout:Mar 21 13:26:09 smithi097 bash[67788]: 71d31bf20f2cc96ab4f9586c9dfe1dfff3e0815f758cc381757454753be55fdd 2024-03-21T13:26:10.054 INFO:journalctl@ceph.rgw.foo.a.smithi097.stdout:Mar 21 13:26:09 smithi097 systemd[1]: Started Ceph rgw.foo.a for a5dd1f32-e785-11ee-95cd-87774f69a715. 2024-03-21T13:26:10.182 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:10.182+0000 7f56867fc700 1 -- 172.21.15.144:0/2725654439 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]=0 pool 'datapool' created v59) v1 ==== 160+0+0 (secure 0 0 0) 0x7f568805b0b0 con 0x7f5690102c60 2024-03-21T13:26:10.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:10.187+0000 7f569784b700 1 -- 172.21.15.144:0/2725654439 >> v2:172.21.15.97:6800/1943212787 conn(0x7f567c06e210 msgr2=0x7f567c0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:10.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:10.187+0000 7f569784b700 1 --2- 172.21.15.144:0/2725654439 >> v2:172.21.15.97:6800/1943212787 conn(0x7f567c06e210 0x7f567c0706d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5690102120 tx=0x7f568c00b410 comp rx=0 tx=0).stop 2024-03-21T13:26:10.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:10.187+0000 7f569784b700 1 -- 172.21.15.144:0/2725654439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5690102c60 msgr2=0x7f569019ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:10.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:10.187+0000 7f569784b700 1 --2- 172.21.15.144:0/2725654439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5690102c60 0x7f569019ee30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5688007ee0 tx=0x7f568800d3b0 comp rx=0 tx=0).stop 2024-03-21T13:26:10.188 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:10.187+0000 7f569784b700 1 -- 172.21.15.144:0/2725654439 shutdown_connections 2024-03-21T13:26:10.188 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:10.187+0000 7f569784b700 1 --2- 172.21.15.144:0/2725654439 >> v2:172.21.15.97:6800/1943212787 conn(0x7f567c06e210 0x7f567c0706d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:10.188 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:10.187+0000 7f569784b700 1 --2- 172.21.15.144:0/2725654439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5690102c60 0x7f569019ee30 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:10.188 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:10.187+0000 7f569784b700 1 --2- 172.21.15.144:0/2725654439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56901010c0 0x7f569006b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:10.188 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:10.187+0000 7f569784b700 1 --2- 172.21.15.144:0/2725654439 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f56901022c0 0x7f569019c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:10.188 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:10.187+0000 7f569784b700 1 -- 172.21.15.144:0/2725654439 >> 172.21.15.144:0/2725654439 conn(0x7f56900fc680 msgr2=0x7f56901055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:26:10.188 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:10.187+0000 7f569784b700 1 -- 172.21.15.144:0/2725654439 shutdown_connections 2024-03-21T13:26:10.189 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:10.187+0000 7f569784b700 1 -- 172.21.15.144:0/2725654439 wait complete. 2024-03-21T13:26:10.189 INFO:teuthology.orchestra.run.smithi144.stderr:pool 'datapool' created 2024-03-21T13:26:10.490 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:10 smithi144 ceph-mon[33633]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:26:10.491 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:10 smithi144 ceph-mon[33633]: from='client.? 172.21.15.144:0/2725654439' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-03-21T13:26:10.491 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:10 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-03-21T13:26:10.491 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:10 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:10.491 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:10 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:10.491 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:10 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:10.491 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:10 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:10.491 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:10 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:10.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:10 smithi097 ceph-mon[39416]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:26:10.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:10 smithi097 ceph-mon[39416]: from='client.? 172.21.15.144:0/2725654439' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-03-21T13:26:10.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:10 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-03-21T13:26:10.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:10 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:10.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:10 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:10.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:10 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:10.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:10 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:10.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:10 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:10.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:10 smithi097 ceph-mon[30040]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:26:10.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:10 smithi097 ceph-mon[30040]: from='client.? 172.21.15.144:0/2725654439' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-03-21T13:26:10.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:10 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-03-21T13:26:10.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:10 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:10.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:10 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:10.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:10 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:10.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:10 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:10.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:10 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:10.792 DEBUG:teuthology.orchestra.run.smithi144:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- rbd pool init datapool 2024-03-21T13:26:11.032 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T13:26:11.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:11 smithi097 ceph-mon[30040]: Saving service rgw.foo.a spec with placement smithi097=foo.a;count:1 2024-03-21T13:26:11.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:11 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-03-21T13:26:11.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:11 smithi097 ceph-mon[30040]: osdmap e59: 8 total, 8 up, 8 in 2024-03-21T13:26:11.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:11 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/279630241' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-03-21T13:26:11.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:11 smithi097 ceph-mon[30040]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-03-21T13:26:11.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:11 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:26:11.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:11 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:11.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:11 smithi097 ceph-mon[39416]: Saving service rgw.foo.a spec with placement smithi097=foo.a;count:1 2024-03-21T13:26:11.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:11 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-03-21T13:26:11.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:11 smithi097 ceph-mon[39416]: osdmap e59: 8 total, 8 up, 8 in 2024-03-21T13:26:11.306 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:11 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/279630241' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-03-21T13:26:11.306 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:11 smithi097 ceph-mon[39416]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-03-21T13:26:11.306 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:11 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:26:11.306 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:11 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:11.411 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:11 smithi144 ceph-mon[33633]: Saving service rgw.foo.a spec with placement smithi097=foo.a;count:1 2024-03-21T13:26:11.411 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:11 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-03-21T13:26:11.411 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:11 smithi144 ceph-mon[33633]: osdmap e59: 8 total, 8 up, 8 in 2024-03-21T13:26:11.411 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:11 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/279630241' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-03-21T13:26:11.411 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:11 smithi144 ceph-mon[33633]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-03-21T13:26:11.411 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:11 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:26:11.412 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:11 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:12.446 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:12 smithi144 ceph-mon[33633]: pgmap v189: 36 pgs: 6 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:26:12.447 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:12 smithi144 ceph-mon[33633]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:26:12.447 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:12 smithi144 ceph-mon[33633]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-03-21T13:26:12.447 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:12 smithi144 ceph-mon[33633]: osdmap e60: 8 total, 8 up, 8 in 2024-03-21T13:26:12.447 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:12 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:12.447 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:12 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:12.447 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:12 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:26:12.447 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:12 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:26:12.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:12 smithi097 ceph-mon[30040]: pgmap v189: 36 pgs: 6 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:26:12.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:12 smithi097 ceph-mon[30040]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:26:12.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:12 smithi097 ceph-mon[30040]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-03-21T13:26:12.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:12 smithi097 ceph-mon[30040]: osdmap e60: 8 total, 8 up, 8 in 2024-03-21T13:26:12.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:12 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:12.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:12 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:12.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:12 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:26:12.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:12 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:26:12.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:12 smithi097 ceph-mon[39416]: pgmap v189: 36 pgs: 6 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:26:12.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:12 smithi097 ceph-mon[39416]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:26:12.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:12 smithi097 ceph-mon[39416]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-03-21T13:26:12.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:12 smithi097 ceph-mon[39416]: osdmap e60: 8 total, 8 up, 8 in 2024-03-21T13:26:12.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:12 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:12.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:12 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:12.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:12 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:26:12.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:12 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:26:13.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:13 smithi097 ceph-mon[39416]: Checking dashboard <-> RGW credentials 2024-03-21T13:26:13.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:13 smithi097 ceph-mon[39416]: osdmap e61: 8 total, 8 up, 8 in 2024-03-21T13:26:13.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:13 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/110547329' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-03-21T13:26:13.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:13 smithi097 ceph-mon[39416]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-03-21T13:26:13.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:13 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/66016532' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-03-21T13:26:13.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:13 smithi097 ceph-mon[39416]: from='client.? 172.21.15.144:0/2868580730' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-03-21T13:26:13.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:13 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-03-21T13:26:13.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:13 smithi097 ceph-mon[30040]: Checking dashboard <-> RGW credentials 2024-03-21T13:26:13.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:13 smithi097 ceph-mon[30040]: osdmap e61: 8 total, 8 up, 8 in 2024-03-21T13:26:13.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:13 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/110547329' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-03-21T13:26:13.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:13 smithi097 ceph-mon[30040]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-03-21T13:26:13.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:13 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/66016532' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-03-21T13:26:13.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:13 smithi097 ceph-mon[30040]: from='client.? 172.21.15.144:0/2868580730' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-03-21T13:26:13.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:13 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-03-21T13:26:13.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:13 smithi144 ceph-mon[33633]: Checking dashboard <-> RGW credentials 2024-03-21T13:26:13.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:13 smithi144 ceph-mon[33633]: osdmap e61: 8 total, 8 up, 8 in 2024-03-21T13:26:13.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:13 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/110547329' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-03-21T13:26:13.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:13 smithi144 ceph-mon[33633]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-03-21T13:26:13.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:13 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/66016532' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-03-21T13:26:13.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:13 smithi144 ceph-mon[33633]: from='client.? 172.21.15.144:0/2868580730' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-03-21T13:26:13.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:13 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-03-21T13:26:14.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:14 smithi097 ceph-mon[30040]: pgmap v192: 68 pgs: 6 creating+peering, 18 active+clean, 44 unknown; 578 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 682 B/s wr, 2 op/s 2024-03-21T13:26:14.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:14 smithi097 ceph-mon[30040]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-03-21T13:26:14.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:14 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/66016532' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-03-21T13:26:14.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:14 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-03-21T13:26:14.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:14 smithi097 ceph-mon[30040]: osdmap e62: 8 total, 8 up, 8 in 2024-03-21T13:26:14.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:14 smithi097 ceph-mon[39416]: pgmap v192: 68 pgs: 6 creating+peering, 18 active+clean, 44 unknown; 578 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 682 B/s wr, 2 op/s 2024-03-21T13:26:14.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:14 smithi097 ceph-mon[39416]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-03-21T13:26:14.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:14 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/66016532' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-03-21T13:26:14.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:14 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-03-21T13:26:14.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:14 smithi097 ceph-mon[39416]: osdmap e62: 8 total, 8 up, 8 in 2024-03-21T13:26:14.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:14 smithi144 ceph-mon[33633]: pgmap v192: 68 pgs: 6 creating+peering, 18 active+clean, 44 unknown; 578 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 682 B/s wr, 2 op/s 2024-03-21T13:26:14.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:14 smithi144 ceph-mon[33633]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-03-21T13:26:14.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:14 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/66016532' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-03-21T13:26:14.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:14 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-03-21T13:26:14.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:14 smithi144 ceph-mon[33633]: osdmap e62: 8 total, 8 up, 8 in 2024-03-21T13:26:15.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:15 smithi097 ceph-mon[39416]: osdmap e63: 8 total, 8 up, 8 in 2024-03-21T13:26:15.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:15 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/66016532' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-03-21T13:26:15.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:15 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/110547329' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-03-21T13:26:15.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:15 smithi097 ceph-mon[39416]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-03-21T13:26:15.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:15 smithi097 ceph-mon[30040]: osdmap e63: 8 total, 8 up, 8 in 2024-03-21T13:26:15.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:15 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/66016532' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-03-21T13:26:15.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:15 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/110547329' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-03-21T13:26:15.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:15 smithi097 ceph-mon[30040]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-03-21T13:26:15.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:15 smithi144 ceph-mon[33633]: osdmap e63: 8 total, 8 up, 8 in 2024-03-21T13:26:15.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:15 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/66016532' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-03-21T13:26:15.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:15 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/110547329' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-03-21T13:26:15.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:15 smithi144 ceph-mon[33633]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-03-21T13:26:15.905 DEBUG:teuthology.orchestra.run.smithi144:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph orch apply iscsi datapool admin admin --trusted_ip_list 172.21.15.144 --placement '1;smithi144=iscsi.a' 2024-03-21T13:26:16.146 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T13:26:16.533 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:16 smithi144 ceph-mon[33633]: pgmap v195: 100 pgs: 9 creating+peering, 42 active+clean, 49 unknown; 578 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-03-21T13:26:16.533 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:16 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/66016532' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-03-21T13:26:16.534 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:16 smithi144 ceph-mon[33633]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-03-21T13:26:16.534 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:16 smithi144 ceph-mon[33633]: osdmap e64: 8 total, 8 up, 8 in 2024-03-21T13:26:16.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:16 smithi097 ceph-mon[30040]: pgmap v195: 100 pgs: 9 creating+peering, 42 active+clean, 49 unknown; 578 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-03-21T13:26:16.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:16 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/66016532' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-03-21T13:26:16.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:16 smithi097 ceph-mon[30040]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-03-21T13:26:16.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:16 smithi097 ceph-mon[30040]: osdmap e64: 8 total, 8 up, 8 in 2024-03-21T13:26:16.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:16 smithi097 ceph-mon[39416]: pgmap v195: 100 pgs: 9 creating+peering, 42 active+clean, 49 unknown; 578 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-03-21T13:26:16.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:16 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/66016532' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-03-21T13:26:16.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:16 smithi097 ceph-mon[39416]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-03-21T13:26:16.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:16 smithi097 ceph-mon[39416]: osdmap e64: 8 total, 8 up, 8 in 2024-03-21T13:26:17.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:17 smithi144 ceph-mon[33633]: osdmap e65: 8 total, 8 up, 8 in 2024-03-21T13:26:17.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:17 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/66016532' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-03-21T13:26:17.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:17 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/110547329' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-03-21T13:26:17.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:17 smithi144 ceph-mon[33633]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-03-21T13:26:17.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:17 smithi144 ceph-mon[33633]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:26:17.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:17 smithi097 ceph-mon[30040]: osdmap e65: 8 total, 8 up, 8 in 2024-03-21T13:26:17.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:17 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/66016532' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-03-21T13:26:17.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:17 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/110547329' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-03-21T13:26:17.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:17 smithi097 ceph-mon[30040]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-03-21T13:26:17.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:17 smithi097 ceph-mon[30040]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:26:17.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:17 smithi097 ceph-mon[39416]: osdmap e65: 8 total, 8 up, 8 in 2024-03-21T13:26:17.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:17 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/66016532' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-03-21T13:26:17.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:17 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/110547329' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-03-21T13:26:17.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:17 smithi097 ceph-mon[39416]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-03-21T13:26:17.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:17 smithi097 ceph-mon[39416]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:26:17.762 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.759+0000 7f4ff790e700 1 -- 172.21.15.144:0/2821517350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ff0103e50 msgr2=0x7f4ff010a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:17.762 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.759+0000 7f4ff790e700 1 --2- 172.21.15.144:0/2821517350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ff0103e50 0x7f4ff010a700 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4fe800b0a0 tx=0x7f4fe800b3b0 comp rx=0 tx=0).stop 2024-03-21T13:26:17.762 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.759+0000 7f4ff790e700 1 -- 172.21.15.144:0/2821517350 shutdown_connections 2024-03-21T13:26:17.762 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.759+0000 7f4ff790e700 1 --2- 172.21.15.144:0/2821517350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ff0103e50 0x7f4ff010a700 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:17.762 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.759+0000 7f4ff790e700 1 --2- 172.21.15.144:0/2821517350 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4ff01004d0 0x7f4ff01008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:17.762 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.760+0000 7f4ff790e700 1 --2- 172.21.15.144:0/2821517350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ff01034b0 0x7f4ff0103910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:17.762 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.760+0000 7f4ff790e700 1 -- 172.21.15.144:0/2821517350 >> 172.21.15.144:0/2821517350 conn(0x7f4ff00fbe30 msgr2=0x7f4ff00fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:26:17.762 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.760+0000 7f4ff790e700 1 -- 172.21.15.144:0/2821517350 shutdown_connections 2024-03-21T13:26:17.762 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.760+0000 7f4ff790e700 1 -- 172.21.15.144:0/2821517350 wait complete. 2024-03-21T13:26:17.762 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.760+0000 7f4ff790e700 1 Processor -- start 2024-03-21T13:26:17.763 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.760+0000 7f4ff790e700 1 -- start start 2024-03-21T13:26:17.763 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.760+0000 7f4ff790e700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4ff01004d0 0x7f4ff0197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:17.763 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.760+0000 7f4ff790e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ff01034b0 0x7f4ff0198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:17.763 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.760+0000 7f4ff790e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ff0103e50 0x7f4ff019f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:17.765 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.760+0000 7f4ff790e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff010d590 con 0x7f4ff01034b0 2024-03-21T13:26:17.765 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.760+0000 7f4ff790e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff010d410 con 0x7f4ff0103e50 2024-03-21T13:26:17.765 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.760+0000 7f4ff790e700 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff010d710 con 0x7f4ff01004d0 2024-03-21T13:26:17.765 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.760+0000 7f4ff5eab700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ff0103e50 0x7f4ff019f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:17.765 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.760+0000 7f4ff5eab700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ff0103e50 0x7f4ff019f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34032/0 (socket says 172.21.15.144:34032) 2024-03-21T13:26:17.765 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.760+0000 7f4ff5eab700 1 -- 172.21.15.144:0/3875070005 learned_addr learned my addr 172.21.15.144:0/3875070005 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-21T13:26:17.766 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.761+0000 7f4ff4ea9700 1 --2- 172.21.15.144:0/3875070005 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ff01034b0 0x7f4ff0198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:17.766 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.761+0000 7f4ff5eab700 1 -- 172.21.15.144:0/3875070005 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4ff01004d0 msgr2=0x7f4ff0197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:17.766 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.761+0000 7f4ff56aa700 1 --2- 172.21.15.144:0/3875070005 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4ff01004d0 0x7f4ff0197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:17.766 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.761+0000 7f4ff5eab700 1 --2- 172.21.15.144:0/3875070005 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4ff01004d0 0x7f4ff0197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:17.766 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.761+0000 7f4ff5eab700 1 -- 172.21.15.144:0/3875070005 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ff01034b0 msgr2=0x7f4ff0198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:17.766 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.761+0000 7f4ff5eab700 1 --2- 172.21.15.144:0/3875070005 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ff01034b0 0x7f4ff0198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:17.766 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.761+0000 7f4ff5eab700 1 -- 172.21.15.144:0/3875070005 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff019f6d0 con 0x7f4ff0103e50 2024-03-21T13:26:17.766 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.761+0000 7f4ff4ea9700 1 --2- 172.21.15.144:0/3875070005 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ff01034b0 0x7f4ff0198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T13:26:17.766 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.761+0000 7f4ff5eab700 1 --2- 172.21.15.144:0/3875070005 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ff0103e50 0x7f4ff019f030 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8007d90 tx=0x7f4fe800bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:26:17.766 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.761+0000 7f4fe67fc700 1 -- 172.21.15.144:0/3875070005 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8021070 con 0x7f4ff0103e50 2024-03-21T13:26:17.767 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.761+0000 7f4ff790e700 1 -- 172.21.15.144:0/3875070005 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ff019f960 con 0x7f4ff0103e50 2024-03-21T13:26:17.767 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.761+0000 7f4fe67fc700 1 -- 172.21.15.144:0/3875070005 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f4fe801dd10 con 0x7f4ff0103e50 2024-03-21T13:26:17.767 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.761+0000 7f4fe67fc700 1 -- 172.21.15.144:0/3875070005 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe801be60 con 0x7f4ff0103e50 2024-03-21T13:26:17.767 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.761+0000 7f4ff790e700 1 -- 172.21.15.144:0/3875070005 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ff019fe40 con 0x7f4ff0103e50 2024-03-21T13:26:17.768 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.762+0000 7f4ff790e700 1 -- 172.21.15.144:0/3875070005 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ff00506a0 con 0x7f4ff0103e50 2024-03-21T13:26:17.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.762+0000 7f4fe67fc700 1 -- 172.21.15.144:0/3875070005 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 15) v1 ==== 92095+0+0 (secure 0 0 0) 0x7f4fe8024070 con 0x7f4ff0103e50 2024-03-21T13:26:17.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.763+0000 7f4fe67fc700 1 --2- 172.21.15.144:0/3875070005 >> v2:172.21.15.97:6800/1943212787 conn(0x7f4fdc06e2b0 0x7f4fdc070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:17.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.763+0000 7f4fe67fc700 1 -- 172.21.15.144:0/3875070005 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5926+0+0 (secure 0 0 0) 0x7f4fe808f570 con 0x7f4ff0103e50 2024-03-21T13:26:17.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.763+0000 7f4ff56aa700 1 --2- 172.21.15.144:0/3875070005 >> v2:172.21.15.97:6800/1943212787 conn(0x7f4fdc06e2b0 0x7f4fdc070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:17.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.763+0000 7f4ff56aa700 1 --2- 172.21.15.144:0/3875070005 >> v2:172.21.15.97:6800/1943212787 conn(0x7f4fdc06e2b0 0x7f4fdc070770 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4fec006fd0 tx=0x7f4fec008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:26:17.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.768+0000 7f4fe67fc700 1 -- 172.21.15.144:0/3875070005 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4fe805ce10 con 0x7f4ff0103e50 2024-03-21T13:26:17.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.975+0000 7f4ff790e700 1 -- 172.21.15.144:0/3875070005 --> v2:172.21.15.97:6800/1943212787 -- mgr_command(tid 0: {"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.144", "placement": "1;smithi144=iscsi.a", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ff00625d0 con 0x7f4fdc06e2b0 2024-03-21T13:26:17.983 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.983+0000 7f4fe67fc700 1 -- 172.21.15.144:0/3875070005 <== mgr.14152 v2:172.21.15.97:6800/1943212787 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+35 (secure 0 0 0) 0x7f4ff00625d0 con 0x7f4fdc06e2b0 2024-03-21T13:26:17.984 INFO:teuthology.orchestra.run.smithi144.stdout:Scheduled iscsi.datapool update... 2024-03-21T13:26:17.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.986+0000 7f4ff790e700 1 -- 172.21.15.144:0/3875070005 >> v2:172.21.15.97:6800/1943212787 conn(0x7f4fdc06e2b0 msgr2=0x7f4fdc070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:17.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.986+0000 7f4ff790e700 1 --2- 172.21.15.144:0/3875070005 >> v2:172.21.15.97:6800/1943212787 conn(0x7f4fdc06e2b0 0x7f4fdc070770 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4fec006fd0 tx=0x7f4fec008040 comp rx=0 tx=0).stop 2024-03-21T13:26:17.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.986+0000 7f4ff790e700 1 -- 172.21.15.144:0/3875070005 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ff0103e50 msgr2=0x7f4ff019f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:17.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.986+0000 7f4ff790e700 1 --2- 172.21.15.144:0/3875070005 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ff0103e50 0x7f4ff019f030 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8007d90 tx=0x7f4fe800bd70 comp rx=0 tx=0).stop 2024-03-21T13:26:17.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.986+0000 7f4ff790e700 1 -- 172.21.15.144:0/3875070005 shutdown_connections 2024-03-21T13:26:17.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.986+0000 7f4ff790e700 1 --2- 172.21.15.144:0/3875070005 >> v2:172.21.15.97:6800/1943212787 conn(0x7f4fdc06e2b0 0x7f4fdc070770 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:17.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.986+0000 7f4ff790e700 1 --2- 172.21.15.144:0/3875070005 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ff0103e50 0x7f4ff019f030 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:17.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.986+0000 7f4ff790e700 1 --2- 172.21.15.144:0/3875070005 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4ff01004d0 0x7f4ff0197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:17.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.986+0000 7f4ff790e700 1 --2- 172.21.15.144:0/3875070005 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ff01034b0 0x7f4ff0198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:17.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.986+0000 7f4ff790e700 1 -- 172.21.15.144:0/3875070005 >> 172.21.15.144:0/3875070005 conn(0x7f4ff00fbe30 msgr2=0x7f4ff00fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:26:17.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.987+0000 7f4ff790e700 1 -- 172.21.15.144:0/3875070005 shutdown_connections 2024-03-21T13:26:17.988 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:17.987+0000 7f4ff790e700 1 -- 172.21.15.144:0/3875070005 wait complete. 2024-03-21T13:26:18.352 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:18 smithi144 ceph-mon[33633]: pgmap v198: 132 pgs: 11 creating+peering, 83 active+clean, 38 unknown; 578 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-03-21T13:26:18.352 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:18 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/66016532' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-03-21T13:26:18.352 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:18 smithi144 ceph-mon[33633]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-03-21T13:26:18.352 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:18 smithi144 ceph-mon[33633]: osdmap e66: 8 total, 8 up, 8 in 2024-03-21T13:26:18.352 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:18 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/110547329' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-03-21T13:26:18.352 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:18 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/66016532' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-03-21T13:26:18.352 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:18 smithi144 ceph-mon[33633]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-03-21T13:26:18.352 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:18 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:18.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:18 smithi097 ceph-mon[30040]: pgmap v198: 132 pgs: 11 creating+peering, 83 active+clean, 38 unknown; 578 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-03-21T13:26:18.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:18 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/66016532' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-03-21T13:26:18.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:18 smithi097 ceph-mon[30040]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-03-21T13:26:18.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:18 smithi097 ceph-mon[30040]: osdmap e66: 8 total, 8 up, 8 in 2024-03-21T13:26:18.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:18 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/110547329' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-03-21T13:26:18.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:18 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/66016532' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-03-21T13:26:18.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:18 smithi097 ceph-mon[30040]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-03-21T13:26:18.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:18 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:18.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:18 smithi097 ceph-mon[39416]: pgmap v198: 132 pgs: 11 creating+peering, 83 active+clean, 38 unknown; 578 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-03-21T13:26:18.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:18 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/66016532' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-03-21T13:26:18.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:18 smithi097 ceph-mon[39416]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-03-21T13:26:18.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:18 smithi097 ceph-mon[39416]: osdmap e66: 8 total, 8 up, 8 in 2024-03-21T13:26:18.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:18 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/110547329' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-03-21T13:26:18.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:18 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/66016532' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-03-21T13:26:18.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:18 smithi097 ceph-mon[39416]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-03-21T13:26:18.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:18 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:18.701 INFO:tasks.cephadm:Distributing iscsi-gateway.cfg... 2024-03-21T13:26:18.701 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-21T13:26:18.702 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-03-21T13:26:18.732 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-21T13:26:18.732 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-03-21T13:26:18.761 DEBUG:teuthology.orchestra.run.smithi144:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@iscsi.iscsi.a.service 2024-03-21T13:26:18.804 INFO:tasks.cephadm:Adding prometheus.a on smithi144 2024-03-21T13:26:18.805 DEBUG:teuthology.orchestra.run.smithi144:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph orch apply prometheus '1;smithi144=a' 2024-03-21T13:26:19.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:19 smithi144 ceph-mon[33633]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.144", "placement": "1;smithi144=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:26:19.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:19 smithi144 ceph-mon[33633]: Saving service iscsi.datapool spec with placement smithi144=iscsi.a;count:1 2024-03-21T13:26:19.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:19 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/66016532' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-03-21T13:26:19.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:19 smithi144 ceph-mon[33633]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-03-21T13:26:19.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:19 smithi144 ceph-mon[33633]: osdmap e67: 8 total, 8 up, 8 in 2024-03-21T13:26:19.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:19 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:19.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:19 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:19.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:19 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:19.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:19 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-03-21T13:26:19.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:19 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-03-21T13:26:19.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:19 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:26:19.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:19 smithi097 ceph-mon[39416]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.144", "placement": "1;smithi144=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:26:19.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:19 smithi097 ceph-mon[39416]: Saving service iscsi.datapool spec with placement smithi144=iscsi.a;count:1 2024-03-21T13:26:19.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:19 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/66016532' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-03-21T13:26:19.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:19 smithi097 ceph-mon[39416]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-03-21T13:26:19.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:19 smithi097 ceph-mon[39416]: osdmap e67: 8 total, 8 up, 8 in 2024-03-21T13:26:19.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:19 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:19.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:19 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:19.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:19 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:19.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:19 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-03-21T13:26:19.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:19 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-03-21T13:26:19.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:19 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:26:19.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:19 smithi097 ceph-mon[30040]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.144", "placement": "1;smithi144=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:26:19.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:19 smithi097 ceph-mon[30040]: Saving service iscsi.datapool spec with placement smithi144=iscsi.a;count:1 2024-03-21T13:26:19.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:19 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/66016532' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-03-21T13:26:19.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:19 smithi097 ceph-mon[30040]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-03-21T13:26:19.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:19 smithi097 ceph-mon[30040]: osdmap e67: 8 total, 8 up, 8 in 2024-03-21T13:26:19.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:19 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:19.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:19 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:19.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:19 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:19.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:19 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-03-21T13:26:19.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:19 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-03-21T13:26:19.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:19 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:26:19.719 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T13:26:20.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:20 smithi097 ceph-mon[30040]: Deploying daemon iscsi.iscsi.a on smithi144 2024-03-21T13:26:20.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:20 smithi097 ceph-mon[30040]: pgmap v201: 132 pgs: 13 creating+peering, 110 active+clean, 9 unknown; 579 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 11 KiB/s rd, 2.5 KiB/s wr, 27 op/s 2024-03-21T13:26:20.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:20 smithi097 ceph-mon[30040]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-21T13:26:20.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:20 smithi097 ceph-mon[30040]: Cluster is now healthy 2024-03-21T13:26:20.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:20 smithi097 ceph-mon[39416]: Deploying daemon iscsi.iscsi.a on smithi144 2024-03-21T13:26:20.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:20 smithi097 ceph-mon[39416]: pgmap v201: 132 pgs: 13 creating+peering, 110 active+clean, 9 unknown; 579 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 11 KiB/s rd, 2.5 KiB/s wr, 27 op/s 2024-03-21T13:26:20.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:20 smithi097 ceph-mon[39416]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-21T13:26:20.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:20 smithi097 ceph-mon[39416]: Cluster is now healthy 2024-03-21T13:26:20.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:20 smithi144 ceph-mon[33633]: Deploying daemon iscsi.iscsi.a on smithi144 2024-03-21T13:26:20.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:20 smithi144 ceph-mon[33633]: pgmap v201: 132 pgs: 13 creating+peering, 110 active+clean, 9 unknown; 579 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 11 KiB/s rd, 2.5 KiB/s wr, 27 op/s 2024-03-21T13:26:20.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:20 smithi144 ceph-mon[33633]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-21T13:26:20.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:20 smithi144 ceph-mon[33633]: Cluster is now healthy 2024-03-21T13:26:21.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:21.936+0000 7fc8957a3700 1 -- 172.21.15.144:0/1989176044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8901010f0 msgr2=0x7fc890101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:21.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:21.936+0000 7fc8957a3700 1 --2- 172.21.15.144:0/1989176044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8901010f0 0x7fc890101510 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc8800099c0 tx=0x7fc880009cd0 comp rx=0 tx=0).stop 2024-03-21T13:26:21.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:21.936+0000 7fc8957a3700 1 -- 172.21.15.144:0/1989176044 shutdown_connections 2024-03-21T13:26:21.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:21.936+0000 7fc8957a3700 1 --2- 172.21.15.144:0/1989176044 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc890102c90 0x7fc890109540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:21.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:21.936+0000 7fc8957a3700 1 --2- 172.21.15.144:0/1989176044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8901010f0 0x7fc890101510 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:21.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:21.936+0000 7fc8957a3700 1 --2- 172.21.15.144:0/1989176044 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8901022f0 0x7fc890102750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:21.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:21.936+0000 7fc8957a3700 1 -- 172.21.15.144:0/1989176044 >> 172.21.15.144:0/1989176044 conn(0x7fc8900fc690 msgr2=0x7fc8900fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:26:21.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:21.936+0000 7fc8957a3700 1 -- 172.21.15.144:0/1989176044 shutdown_connections 2024-03-21T13:26:21.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:21.936+0000 7fc8957a3700 1 -- 172.21.15.144:0/1989176044 wait complete. 2024-03-21T13:26:21.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:21.937+0000 7fc8957a3700 1 Processor -- start 2024-03-21T13:26:21.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:21.937+0000 7fc8957a3700 1 -- start start 2024-03-21T13:26:21.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:21.937+0000 7fc8957a3700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc8901010f0 0x7fc8901969a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:21.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:21.937+0000 7fc8957a3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8901022f0 0x7fc890196ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:21.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:21.937+0000 7fc8957a3700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc890102c90 0x7fc89019df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:21.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:21.937+0000 7fc8957a3700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89010c410 con 0x7fc890102c90 2024-03-21T13:26:21.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:21.937+0000 7fc8957a3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89010c290 con 0x7fc8901022f0 2024-03-21T13:26:21.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:21.937+0000 7fc8957a3700 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc89010c590 con 0x7fc8901010f0 2024-03-21T13:26:21.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:21.937+0000 7fc8867fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8901022f0 0x7fc890196ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:21.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:21.937+0000 7fc8867fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8901022f0 0x7fc890196ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:43146/0 (socket says 172.21.15.144:43146) 2024-03-21T13:26:21.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:21.937+0000 7fc8867fc700 1 -- 172.21.15.144:0/633898001 learned_addr learned my addr 172.21.15.144:0/633898001 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-21T13:26:21.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:21.937+0000 7fc88effd700 1 --2- 172.21.15.144:0/633898001 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc8901010f0 0x7fc8901969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:21.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:21.937+0000 7fc88f7fe700 1 --2- 172.21.15.144:0/633898001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc890102c90 0x7fc89019df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:21.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:21.937+0000 7fc8867fc700 1 -- 172.21.15.144:0/633898001 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc8901010f0 msgr2=0x7fc8901969a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:21.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:21.937+0000 7fc8867fc700 1 --2- 172.21.15.144:0/633898001 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc8901010f0 0x7fc8901969a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:21.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:21.937+0000 7fc8867fc700 1 -- 172.21.15.144:0/633898001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc890102c90 msgr2=0x7fc89019df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:21.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:21.937+0000 7fc8867fc700 1 --2- 172.21.15.144:0/633898001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc890102c90 0x7fc89019df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:21.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:21.938+0000 7fc8867fc700 1 -- 172.21.15.144:0/633898001 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc89019e600 con 0x7fc8901022f0 2024-03-21T13:26:21.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:21.938+0000 7fc88f7fe700 1 --2- 172.21.15.144:0/633898001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc890102c90 0x7fc89019df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T13:26:21.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:21.938+0000 7fc8867fc700 1 --2- 172.21.15.144:0/633898001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8901022f0 0x7fc890196ee0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc87800d980 tx=0x7fc87800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:26:21.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:21.938+0000 7fc88cff9700 1 -- 172.21.15.144:0/633898001 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc878016070 con 0x7fc8901022f0 2024-03-21T13:26:21.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:21.938+0000 7fc88cff9700 1 -- 172.21.15.144:0/633898001 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fc878004830 con 0x7fc8901022f0 2024-03-21T13:26:21.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:21.938+0000 7fc8957a3700 1 -- 172.21.15.144:0/633898001 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc89019e8f0 con 0x7fc8901022f0 2024-03-21T13:26:21.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:21.938+0000 7fc88cff9700 1 -- 172.21.15.144:0/633898001 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc878005490 con 0x7fc8901022f0 2024-03-21T13:26:21.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:21.938+0000 7fc8957a3700 1 -- 172.21.15.144:0/633898001 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc89019ee80 con 0x7fc8901022f0 2024-03-21T13:26:21.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:21.939+0000 7fc8957a3700 1 -- 172.21.15.144:0/633898001 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc890068e50 con 0x7fc8901022f0 2024-03-21T13:26:21.942 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:21.939+0000 7fc88cff9700 1 -- 172.21.15.144:0/633898001 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 15) v1 ==== 92095+0+0 (secure 0 0 0) 0x7fc878004220 con 0x7fc8901022f0 2024-03-21T13:26:21.942 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:21.940+0000 7fc88cff9700 1 --2- 172.21.15.144:0/633898001 >> v2:172.21.15.97:6800/1943212787 conn(0x7fc87c06e210 0x7fc87c0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:21.942 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:21.940+0000 7fc88cff9700 1 -- 172.21.15.144:0/633898001 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5942+0+0 (secure 0 0 0) 0x7fc87808df70 con 0x7fc8901022f0 2024-03-21T13:26:21.942 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:21.940+0000 7fc88effd700 1 --2- 172.21.15.144:0/633898001 >> v2:172.21.15.97:6800/1943212787 conn(0x7fc87c06e210 0x7fc87c0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:21.942 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:21.941+0000 7fc88effd700 1 --2- 172.21.15.144:0/633898001 >> v2:172.21.15.97:6800/1943212787 conn(0x7fc87c06e210 0x7fc87c0706d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc880000c00 tx=0x7fc880005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:26:21.944 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:21.944+0000 7fc88cff9700 1 -- 172.21.15.144:0/633898001 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc87801f020 con 0x7fc8901022f0 2024-03-21T13:26:22.156 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:22.156+0000 7fc8957a3700 1 -- 172.21.15.144:0/633898001 --> v2:172.21.15.97:6800/1943212787 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi144=a", "target": ["mon-mgr", ""]}) v1 -- 0x7fc8900625d0 con 0x7fc87c06e210 2024-03-21T13:26:22.163 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:22.163+0000 7fc88cff9700 1 -- 172.21.15.144:0/633898001 <== mgr.14152 v2:172.21.15.97:6800/1943212787 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7fc8900625d0 con 0x7fc87c06e210 2024-03-21T13:26:22.163 INFO:teuthology.orchestra.run.smithi144.stdout:Scheduled prometheus update... 2024-03-21T13:26:22.166 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:22.166+0000 7fc8957a3700 1 -- 172.21.15.144:0/633898001 >> v2:172.21.15.97:6800/1943212787 conn(0x7fc87c06e210 msgr2=0x7fc87c0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:22.166 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:22.166+0000 7fc8957a3700 1 --2- 172.21.15.144:0/633898001 >> v2:172.21.15.97:6800/1943212787 conn(0x7fc87c06e210 0x7fc87c0706d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc880000c00 tx=0x7fc880005fb0 comp rx=0 tx=0).stop 2024-03-21T13:26:22.166 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:22.166+0000 7fc8957a3700 1 -- 172.21.15.144:0/633898001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8901022f0 msgr2=0x7fc890196ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:22.166 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:22.166+0000 7fc8957a3700 1 --2- 172.21.15.144:0/633898001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8901022f0 0x7fc890196ee0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc87800d980 tx=0x7fc87800dc90 comp rx=0 tx=0).stop 2024-03-21T13:26:22.166 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:22.166+0000 7fc8957a3700 1 -- 172.21.15.144:0/633898001 shutdown_connections 2024-03-21T13:26:22.167 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:22.166+0000 7fc8957a3700 1 --2- 172.21.15.144:0/633898001 >> v2:172.21.15.97:6800/1943212787 conn(0x7fc87c06e210 0x7fc87c0706d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:22.167 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:22.166+0000 7fc8957a3700 1 --2- 172.21.15.144:0/633898001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc890102c90 0x7fc89019df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:22.167 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:22.166+0000 7fc8957a3700 1 --2- 172.21.15.144:0/633898001 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc8901010f0 0x7fc8901969a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:22.167 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:22.166+0000 7fc8957a3700 1 --2- 172.21.15.144:0/633898001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8901022f0 0x7fc890196ee0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:22.167 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:22.166+0000 7fc8957a3700 1 -- 172.21.15.144:0/633898001 >> 172.21.15.144:0/633898001 conn(0x7fc8900fc690 msgr2=0x7fc8901055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:26:22.167 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:22.166+0000 7fc8957a3700 1 -- 172.21.15.144:0/633898001 shutdown_connections 2024-03-21T13:26:22.167 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:22.166+0000 7fc8957a3700 1 -- 172.21.15.144:0/633898001 wait complete. 2024-03-21T13:26:22.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:22 smithi144 ceph-mon[33633]: pgmap v202: 132 pgs: 10 creating+peering, 122 active+clean; 580 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 24 KiB/s rd, 3.0 KiB/s wr, 59 op/s 2024-03-21T13:26:22.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:22 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:22.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:22 smithi144 ceph-mon[33633]: osdmap e68: 8 total, 8 up, 8 in 2024-03-21T13:26:22.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:22 smithi097 ceph-mon[30040]: pgmap v202: 132 pgs: 10 creating+peering, 122 active+clean; 580 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 24 KiB/s rd, 3.0 KiB/s wr, 59 op/s 2024-03-21T13:26:22.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:22 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:22.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:22 smithi097 ceph-mon[30040]: osdmap e68: 8 total, 8 up, 8 in 2024-03-21T13:26:22.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:22 smithi097 ceph-mon[39416]: pgmap v202: 132 pgs: 10 creating+peering, 122 active+clean; 580 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 24 KiB/s rd, 3.0 KiB/s wr, 59 op/s 2024-03-21T13:26:22.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:22 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:22.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:22 smithi097 ceph-mon[39416]: osdmap e68: 8 total, 8 up, 8 in 2024-03-21T13:26:22.815 DEBUG:teuthology.orchestra.run.smithi144:prometheus.a> sudo journalctl -f -n 0 -u ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@prometheus.a.service 2024-03-21T13:26:22.818 INFO:tasks.cephadm:Adding node-exporter.a on smithi097 2024-03-21T13:26:22.819 INFO:tasks.cephadm:Adding node-exporter.b on smithi144 2024-03-21T13:26:22.819 DEBUG:teuthology.orchestra.run.smithi144:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph orch apply node-exporter '2;smithi097=a;smithi144=b' 2024-03-21T13:26:23.054 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:26:22 smithi144 podman[62168]: 2024-03-21 13:26:22.980112604 +0000 UTC m=+1.003769198 container start 260ec876767b01441664452e8f814301a4c0a529cd08ce2ff765a2760a4baa43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T13:26:23.054 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:26:23 smithi144 bash[62168]: 260ec876767b01441664452e8f814301a4c0a529cd08ce2ff765a2760a4baa43 2024-03-21T13:26:23.156 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T13:26:23.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:23 smithi144 ceph-mon[33633]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi144=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:26:23.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:23 smithi144 ceph-mon[33633]: Saving service prometheus spec with placement smithi144=a;count:1 2024-03-21T13:26:23.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:23 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:23.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:23 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:23.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:23 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:23.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:23 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:23.372 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:26:23 smithi144 systemd[1]: Started Ceph iscsi.iscsi.a for a5dd1f32-e785-11ee-95cd-87774f69a715. 2024-03-21T13:26:23.372 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:26:23 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug Started the configuration object watcher 2024-03-21T13:26:23.372 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:26:23 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug Checking for config object changes every 1s 2024-03-21T13:26:23.372 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:26:23 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug Processing osd blocklist entries for this node 2024-03-21T13:26:23.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:23 smithi097 ceph-mon[39416]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi144=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:26:23.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:23 smithi097 ceph-mon[39416]: Saving service prometheus spec with placement smithi144=a;count:1 2024-03-21T13:26:23.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:23 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:23.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:23 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:23.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:23 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:23.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:23 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:23.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:23 smithi097 ceph-mon[30040]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi144=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:26:23.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:23 smithi097 ceph-mon[30040]: Saving service prometheus spec with placement smithi144=a;count:1 2024-03-21T13:26:23.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:23 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:23.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:23 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:23.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:23 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:23.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:23 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:23.769 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:26:23 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug Reading the configuration object to update local LIO configuration 2024-03-21T13:26:23.769 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:26:23 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug Configuration does not have an entry for this host(smithi144.front.sepia.ceph.com) - nothing to define to LIO 2024-03-21T13:26:23.769 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:26:23 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug * Running on http://[::]:5000/ (Press CTRL+C to quit) 2024-03-21T13:26:24.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:24 smithi144 ceph-mon[33633]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 67 KiB/s rd, 5.7 KiB/s wr, 164 op/s 2024-03-21T13:26:24.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:24 smithi144 ceph-mon[33633]: Checking pool "datapool" exists for service iscsi.datapool 2024-03-21T13:26:24.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:24 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:26:24.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:24 smithi144 ceph-mon[33633]: from='client.? 172.21.15.144:0/969044511' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T13:26:24.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:24 smithi097 ceph-mon[30040]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 67 KiB/s rd, 5.7 KiB/s wr, 164 op/s 2024-03-21T13:26:24.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:24 smithi097 ceph-mon[30040]: Checking pool "datapool" exists for service iscsi.datapool 2024-03-21T13:26:24.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:24 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:26:24.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:24 smithi097 ceph-mon[30040]: from='client.? 172.21.15.144:0/969044511' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T13:26:24.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:24 smithi097 ceph-mon[39416]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 67 KiB/s rd, 5.7 KiB/s wr, 164 op/s 2024-03-21T13:26:24.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:24 smithi097 ceph-mon[39416]: Checking pool "datapool" exists for service iscsi.datapool 2024-03-21T13:26:24.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:24 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:26:24.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:24 smithi097 ceph-mon[39416]: from='client.? 172.21.15.144:0/969044511' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T13:26:25.561 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.561+0000 7f1375d27700 1 -- 172.21.15.144:0/2977925114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13701055b0 msgr2=0x7f13701079a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:25.562 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.561+0000 7f1375d27700 1 --2- 172.21.15.144:0/2977925114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13701055b0 0x7f13701079a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1364009990 tx=0x7f1364009ca0 comp rx=0 tx=0).stop 2024-03-21T13:26:25.562 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.561+0000 7f1375d27700 1 -- 172.21.15.144:0/2977925114 shutdown_connections 2024-03-21T13:26:25.562 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.561+0000 7f1375d27700 1 --2- 172.21.15.144:0/2977925114 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1370107ee0 0x7f137010a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:25.562 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.561+0000 7f1375d27700 1 --2- 172.21.15.144:0/2977925114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1370102c80 0x7f1370105070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:25.562 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.561+0000 7f1375d27700 1 --2- 172.21.15.144:0/2977925114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13701055b0 0x7f13701079a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:25.562 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.561+0000 7f1375d27700 1 -- 172.21.15.144:0/2977925114 >> 172.21.15.144:0/2977925114 conn(0x7f13700fc690 msgr2=0x7f13700fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:26:25.562 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.561+0000 7f1375d27700 1 -- 172.21.15.144:0/2977925114 shutdown_connections 2024-03-21T13:26:25.562 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.561+0000 7f1375d27700 1 -- 172.21.15.144:0/2977925114 wait complete. 2024-03-21T13:26:25.562 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.561+0000 7f1375d27700 1 Processor -- start 2024-03-21T13:26:25.562 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.562+0000 7f1375d27700 1 -- start start 2024-03-21T13:26:25.563 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.562+0000 7f1375d27700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1370102c80 0x7f13701025e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:25.563 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.562+0000 7f1375d27700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13701055b0 0x7f1370100c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:25.564 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.562+0000 7f1375d27700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1370107ee0 0x7f1370101170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:25.564 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.562+0000 7f1375d27700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137010d560 con 0x7f13701055b0 2024-03-21T13:26:25.564 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.562+0000 7f1375d27700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137010d3e0 con 0x7f1370107ee0 2024-03-21T13:26:25.564 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.562+0000 7f1375d27700 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f137010d6e0 con 0x7f1370102c80 2024-03-21T13:26:25.564 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.562+0000 7f136f7fe700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1370102c80 0x7f13701025e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:25.564 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.562+0000 7f136ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1370107ee0 0x7f1370101170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:25.564 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.562+0000 7f136f7fe700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1370102c80 0x7f13701025e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.144:42728/0 (socket says 172.21.15.144:42728) 2024-03-21T13:26:25.564 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.562+0000 7f136effd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13701055b0 0x7f1370100c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:25.564 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.562+0000 7f136f7fe700 1 -- 172.21.15.144:0/1413423363 learned_addr learned my addr 172.21.15.144:0/1413423363 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-21T13:26:25.564 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.562+0000 7f136ffff700 1 -- 172.21.15.144:0/1413423363 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1370102c80 msgr2=0x7f13701025e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:25.564 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.562+0000 7f136ffff700 1 --2- 172.21.15.144:0/1413423363 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1370102c80 0x7f13701025e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:25.564 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.562+0000 7f136ffff700 1 -- 172.21.15.144:0/1413423363 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13701055b0 msgr2=0x7f1370100c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:25.564 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.562+0000 7f136ffff700 1 --2- 172.21.15.144:0/1413423363 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13701055b0 0x7f1370100c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:25.564 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.562+0000 7f136ffff700 1 -- 172.21.15.144:0/1413423363 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1370101a20 con 0x7f1370107ee0 2024-03-21T13:26:25.565 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.562+0000 7f136f7fe700 1 --2- 172.21.15.144:0/1413423363 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1370102c80 0x7f13701025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T13:26:25.565 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.562+0000 7f136effd700 1 --2- 172.21.15.144:0/1413423363 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13701055b0 0x7f1370100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T13:26:25.565 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.563+0000 7f136ffff700 1 --2- 172.21.15.144:0/1413423363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1370107ee0 0x7f1370101170 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1368007ee0 tx=0x7f136800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:26:25.565 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.563+0000 7f136cff9700 1 -- 172.21.15.144:0/1413423363 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1368004db0 con 0x7f1370107ee0 2024-03-21T13:26:25.565 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.563+0000 7f136cff9700 1 -- 172.21.15.144:0/1413423363 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f1368003a00 con 0x7f1370107ee0 2024-03-21T13:26:25.565 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.563+0000 7f1375d27700 1 -- 172.21.15.144:0/1413423363 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13701a1cc0 con 0x7f1370107ee0 2024-03-21T13:26:25.565 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.563+0000 7f1375d27700 1 -- 172.21.15.144:0/1413423363 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13701a2180 con 0x7f1370107ee0 2024-03-21T13:26:25.565 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.563+0000 7f136cff9700 1 -- 172.21.15.144:0/1413423363 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1368004470 con 0x7f1370107ee0 2024-03-21T13:26:25.566 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.564+0000 7f1375d27700 1 -- 172.21.15.144:0/1413423363 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1370068e50 con 0x7f1370107ee0 2024-03-21T13:26:25.566 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.565+0000 7f136cff9700 1 -- 172.21.15.144:0/1413423363 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 16) v1 ==== 92146+0+0 (secure 0 0 0) 0x7f136801c020 con 0x7f1370107ee0 2024-03-21T13:26:25.566 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.566+0000 7f136cff9700 1 --2- 172.21.15.144:0/1413423363 >> v2:172.21.15.97:6800/1943212787 conn(0x7f135806e2f0 0x7f13580707b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:25.566 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.566+0000 7f136cff9700 1 -- 172.21.15.144:0/1413423363 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5942+0+0 (secure 0 0 0) 0x7f136808d8c0 con 0x7f1370107ee0 2024-03-21T13:26:25.566 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.566+0000 7f136f7fe700 1 --2- 172.21.15.144:0/1413423363 >> v2:172.21.15.97:6800/1943212787 conn(0x7f135806e2f0 0x7f13580707b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:25.566 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.566+0000 7f136f7fe700 1 --2- 172.21.15.144:0/1413423363 >> v2:172.21.15.97:6800/1943212787 conn(0x7f135806e2f0 0x7f13580707b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f13600104e0 tx=0x7f1360007480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:26:25.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.571+0000 7f136cff9700 1 -- 172.21.15.144:0/1413423363 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1368010070 con 0x7f1370107ee0 2024-03-21T13:26:25.787 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.787+0000 7f1375d27700 1 -- 172.21.15.144:0/1413423363 --> v2:172.21.15.97:6800/1943212787 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi097=a;smithi144=b", "target": ["mon-mgr", ""]}) v1 -- 0x7f137002de60 con 0x7f135806e2f0 2024-03-21T13:26:25.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.793+0000 7f136cff9700 1 -- 172.21.15.144:0/1413423363 <== mgr.14152 v2:172.21.15.97:6800/1943212787 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f137002de60 con 0x7f135806e2f0 2024-03-21T13:26:25.794 INFO:teuthology.orchestra.run.smithi144.stdout:Scheduled node-exporter update... 2024-03-21T13:26:25.796 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.796+0000 7f1375d27700 1 -- 172.21.15.144:0/1413423363 >> v2:172.21.15.97:6800/1943212787 conn(0x7f135806e2f0 msgr2=0x7f13580707b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:25.796 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.796+0000 7f1375d27700 1 --2- 172.21.15.144:0/1413423363 >> v2:172.21.15.97:6800/1943212787 conn(0x7f135806e2f0 0x7f13580707b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f13600104e0 tx=0x7f1360007480 comp rx=0 tx=0).stop 2024-03-21T13:26:25.796 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.796+0000 7f1375d27700 1 -- 172.21.15.144:0/1413423363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1370107ee0 msgr2=0x7f1370101170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:25.796 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.796+0000 7f1375d27700 1 --2- 172.21.15.144:0/1413423363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1370107ee0 0x7f1370101170 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1368007ee0 tx=0x7f136800d3b0 comp rx=0 tx=0).stop 2024-03-21T13:26:25.797 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.796+0000 7f1375d27700 1 -- 172.21.15.144:0/1413423363 shutdown_connections 2024-03-21T13:26:25.797 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.796+0000 7f1375d27700 1 --2- 172.21.15.144:0/1413423363 >> v2:172.21.15.97:6800/1943212787 conn(0x7f135806e2f0 0x7f13580707b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:25.797 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.796+0000 7f1375d27700 1 --2- 172.21.15.144:0/1413423363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1370107ee0 0x7f1370101170 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:25.797 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.796+0000 7f1375d27700 1 --2- 172.21.15.144:0/1413423363 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1370102c80 0x7f13701025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:25.797 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.796+0000 7f1375d27700 1 --2- 172.21.15.144:0/1413423363 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13701055b0 0x7f1370100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:25.797 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.796+0000 7f1375d27700 1 -- 172.21.15.144:0/1413423363 >> 172.21.15.144:0/1413423363 conn(0x7f13700fc690 msgr2=0x7f13701035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:26:25.797 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.796+0000 7f1375d27700 1 -- 172.21.15.144:0/1413423363 shutdown_connections 2024-03-21T13:26:25.797 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:25.796+0000 7f1375d27700 1 -- 172.21.15.144:0/1413423363 wait complete. 2024-03-21T13:26:26.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:26 smithi097 ceph-mon[30040]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s rd, 4.3 KiB/s wr, 141 op/s 2024-03-21T13:26:26.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:26 smithi097 ceph-mon[30040]: mgrmap e16: y(active, since 4m), standbys: x 2024-03-21T13:26:26.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:26 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:26.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:26 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:26.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:26 smithi097 ceph-mon[39416]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s rd, 4.3 KiB/s wr, 141 op/s 2024-03-21T13:26:26.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:26 smithi097 ceph-mon[39416]: mgrmap e16: y(active, since 4m), standbys: x 2024-03-21T13:26:26.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:26 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:26.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:26 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:26.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:26 smithi144 ceph-mon[33633]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s rd, 4.3 KiB/s wr, 141 op/s 2024-03-21T13:26:26.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:26 smithi144 ceph-mon[33633]: mgrmap e16: y(active, since 4m), standbys: x 2024-03-21T13:26:26.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:26 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:26.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:26 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:26.704 DEBUG:teuthology.orchestra.run.smithi097:node-exporter.a> sudo journalctl -f -n 0 -u ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@node-exporter.a.service 2024-03-21T13:26:26.707 DEBUG:teuthology.orchestra.run.smithi144:node-exporter.b> sudo journalctl -f -n 0 -u ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@node-exporter.b.service 2024-03-21T13:26:26.710 INFO:tasks.cephadm:Adding alertmanager.a on smithi097 2024-03-21T13:26:26.710 DEBUG:teuthology.orchestra.run.smithi144:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph orch apply alertmanager '1;smithi097=a' 2024-03-21T13:26:27.305 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T13:26:27.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:27 smithi097 ceph-mon[30040]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi097=a;smithi144=b", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:26:27.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:27 smithi097 ceph-mon[30040]: Saving service node-exporter spec with placement smithi097=a;smithi144=b;count:2 2024-03-21T13:26:27.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:27 smithi097 ceph-mon[39416]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi097=a;smithi144=b", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:26:27.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:27 smithi097 ceph-mon[39416]: Saving service node-exporter spec with placement smithi097=a;smithi144=b;count:2 2024-03-21T13:26:27.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:27 smithi144 ceph-mon[33633]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi097=a;smithi144=b", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:26:27.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:27 smithi144 ceph-mon[33633]: Saving service node-exporter spec with placement smithi097=a;smithi144=b;count:2 2024-03-21T13:26:28.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:28 smithi097 ceph-mon[30040]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 52 KiB/s rd, 3.8 KiB/s wr, 125 op/s 2024-03-21T13:26:28.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:28 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:28.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:28 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:28.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:28 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:26:28.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:28 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:26:28.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:28 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:28.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:28 smithi097 ceph-mon[39416]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 52 KiB/s rd, 3.8 KiB/s wr, 125 op/s 2024-03-21T13:26:28.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:28 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:28.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:28 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:28.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:28 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:26:28.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:28 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:26:28.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:28 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:28.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:28 smithi144 ceph-mon[33633]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 52 KiB/s rd, 3.8 KiB/s wr, 125 op/s 2024-03-21T13:26:28.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:28 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:28.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:28 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:28.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:28 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:26:28.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:28 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:26:28.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:28 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:29.190 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.169+0000 7f766da35700 1 -- 172.21.15.144:0/2041804397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76681033e0 msgr2=0x7f7668103840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:29.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.169+0000 7f766da35700 1 --2- 172.21.15.144:0/2041804397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76681033e0 0x7f7668103840 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f76500099c0 tx=0x7f7650009cd0 comp rx=0 tx=0).stop 2024-03-21T13:26:29.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.169+0000 7f766da35700 1 -- 172.21.15.144:0/2041804397 shutdown_connections 2024-03-21T13:26:29.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.169+0000 7f766da35700 1 --2- 172.21.15.144:0/2041804397 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7668103d80 0x7f766810a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:29.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.169+0000 7f766da35700 1 --2- 172.21.15.144:0/2041804397 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f76681021e0 0x7f7668102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:29.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.169+0000 7f766da35700 1 --2- 172.21.15.144:0/2041804397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76681033e0 0x7f7668103840 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:29.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.169+0000 7f766da35700 1 -- 172.21.15.144:0/2041804397 >> 172.21.15.144:0/2041804397 conn(0x7f76680fd7a0 msgr2=0x7f76680ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:26:29.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.169+0000 7f766da35700 1 -- 172.21.15.144:0/2041804397 shutdown_connections 2024-03-21T13:26:29.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.169+0000 7f766da35700 1 -- 172.21.15.144:0/2041804397 wait complete. 2024-03-21T13:26:29.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.169+0000 7f766da35700 1 Processor -- start 2024-03-21T13:26:29.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.169+0000 7f766da35700 1 -- start start 2024-03-21T13:26:29.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.170+0000 7f766da35700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76681021e0 0x7f7668075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:29.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.170+0000 7f766da35700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76681033e0 0x7f7668074580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:29.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.170+0000 7f766da35700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7668103d80 0x7f7668074ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:29.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.170+0000 7f766da35700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766810d570 con 0x7f76681033e0 2024-03-21T13:26:29.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.170+0000 7f766da35700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766810d3f0 con 0x7f76681021e0 2024-03-21T13:26:29.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.170+0000 7f766da35700 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f766810d6f0 con 0x7f7668103d80 2024-03-21T13:26:29.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.170+0000 7f7666ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76681021e0 0x7f7668075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:29.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.170+0000 7f7666ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76681021e0 0x7f7668075f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:43250/0 (socket says 172.21.15.144:43250) 2024-03-21T13:26:29.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.170+0000 7f7666ffd700 1 -- 172.21.15.144:0/4268348319 learned_addr learned my addr 172.21.15.144:0/4268348319 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-21T13:26:29.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.170+0000 7f765e7fc700 1 --2- 172.21.15.144:0/4268348319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76681033e0 0x7f7668074580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:29.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.170+0000 7f76677fe700 1 --2- 172.21.15.144:0/4268348319 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7668103d80 0x7f7668074ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:29.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.170+0000 7f7666ffd700 1 -- 172.21.15.144:0/4268348319 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7668103d80 msgr2=0x7f7668074ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:29.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.170+0000 7f7666ffd700 1 --2- 172.21.15.144:0/4268348319 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7668103d80 0x7f7668074ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:29.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.170+0000 7f7666ffd700 1 -- 172.21.15.144:0/4268348319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76681033e0 msgr2=0x7f7668074580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:29.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.170+0000 7f7666ffd700 1 --2- 172.21.15.144:0/4268348319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76681033e0 0x7f7668074580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:29.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.170+0000 7f7666ffd700 1 -- 172.21.15.144:0/4268348319 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7668075370 con 0x7f76681021e0 2024-03-21T13:26:29.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.170+0000 7f7666ffd700 1 --2- 172.21.15.144:0/4268348319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76681021e0 0x7f7668075f30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f765800b500 tx=0x7f765800b810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:26:29.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.170+0000 7f766ca33700 1 -- 172.21.15.144:0/4268348319 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7658015020 con 0x7f76681021e0 2024-03-21T13:26:29.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.171+0000 7f766da35700 1 -- 172.21.15.144:0/4268348319 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76681a1dd0 con 0x7f76681021e0 2024-03-21T13:26:29.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.171+0000 7f766ca33700 1 -- 172.21.15.144:0/4268348319 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f7658011ce0 con 0x7f76681021e0 2024-03-21T13:26:29.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.171+0000 7f766ca33700 1 -- 172.21.15.144:0/4268348319 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7658010c50 con 0x7f76681021e0 2024-03-21T13:26:29.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.171+0000 7f766da35700 1 -- 172.21.15.144:0/4268348319 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76681a2250 con 0x7f76681021e0 2024-03-21T13:26:29.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.172+0000 7f766da35700 1 -- 172.21.15.144:0/4268348319 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7668068e50 con 0x7f76681021e0 2024-03-21T13:26:29.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.172+0000 7f766ca33700 1 -- 172.21.15.144:0/4268348319 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 16) v1 ==== 92146+0+0 (secure 0 0 0) 0x7f7658018020 con 0x7f76681021e0 2024-03-21T13:26:29.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.172+0000 7f766ca33700 1 --2- 172.21.15.144:0/4268348319 >> v2:172.21.15.97:6800/1943212787 conn(0x7f765406e2a0 0x7f7654070760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:29.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.173+0000 7f766ca33700 1 -- 172.21.15.144:0/4268348319 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5942+0+0 (secure 0 0 0) 0x7f765808d8f0 con 0x7f76681021e0 2024-03-21T13:26:29.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.173+0000 7f765e7fc700 1 --2- 172.21.15.144:0/4268348319 >> v2:172.21.15.97:6800/1943212787 conn(0x7f765406e2a0 0x7f7654070760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:29.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.173+0000 7f765e7fc700 1 --2- 172.21.15.144:0/4268348319 >> v2:172.21.15.97:6800/1943212787 conn(0x7f765406e2a0 0x7f7654070760 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7650009fd0 tx=0x7f7650005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:26:29.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.177+0000 7f766ca33700 1 -- 172.21.15.144:0/4268348319 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f765801f020 con 0x7f76681021e0 2024-03-21T13:26:29.391 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.391+0000 7f766da35700 1 -- 172.21.15.144:0/4268348319 --> v2:172.21.15.97:6800/1943212787 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi097=a", "target": ["mon-mgr", ""]}) v1 -- 0x7f76680625d0 con 0x7f765406e2a0 2024-03-21T13:26:29.398 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.398+0000 7f766ca33700 1 -- 172.21.15.144:0/4268348319 <== mgr.14152 v2:172.21.15.97:6800/1943212787 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7f76680625d0 con 0x7f765406e2a0 2024-03-21T13:26:29.398 INFO:teuthology.orchestra.run.smithi144.stdout:Scheduled alertmanager update... 2024-03-21T13:26:29.400 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.400+0000 7f766da35700 1 -- 172.21.15.144:0/4268348319 >> v2:172.21.15.97:6800/1943212787 conn(0x7f765406e2a0 msgr2=0x7f7654070760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:29.401 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.400+0000 7f766da35700 1 --2- 172.21.15.144:0/4268348319 >> v2:172.21.15.97:6800/1943212787 conn(0x7f765406e2a0 0x7f7654070760 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7650009fd0 tx=0x7f7650005fb0 comp rx=0 tx=0).stop 2024-03-21T13:26:29.401 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.400+0000 7f766da35700 1 -- 172.21.15.144:0/4268348319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76681021e0 msgr2=0x7f7668075f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:29.401 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.400+0000 7f766da35700 1 --2- 172.21.15.144:0/4268348319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76681021e0 0x7f7668075f30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f765800b500 tx=0x7f765800b810 comp rx=0 tx=0).stop 2024-03-21T13:26:29.401 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.400+0000 7f766da35700 1 -- 172.21.15.144:0/4268348319 shutdown_connections 2024-03-21T13:26:29.401 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.400+0000 7f766da35700 1 --2- 172.21.15.144:0/4268348319 >> v2:172.21.15.97:6800/1943212787 conn(0x7f765406e2a0 0x7f7654070760 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:29.401 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.400+0000 7f766da35700 1 --2- 172.21.15.144:0/4268348319 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7668103d80 0x7f7668074ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:29.401 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.400+0000 7f766da35700 1 --2- 172.21.15.144:0/4268348319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76681021e0 0x7f7668075f30 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:29.401 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.400+0000 7f766da35700 1 --2- 172.21.15.144:0/4268348319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76681033e0 0x7f7668074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:29.401 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.400+0000 7f766da35700 1 -- 172.21.15.144:0/4268348319 >> 172.21.15.144:0/4268348319 conn(0x7f76680fd7a0 msgr2=0x7f76680ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:26:29.401 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.401+0000 7f766da35700 1 -- 172.21.15.144:0/4268348319 shutdown_connections 2024-03-21T13:26:29.401 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:29.401+0000 7f766da35700 1 -- 172.21.15.144:0/4268348319 wait complete. 2024-03-21T13:26:29.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:29 smithi097 ceph-mon[30040]: Deploying daemon prometheus.a on smithi144 2024-03-21T13:26:29.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:29 smithi097 ceph-mon[30040]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 43 KiB/s rd, 2.9 KiB/s wr, 103 op/s 2024-03-21T13:26:29.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:29 smithi097 ceph-mon[39416]: Deploying daemon prometheus.a on smithi144 2024-03-21T13:26:29.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:29 smithi097 ceph-mon[39416]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 43 KiB/s rd, 2.9 KiB/s wr, 103 op/s 2024-03-21T13:26:29.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:29 smithi144 ceph-mon[33633]: Deploying daemon prometheus.a on smithi144 2024-03-21T13:26:29.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:29 smithi144 ceph-mon[33633]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 43 KiB/s rd, 2.9 KiB/s wr, 103 op/s 2024-03-21T13:26:30.319 DEBUG:teuthology.orchestra.run.smithi097:alertmanager.a> sudo journalctl -f -n 0 -u ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@alertmanager.a.service 2024-03-21T13:26:30.322 INFO:tasks.cephadm:Adding grafana.a on smithi144 2024-03-21T13:26:30.322 DEBUG:teuthology.orchestra.run.smithi144:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph orch apply grafana '1;smithi144=a' 2024-03-21T13:26:30.572 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T13:26:30.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:30 smithi144 ceph-mon[33633]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi097=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:26:30.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:30 smithi144 ceph-mon[33633]: Saving service alertmanager spec with placement smithi097=a;count:1 2024-03-21T13:26:30.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:30 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:30.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:30 smithi097 ceph-mon[30040]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi097=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:26:30.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:30 smithi097 ceph-mon[30040]: Saving service alertmanager spec with placement smithi097=a;count:1 2024-03-21T13:26:30.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:30 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:30.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:30 smithi097 ceph-mon[39416]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi097=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:26:30.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:30 smithi097 ceph-mon[39416]: Saving service alertmanager spec with placement smithi097=a;count:1 2024-03-21T13:26:30.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:30 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:31.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:31 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2024-03-21T13:26:31.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:31 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2024-03-21T13:26:31.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:31 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2024-03-21T13:26:31.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:31 smithi097 ceph-mon[30040]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 33 KiB/s rd, 2.1 KiB/s wr, 79 op/s 2024-03-21T13:26:31.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:31 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:31.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:31 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2024-03-21T13:26:31.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:31 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2024-03-21T13:26:31.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:31 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2024-03-21T13:26:31.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:31 smithi097 ceph-mon[39416]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 33 KiB/s rd, 2.1 KiB/s wr, 79 op/s 2024-03-21T13:26:31.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:31 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:31.874 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:31 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2024-03-21T13:26:31.874 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:31 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2024-03-21T13:26:31.874 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:31 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2024-03-21T13:26:31.874 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:31 smithi144 ceph-mon[33633]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 33 KiB/s rd, 2.1 KiB/s wr, 79 op/s 2024-03-21T13:26:31.874 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:31 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:32.730 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:32 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]': finished 2024-03-21T13:26:32.730 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:32 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]': finished 2024-03-21T13:26:32.730 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:32 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]': finished 2024-03-21T13:26:32.730 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:32 smithi144 ceph-mon[33633]: osdmap e69: 8 total, 8 up, 8 in 2024-03-21T13:26:32.766 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.728+0000 7f74f9bd9700 1 -- 172.21.15.144:0/3692261396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74f4103d60 msgr2=0x7f74f410a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:32.766 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.728+0000 7f74f9bd9700 1 --2- 172.21.15.144:0/3692261396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74f4103d60 0x7f74f410a610 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f74ec00b0d0 tx=0x7f74ec00b3e0 comp rx=0 tx=0).stop 2024-03-21T13:26:32.766 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.728+0000 7f74f9bd9700 1 -- 172.21.15.144:0/3692261396 shutdown_connections 2024-03-21T13:26:32.766 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.728+0000 7f74f9bd9700 1 --2- 172.21.15.144:0/3692261396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74f4103d60 0x7f74f410a610 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:32.766 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.728+0000 7f74f9bd9700 1 --2- 172.21.15.144:0/3692261396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74f41021c0 0x7f74f41025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:32.767 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.728+0000 7f74f9bd9700 1 --2- 172.21.15.144:0/3692261396 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f74f41033c0 0x7f74f4103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:32.767 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.728+0000 7f74f9bd9700 1 -- 172.21.15.144:0/3692261396 >> 172.21.15.144:0/3692261396 conn(0x7f74f40fd7a0 msgr2=0x7f74f40ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:26:32.767 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.728+0000 7f74f9bd9700 1 -- 172.21.15.144:0/3692261396 shutdown_connections 2024-03-21T13:26:32.767 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.728+0000 7f74f9bd9700 1 -- 172.21.15.144:0/3692261396 wait complete. 2024-03-21T13:26:32.767 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.729+0000 7f74f9bd9700 1 Processor -- start 2024-03-21T13:26:32.767 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.729+0000 7f74f9bd9700 1 -- start start 2024-03-21T13:26:32.767 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.729+0000 7f74f9bd9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74f41021c0 0x7f74f4075ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:32.767 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.729+0000 7f74f9bd9700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74f41033c0 0x7f74f4074580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:32.767 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.729+0000 7f74f9bd9700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f74f4103d60 0x7f74f4074ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:32.767 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.729+0000 7f74f9bd9700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f410d550 con 0x7f74f41033c0 2024-03-21T13:26:32.767 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.729+0000 7f74f9bd9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f410d3d0 con 0x7f74f41021c0 2024-03-21T13:26:32.767 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.729+0000 7f74f9bd9700 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74f410d6d0 con 0x7f74f4103d60 2024-03-21T13:26:32.768 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.729+0000 7f74f37fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74f41021c0 0x7f74f4075ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:32.768 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.729+0000 7f74f37fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74f41021c0 0x7f74f4075ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:48806/0 (socket says 172.21.15.144:48806) 2024-03-21T13:26:32.768 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.729+0000 7f74f37fe700 1 -- 172.21.15.144:0/3196274848 learned_addr learned my addr 172.21.15.144:0/3196274848 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-21T13:26:32.768 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.729+0000 7f74f3fff700 1 --2- 172.21.15.144:0/3196274848 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f74f4103d60 0x7f74f4074ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:32.768 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.730+0000 7f74eaffd700 1 --2- 172.21.15.144:0/3196274848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74f41033c0 0x7f74f4074580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:32.768 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.730+0000 7f74f37fe700 1 -- 172.21.15.144:0/3196274848 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f74f4103d60 msgr2=0x7f74f4074ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:32.768 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.730+0000 7f74f37fe700 1 --2- 172.21.15.144:0/3196274848 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f74f4103d60 0x7f74f4074ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:32.768 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.730+0000 7f74f37fe700 1 -- 172.21.15.144:0/3196274848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74f41033c0 msgr2=0x7f74f4074580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:32.768 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.730+0000 7f74f37fe700 1 --2- 172.21.15.144:0/3196274848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74f41033c0 0x7f74f4074580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:32.768 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.730+0000 7f74f37fe700 1 -- 172.21.15.144:0/3196274848 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f4075390 con 0x7f74f41021c0 2024-03-21T13:26:32.768 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.730+0000 7f74f37fe700 1 --2- 172.21.15.144:0/3196274848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74f41021c0 0x7f74f4075ee0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f74e400b500 tx=0x7f74e400b810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:26:32.768 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.730+0000 7f74f17fa700 1 -- 172.21.15.144:0/3196274848 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e4015020 con 0x7f74f41021c0 2024-03-21T13:26:32.768 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.730+0000 7f74f17fa700 1 -- 172.21.15.144:0/3196274848 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f74e4011ce0 con 0x7f74f41021c0 2024-03-21T13:26:32.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.730+0000 7f74f9bd9700 1 -- 172.21.15.144:0/3196274848 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74f41a1cf0 con 0x7f74f41021c0 2024-03-21T13:26:32.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.730+0000 7f74f17fa700 1 -- 172.21.15.144:0/3196274848 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e4010c50 con 0x7f74f41021c0 2024-03-21T13:26:32.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.730+0000 7f74f9bd9700 1 -- 172.21.15.144:0/3196274848 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74f41a21b0 con 0x7f74f41021c0 2024-03-21T13:26:32.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.731+0000 7f74f9bd9700 1 -- 172.21.15.144:0/3196274848 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74f4068e50 con 0x7f74f41021c0 2024-03-21T13:26:32.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.732+0000 7f74f17fa700 1 -- 172.21.15.144:0/3196274848 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 16) v1 ==== 92146+0+0 (secure 0 0 0) 0x7f74e4018020 con 0x7f74f41021c0 2024-03-21T13:26:32.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.732+0000 7f74f17fa700 1 --2- 172.21.15.144:0/3196274848 >> v2:172.21.15.97:6800/1943212787 conn(0x7f74e006e2a0 0x7f74e0070760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:32.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.732+0000 7f74f17fa700 1 -- 172.21.15.144:0/3196274848 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f74e408d9a0 con 0x7f74f41021c0 2024-03-21T13:26:32.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.732+0000 7f74eaffd700 1 --2- 172.21.15.144:0/3196274848 >> v2:172.21.15.97:6800/1943212787 conn(0x7f74e006e2a0 0x7f74e0070760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:32.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.733+0000 7f74eaffd700 1 --2- 172.21.15.144:0/3196274848 >> v2:172.21.15.97:6800/1943212787 conn(0x7f74e006e2a0 0x7f74e0070760 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f74dc009a10 tx=0x7f74dc009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:26:32.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.737+0000 7f74f17fa700 1 -- 172.21.15.144:0/3196274848 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f74e401f020 con 0x7f74f41021c0 2024-03-21T13:26:32.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:32 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]': finished 2024-03-21T13:26:32.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:32 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]': finished 2024-03-21T13:26:32.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:32 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]': finished 2024-03-21T13:26:32.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:32 smithi097 ceph-mon[39416]: osdmap e69: 8 total, 8 up, 8 in 2024-03-21T13:26:32.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:32 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]': finished 2024-03-21T13:26:32.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:32 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]': finished 2024-03-21T13:26:32.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:32 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]': finished 2024-03-21T13:26:32.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:32 smithi097 ceph-mon[30040]: osdmap e69: 8 total, 8 up, 8 in 2024-03-21T13:26:32.957 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.955+0000 7f74f9bd9700 1 -- 172.21.15.144:0/3196274848 --> v2:172.21.15.97:6800/1943212787 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi144=a", "target": ["mon-mgr", ""]}) v1 -- 0x7f74f40625d0 con 0x7f74e006e2a0 2024-03-21T13:26:32.964 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.963+0000 7f74f17fa700 1 -- 172.21.15.144:0/3196274848 <== mgr.14152 v2:172.21.15.97:6800/1943212787 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f74f40625d0 con 0x7f74e006e2a0 2024-03-21T13:26:32.964 INFO:teuthology.orchestra.run.smithi144.stdout:Scheduled grafana update... 2024-03-21T13:26:32.966 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.965+0000 7f74f9bd9700 1 -- 172.21.15.144:0/3196274848 >> v2:172.21.15.97:6800/1943212787 conn(0x7f74e006e2a0 msgr2=0x7f74e0070760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:32.966 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.965+0000 7f74f9bd9700 1 --2- 172.21.15.144:0/3196274848 >> v2:172.21.15.97:6800/1943212787 conn(0x7f74e006e2a0 0x7f74e0070760 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f74dc009a10 tx=0x7f74dc009190 comp rx=0 tx=0).stop 2024-03-21T13:26:32.967 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.965+0000 7f74f9bd9700 1 -- 172.21.15.144:0/3196274848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74f41021c0 msgr2=0x7f74f4075ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:32.967 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.965+0000 7f74f9bd9700 1 --2- 172.21.15.144:0/3196274848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74f41021c0 0x7f74f4075ee0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f74e400b500 tx=0x7f74e400b810 comp rx=0 tx=0).stop 2024-03-21T13:26:32.967 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.965+0000 7f74f9bd9700 1 -- 172.21.15.144:0/3196274848 shutdown_connections 2024-03-21T13:26:32.967 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.965+0000 7f74f9bd9700 1 --2- 172.21.15.144:0/3196274848 >> v2:172.21.15.97:6800/1943212787 conn(0x7f74e006e2a0 0x7f74e0070760 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:32.967 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.965+0000 7f74f9bd9700 1 --2- 172.21.15.144:0/3196274848 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f74f4103d60 0x7f74f4074ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:32.967 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.965+0000 7f74f9bd9700 1 --2- 172.21.15.144:0/3196274848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74f41021c0 0x7f74f4075ee0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:32.967 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.965+0000 7f74f9bd9700 1 --2- 172.21.15.144:0/3196274848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74f41033c0 0x7f74f4074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:32.967 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.965+0000 7f74f9bd9700 1 -- 172.21.15.144:0/3196274848 >> 172.21.15.144:0/3196274848 conn(0x7f74f40fd7a0 msgr2=0x7f74f40ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:26:32.967 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.965+0000 7f74f9bd9700 1 -- 172.21.15.144:0/3196274848 shutdown_connections 2024-03-21T13:26:32.967 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:32.965+0000 7f74f9bd9700 1 -- 172.21.15.144:0/3196274848 wait complete. 2024-03-21T13:26:33.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:26:33 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:26:33.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:33 smithi144 ceph-mon[33633]: osdmap e70: 8 total, 8 up, 8 in 2024-03-21T13:26:33.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:33 smithi144 ceph-mon[33633]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi144=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:26:33.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:33 smithi144 ceph-mon[33633]: Saving service grafana spec with placement smithi144=a;count:1 2024-03-21T13:26:33.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:33 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:33.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:33 smithi144 ceph-mon[33633]: pgmap v211: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 127 B/s wr, 2 op/s 2024-03-21T13:26:33.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:33 smithi097 ceph-mon[39416]: osdmap e70: 8 total, 8 up, 8 in 2024-03-21T13:26:33.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:33 smithi097 ceph-mon[39416]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi144=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:26:33.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:33 smithi097 ceph-mon[39416]: Saving service grafana spec with placement smithi144=a;count:1 2024-03-21T13:26:33.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:33 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:33.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:33 smithi097 ceph-mon[39416]: pgmap v211: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 127 B/s wr, 2 op/s 2024-03-21T13:26:33.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:33 smithi097 ceph-mon[30040]: osdmap e70: 8 total, 8 up, 8 in 2024-03-21T13:26:33.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:33 smithi097 ceph-mon[30040]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi144=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T13:26:33.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:33 smithi097 ceph-mon[30040]: Saving service grafana spec with placement smithi144=a;count:1 2024-03-21T13:26:33.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:33 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:33.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:33 smithi097 ceph-mon[30040]: pgmap v211: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 127 B/s wr, 2 op/s 2024-03-21T13:26:34.112 DEBUG:teuthology.orchestra.run.smithi144:grafana.a> sudo journalctl -f -n 0 -u ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@grafana.a.service 2024-03-21T13:26:34.115 INFO:tasks.cephadm:Setting up client nodes... 2024-03-21T13:26:34.115 DEBUG:teuthology.orchestra.run.smithi097:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-21T13:26:34.361 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T13:26:35.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:34 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:26:35.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:34 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:26:35.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:34 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:26:36.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:35 smithi144 ceph-mon[33633]: pgmap v212: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 127 B/s wr, 2 op/s; 27 B/s, 1 objects/s recovering 2024-03-21T13:26:36.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.180+0000 7f866455f700 1 -- 172.21.15.97:0/3772603051 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f865c102310 msgr2=0x7f865c102770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:36.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.180+0000 7f866455f700 1 --2- 172.21.15.97:0/3772603051 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f865c102310 0x7f865c102770 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f864c009990 tx=0x7f864c009ca0 comp rx=0 tx=0).stop 2024-03-21T13:26:36.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.180+0000 7f866455f700 1 -- 172.21.15.97:0/3772603051 shutdown_connections 2024-03-21T13:26:36.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.180+0000 7f866455f700 1 --2- 172.21.15.97:0/3772603051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f865c102cb0 0x7f865c109560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:36.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.180+0000 7f866455f700 1 --2- 172.21.15.97:0/3772603051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f865c101110 0x7f865c101530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:36.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.180+0000 7f866455f700 1 --2- 172.21.15.97:0/3772603051 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f865c102310 0x7f865c102770 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:36.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.180+0000 7f866455f700 1 -- 172.21.15.97:0/3772603051 >> 172.21.15.97:0/3772603051 conn(0x7f865c0fc6b0 msgr2=0x7f865c0feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:26:36.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.180+0000 7f866455f700 1 -- 172.21.15.97:0/3772603051 shutdown_connections 2024-03-21T13:26:36.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.180+0000 7f866455f700 1 -- 172.21.15.97:0/3772603051 wait complete. 2024-03-21T13:26:36.182 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:35 smithi097 ceph-mon[39416]: pgmap v212: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 127 B/s wr, 2 op/s; 27 B/s, 1 objects/s recovering 2024-03-21T13:26:36.182 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:35 smithi097 ceph-mon[30040]: pgmap v212: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 127 B/s wr, 2 op/s; 27 B/s, 1 objects/s recovering 2024-03-21T13:26:36.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.181+0000 7f866455f700 1 Processor -- start 2024-03-21T13:26:36.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.181+0000 7f866455f700 1 -- start start 2024-03-21T13:26:36.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.181+0000 7f866455f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f865c101110 0x7f865c196880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:36.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.181+0000 7f866455f700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f865c102310 0x7f865c196dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:36.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.181+0000 7f866455f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f865c102cb0 0x7f865c19de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:36.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.181+0000 7f866455f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865c10c480 con 0x7f865c101110 2024-03-21T13:26:36.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.181+0000 7f866455f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865c10c300 con 0x7f865c102cb0 2024-03-21T13:26:36.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.181+0000 7f866455f700 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f865c10c600 con 0x7f865c102310 2024-03-21T13:26:36.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.181+0000 7f8661afa700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f865c102310 0x7f865c196dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:36.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.181+0000 7f86622fb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f865c101110 0x7f865c196880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:36.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.181+0000 7f8661afa700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f865c102310 0x7f865c196dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.97:36028/0 (socket says 172.21.15.97:36028) 2024-03-21T13:26:36.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.181+0000 7f8661afa700 1 -- 172.21.15.97:0/2858705409 learned_addr learned my addr 172.21.15.97:0/2858705409 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:26:36.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.182+0000 7f8662afc700 1 --2- 172.21.15.97:0/2858705409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f865c102cb0 0x7f865c19de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:36.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.182+0000 7f86622fb700 1 -- 172.21.15.97:0/2858705409 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f865c102310 msgr2=0x7f865c196dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:36.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.182+0000 7f86622fb700 1 --2- 172.21.15.97:0/2858705409 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f865c102310 0x7f865c196dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:36.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.182+0000 7f86622fb700 1 -- 172.21.15.97:0/2858705409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f865c102cb0 msgr2=0x7f865c19de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:36.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.182+0000 7f86622fb700 1 --2- 172.21.15.97:0/2858705409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f865c102cb0 0x7f865c19de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:36.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.182+0000 7f86622fb700 1 -- 172.21.15.97:0/2858705409 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f865c19e4e0 con 0x7f865c101110 2024-03-21T13:26:36.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.182+0000 7f8661afa700 1 --2- 172.21.15.97:0/2858705409 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f865c102310 0x7f865c196dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T13:26:36.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.182+0000 7f8662afc700 1 --2- 172.21.15.97:0/2858705409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f865c102cb0 0x7f865c19de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T13:26:36.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.182+0000 7f86622fb700 1 --2- 172.21.15.97:0/2858705409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f865c101110 0x7f865c196880 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f865800e9f0 tx=0x7f865800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:26:36.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.182+0000 7f86537fe700 1 -- 172.21.15.97:0/2858705409 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865801b070 con 0x7f865c101110 2024-03-21T13:26:36.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.182+0000 7f86537fe700 1 -- 172.21.15.97:0/2858705409 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f8658016430 con 0x7f865c101110 2024-03-21T13:26:36.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.182+0000 7f86537fe700 1 -- 172.21.15.97:0/2858705409 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8658014590 con 0x7f865c101110 2024-03-21T13:26:36.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.182+0000 7f866455f700 1 -- 172.21.15.97:0/2858705409 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f865c06a930 con 0x7f865c101110 2024-03-21T13:26:36.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.182+0000 7f866455f700 1 -- 172.21.15.97:0/2858705409 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f865c06ae20 con 0x7f865c101110 2024-03-21T13:26:36.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.184+0000 7f866455f700 1 -- 172.21.15.97:0/2858705409 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f865c10cbe0 con 0x7f865c101110 2024-03-21T13:26:36.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.184+0000 7f86537fe700 1 -- 172.21.15.97:0/2858705409 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 92146+0+0 (secure 0 0 0) 0x7f865800db80 con 0x7f865c101110 2024-03-21T13:26:36.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.184+0000 7f86537fe700 1 --2- 172.21.15.97:0/2858705409 >> v2:172.21.15.97:6800/1943212787 conn(0x7f864806e340 0x7f8648070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:36.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.185+0000 7f86537fe700 1 -- 172.21.15.97:0/2858705409 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f865808d980 con 0x7f865c101110 2024-03-21T13:26:36.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.185+0000 7f8661afa700 1 --2- 172.21.15.97:0/2858705409 >> v2:172.21.15.97:6800/1943212787 conn(0x7f864806e340 0x7f8648070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:36.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.185+0000 7f8661afa700 1 --2- 172.21.15.97:0/2858705409 >> v2:172.21.15.97:6800/1943212787 conn(0x7f864806e340 0x7f8648070800 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f864c00b2e0 tx=0x7f864c01e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:26:36.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.190+0000 7f86537fe700 1 -- 172.21.15.97:0/2858705409 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f865805b0d0 con 0x7f865c101110 2024-03-21T13:26:36.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.465+0000 7f866455f700 1 -- 172.21.15.97:0/2858705409 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f865c1080a0 con 0x7f865c101110 2024-03-21T13:26:36.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.467+0000 7f86537fe700 1 -- 172.21.15.97:0/2858705409 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v16) v1 ==== 170+0+59 (secure 0 0 0) 0x7f865805a9d0 con 0x7f865c101110 2024-03-21T13:26:36.467 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2024-03-21T13:26:36.467 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQCMNfxlxtDOGxAAkGHTjXTfUe2B9X44HWiaHQ== 2024-03-21T13:26:36.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.469+0000 7f866455f700 1 -- 172.21.15.97:0/2858705409 >> v2:172.21.15.97:6800/1943212787 conn(0x7f864806e340 msgr2=0x7f8648070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:36.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.469+0000 7f866455f700 1 --2- 172.21.15.97:0/2858705409 >> v2:172.21.15.97:6800/1943212787 conn(0x7f864806e340 0x7f8648070800 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f864c00b2e0 tx=0x7f864c01e040 comp rx=0 tx=0).stop 2024-03-21T13:26:36.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.469+0000 7f866455f700 1 -- 172.21.15.97:0/2858705409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f865c101110 msgr2=0x7f865c196880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:36.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.469+0000 7f866455f700 1 --2- 172.21.15.97:0/2858705409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f865c101110 0x7f865c196880 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f865800e9f0 tx=0x7f865800ed00 comp rx=0 tx=0).stop 2024-03-21T13:26:36.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.469+0000 7f866455f700 1 -- 172.21.15.97:0/2858705409 shutdown_connections 2024-03-21T13:26:36.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.469+0000 7f866455f700 1 --2- 172.21.15.97:0/2858705409 >> v2:172.21.15.97:6800/1943212787 conn(0x7f864806e340 0x7f8648070800 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:36.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.469+0000 7f866455f700 1 --2- 172.21.15.97:0/2858705409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f865c102cb0 0x7f865c19de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:36.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.469+0000 7f866455f700 1 --2- 172.21.15.97:0/2858705409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f865c101110 0x7f865c196880 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:36.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.469+0000 7f866455f700 1 --2- 172.21.15.97:0/2858705409 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f865c102310 0x7f865c196dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:36.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.469+0000 7f866455f700 1 -- 172.21.15.97:0/2858705409 >> 172.21.15.97:0/2858705409 conn(0x7f865c0fc6b0 msgr2=0x7f865c105540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:26:36.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.469+0000 7f866455f700 1 -- 172.21.15.97:0/2858705409 shutdown_connections 2024-03-21T13:26:36.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:36.469+0000 7f866455f700 1 -- 172.21.15.97:0/2858705409 wait complete. 2024-03-21T13:26:37.223 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:36 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2858705409' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-21T13:26:37.224 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:36 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2858705409' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-21T13:26:37.224 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:36 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2858705409' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-21T13:26:37.224 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:36 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2858705409' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-21T13:26:37.251 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-21T13:26:37.251 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-21T13:26:37.251 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-21T13:26:37.295 DEBUG:teuthology.orchestra.run.smithi144:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-21T13:26:37.320 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:36 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2858705409' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-21T13:26:37.320 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:36 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2858705409' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-21T13:26:37.851 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T13:26:38.238 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:37 smithi144 ceph-mon[33633]: pgmap v213: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 27 B/s, 1 objects/s recovering 2024-03-21T13:26:38.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:37 smithi097 ceph-mon[30040]: pgmap v213: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 27 B/s, 1 objects/s recovering 2024-03-21T13:26:38.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:37 smithi097 ceph-mon[39416]: pgmap v213: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 27 B/s, 1 objects/s recovering 2024-03-21T13:26:40.472 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:40 smithi144 ceph-mon[33633]: pgmap v214: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 41 B/s, 2 objects/s recovering 2024-03-21T13:26:40.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:40 smithi097 ceph-mon[39416]: pgmap v214: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 41 B/s, 2 objects/s recovering 2024-03-21T13:26:40.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:40 smithi097 ceph-mon[30040]: pgmap v214: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 41 B/s, 2 objects/s recovering 2024-03-21T13:26:42.494 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:42 smithi144 ceph-mon[33633]: pgmap v215: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 33 B/s, 1 objects/s recovering 2024-03-21T13:26:42.546 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.544+0000 7f702763e700 1 -- 172.21.15.144:0/2545594539 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7020102c70 msgr2=0x7f7020109520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:42.546 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.544+0000 7f702763e700 1 --2- 172.21.15.144:0/2545594539 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7020102c70 0x7f7020109520 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f701800b0d0 tx=0x7f701800b3e0 comp rx=0 tx=0).stop 2024-03-21T13:26:42.546 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.544+0000 7f702763e700 1 -- 172.21.15.144:0/2545594539 shutdown_connections 2024-03-21T13:26:42.546 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.544+0000 7f702763e700 1 --2- 172.21.15.144:0/2545594539 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7020102c70 0x7f7020109520 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:42.546 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.544+0000 7f702763e700 1 --2- 172.21.15.144:0/2545594539 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f70201010d0 0x7f70201014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:42.546 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.544+0000 7f702763e700 1 --2- 172.21.15.144:0/2545594539 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70201022d0 0x7f7020102730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:42.546 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.544+0000 7f702763e700 1 -- 172.21.15.144:0/2545594539 >> 172.21.15.144:0/2545594539 conn(0x7f70200fc690 msgr2=0x7f70200feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:26:42.546 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.544+0000 7f702763e700 1 -- 172.21.15.144:0/2545594539 shutdown_connections 2024-03-21T13:26:42.546 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.544+0000 7f702763e700 1 -- 172.21.15.144:0/2545594539 wait complete. 2024-03-21T13:26:42.547 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.544+0000 7f702763e700 1 Processor -- start 2024-03-21T13:26:42.547 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.544+0000 7f702763e700 1 -- start start 2024-03-21T13:26:42.547 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.545+0000 7f702763e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70201010d0 0x7f7020196890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:42.547 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.545+0000 7f702763e700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f70201022d0 0x7f7020196dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:42.547 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.545+0000 7f702763e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7020102c70 0x7f702019de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:42.547 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.545+0000 7f702763e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f702010c400 con 0x7f7020102c70 2024-03-21T13:26:42.547 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.545+0000 7f702763e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f702010c280 con 0x7f70201010d0 2024-03-21T13:26:42.547 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.545+0000 7f702763e700 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f702010c580 con 0x7f70201022d0 2024-03-21T13:26:42.547 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.545+0000 7f70253da700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70201010d0 0x7f7020196890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:42.547 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.545+0000 7f70253da700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70201010d0 0x7f7020196890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:52288/0 (socket says 172.21.15.144:52288) 2024-03-21T13:26:42.547 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.545+0000 7f70253da700 1 -- 172.21.15.144:0/3237017066 learned_addr learned my addr 172.21.15.144:0/3237017066 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-21T13:26:42.547 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.545+0000 7f7024bd9700 1 --2- 172.21.15.144:0/3237017066 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f70201022d0 0x7f7020196dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:42.547 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.545+0000 7f7025bdb700 1 --2- 172.21.15.144:0/3237017066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7020102c70 0x7f702019de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:42.547 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.545+0000 7f70253da700 1 -- 172.21.15.144:0/3237017066 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f70201022d0 msgr2=0x7f7020196dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:42.548 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.545+0000 7f70253da700 1 --2- 172.21.15.144:0/3237017066 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f70201022d0 0x7f7020196dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:42.548 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.545+0000 7f70253da700 1 -- 172.21.15.144:0/3237017066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7020102c70 msgr2=0x7f702019de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:42.548 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.545+0000 7f70253da700 1 --2- 172.21.15.144:0/3237017066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7020102c70 0x7f702019de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:42.548 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.545+0000 7f70253da700 1 -- 172.21.15.144:0/3237017066 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f702019e4f0 con 0x7f70201010d0 2024-03-21T13:26:42.548 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.545+0000 7f70253da700 1 --2- 172.21.15.144:0/3237017066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70201010d0 0x7f7020196890 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f701000d980 tx=0x7f701000dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:26:42.548 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.545+0000 7f70167fc700 1 -- 172.21.15.144:0/3237017066 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7010016070 con 0x7f70201010d0 2024-03-21T13:26:42.548 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.545+0000 7f70167fc700 1 -- 172.21.15.144:0/3237017066 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f701000fd40 con 0x7f70201010d0 2024-03-21T13:26:42.548 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.545+0000 7f70167fc700 1 -- 172.21.15.144:0/3237017066 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7010019360 con 0x7f70201010d0 2024-03-21T13:26:42.548 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.546+0000 7f702763e700 1 -- 172.21.15.144:0/3237017066 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f702019e7e0 con 0x7f70201010d0 2024-03-21T13:26:42.548 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.546+0000 7f702763e700 1 -- 172.21.15.144:0/3237017066 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f702019ecd0 con 0x7f70201010d0 2024-03-21T13:26:42.549 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.546+0000 7f702763e700 1 -- 172.21.15.144:0/3237017066 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70200506a0 con 0x7f70201010d0 2024-03-21T13:26:42.549 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.547+0000 7f70167fc700 1 -- 172.21.15.144:0/3237017066 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 16) v1 ==== 92146+0+0 (secure 0 0 0) 0x7f7010019070 con 0x7f70201010d0 2024-03-21T13:26:42.549 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.547+0000 7f70167fc700 1 --2- 172.21.15.144:0/3237017066 >> v2:172.21.15.97:6800/1943212787 conn(0x7f700c06e340 0x7f700c070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:42.549 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.548+0000 7f70167fc700 1 -- 172.21.15.144:0/3237017066 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f701008dca0 con 0x7f70201010d0 2024-03-21T13:26:42.549 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.548+0000 7f7024bd9700 1 --2- 172.21.15.144:0/3237017066 >> v2:172.21.15.97:6800/1943212787 conn(0x7f700c06e340 0x7f700c070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:42.549 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.548+0000 7f7024bd9700 1 --2- 172.21.15.144:0/3237017066 >> v2:172.21.15.97:6800/1943212787 conn(0x7f700c06e340 0x7f700c070800 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f701c009be0 tx=0x7f701c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:26:42.552 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.552+0000 7f70167fc700 1 -- 172.21.15.144:0/3237017066 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7010092070 con 0x7f70201010d0 2024-03-21T13:26:42.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:42 smithi097 ceph-mon[39416]: pgmap v215: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 33 B/s, 1 objects/s recovering 2024-03-21T13:26:42.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:42 smithi097 ceph-mon[30040]: pgmap v215: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 33 B/s, 1 objects/s recovering 2024-03-21T13:26:42.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.848+0000 7f702763e700 1 -- 172.21.15.144:0/3237017066 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f702002de10 con 0x7f70201010d0 2024-03-21T13:26:42.853 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.853+0000 7f70167fc700 1 -- 172.21.15.144:0/3237017066 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) v1 ==== 170+0+59 (secure 0 0 0) 0x7f701005b3f0 con 0x7f70201010d0 2024-03-21T13:26:42.853 INFO:teuthology.orchestra.run.smithi144.stdout:[client.1] 2024-03-21T13:26:42.853 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQCSNfxlauyvMhAAvQIrs878Sw2fCW3V2ywgGw== 2024-03-21T13:26:42.855 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.855+0000 7f702763e700 1 -- 172.21.15.144:0/3237017066 >> v2:172.21.15.97:6800/1943212787 conn(0x7f700c06e340 msgr2=0x7f700c070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:42.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.855+0000 7f702763e700 1 --2- 172.21.15.144:0/3237017066 >> v2:172.21.15.97:6800/1943212787 conn(0x7f700c06e340 0x7f700c070800 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f701c009be0 tx=0x7f701c009260 comp rx=0 tx=0).stop 2024-03-21T13:26:42.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.855+0000 7f702763e700 1 -- 172.21.15.144:0/3237017066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70201010d0 msgr2=0x7f7020196890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:42.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.855+0000 7f702763e700 1 --2- 172.21.15.144:0/3237017066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70201010d0 0x7f7020196890 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f701000d980 tx=0x7f701000dc90 comp rx=0 tx=0).stop 2024-03-21T13:26:42.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.855+0000 7f702763e700 1 -- 172.21.15.144:0/3237017066 shutdown_connections 2024-03-21T13:26:42.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.855+0000 7f702763e700 1 --2- 172.21.15.144:0/3237017066 >> v2:172.21.15.97:6800/1943212787 conn(0x7f700c06e340 0x7f700c070800 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:42.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.855+0000 7f702763e700 1 --2- 172.21.15.144:0/3237017066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7020102c70 0x7f702019de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:42.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.855+0000 7f702763e700 1 --2- 172.21.15.144:0/3237017066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70201010d0 0x7f7020196890 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:42.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.855+0000 7f702763e700 1 --2- 172.21.15.144:0/3237017066 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f70201022d0 0x7f7020196dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:42.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.855+0000 7f702763e700 1 -- 172.21.15.144:0/3237017066 >> 172.21.15.144:0/3237017066 conn(0x7f70200fc690 msgr2=0x7f7020105500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:26:42.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.855+0000 7f702763e700 1 -- 172.21.15.144:0/3237017066 shutdown_connections 2024-03-21T13:26:42.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-21T13:26:42.855+0000 7f702763e700 1 -- 172.21.15.144:0/3237017066 wait complete. 2024-03-21T13:26:43.475 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:43 smithi144 ceph-mon[33633]: from='client.? 172.21.15.144:0/3237017066' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-21T13:26:43.475 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:43 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-21T13:26:43.475 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:43 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-21T13:26:43.476 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:26:43 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:26:43.501 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-21T13:26:43.501 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-03-21T13:26:43.501 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-21T13:26:43.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:43 smithi097 ceph-mon[39416]: from='client.? 172.21.15.144:0/3237017066' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-21T13:26:43.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:43 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-21T13:26:43.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:43 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-21T13:26:43.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:43 smithi097 ceph-mon[30040]: from='client.? 172.21.15.144:0/3237017066' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-21T13:26:43.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:43 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-21T13:26:43.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:43 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-21T13:26:43.586 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-21T13:26:43.586 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-21T13:26:43.586 DEBUG:teuthology.orchestra.run.smithi097:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph mgr dump --format=json 2024-03-21T13:26:43.803 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:26:43 smithi144 podman[64147]: 2024-03-21 13:26:43.64291658 +0000 UTC m=+0.576023967 container create 6f097bbd054bd04ec33556acd6c318241eb390e8fbd68fe916de113dbe4f7935 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-21T13:26:43.838 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T13:26:44.191 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:26:43 smithi144 podman[64147]: 2024-03-21 13:26:43.920388999 +0000 UTC m=+0.853496398 container init 6f097bbd054bd04ec33556acd6c318241eb390e8fbd68fe916de113dbe4f7935 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-21T13:26:44.192 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:26:43 smithi144 podman[64147]: 2024-03-21 13:26:43.925706712 +0000 UTC m=+0.858814100 container start 6f097bbd054bd04ec33556acd6c318241eb390e8fbd68fe916de113dbe4f7935 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-21T13:26:44.192 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:26:43 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[64191]: ts=2024-03-21T13:26:43.949Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-03-21T13:26:44.192 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:26:43 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[64191]: ts=2024-03-21T13:26:43.949Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-03-21T13:26:44.192 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:26:43 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[64191]: ts=2024-03-21T13:26:43.949Z caller=main.go:570 level=info host_details="(Linux 5.14.0-430.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024 x86_64 smithi144 (none))" 2024-03-21T13:26:44.192 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:26:43 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[64191]: ts=2024-03-21T13:26:43.949Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-03-21T13:26:44.192 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:26:43 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[64191]: ts=2024-03-21T13:26:43.949Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-03-21T13:26:44.192 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:26:43 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[64191]: ts=2024-03-21T13:26:43.950Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-03-21T13:26:44.192 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:26:43 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[64191]: ts=2024-03-21T13:26:43.951Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-03-21T13:26:44.192 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:26:43 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[64191]: ts=2024-03-21T13:26:43.952Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-03-21T13:26:44.192 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:26:43 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[64191]: ts=2024-03-21T13:26:43.952Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-03-21T13:26:44.192 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:26:43 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[64191]: ts=2024-03-21T13:26:43.954Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-03-21T13:26:44.193 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:26:43 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[64191]: ts=2024-03-21T13:26:43.954Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.39µs 2024-03-21T13:26:44.193 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:26:43 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[64191]: ts=2024-03-21T13:26:43.954Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-03-21T13:26:44.193 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:26:43 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[64191]: ts=2024-03-21T13:26:43.954Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-03-21T13:26:44.193 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:26:43 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[64191]: ts=2024-03-21T13:26:43.954Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=16.549µs wal_replay_duration=182.509µs wbl_replay_duration=111ns total_replay_duration=220.564µs 2024-03-21T13:26:44.193 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:26:43 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[64191]: ts=2024-03-21T13:26:43.955Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-03-21T13:26:44.193 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:26:43 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[64191]: ts=2024-03-21T13:26:43.955Z caller=main.go:1029 level=info msg="TSDB started" 2024-03-21T13:26:44.193 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:26:43 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[64191]: ts=2024-03-21T13:26:43.955Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-03-21T13:26:44.193 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:26:43 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[64191]: ts=2024-03-21T13:26:43.976Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=20.992516ms db_storage=732ns remote_storage=997ns web_handler=274ns query_engine=601ns scrape=500.781µs scrape_sd=148.416µs notify=2.068µs notify_sd=5.446µs rules=20.050276ms tracing=5.635µs 2024-03-21T13:26:44.193 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:26:43 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[64191]: ts=2024-03-21T13:26:43.976Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-03-21T13:26:44.193 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:26:43 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[64191]: ts=2024-03-21T13:26:43.976Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-03-21T13:26:44.193 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:26:44 smithi144 bash[64147]: 6f097bbd054bd04ec33556acd6c318241eb390e8fbd68fe916de113dbe4f7935 2024-03-21T13:26:44.193 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:26:44 smithi144 systemd[1]: Started Ceph prometheus.a for a5dd1f32-e785-11ee-95cd-87774f69a715. 2024-03-21T13:26:44.277 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:44 smithi097 ceph-mon[39416]: pgmap v216: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s; 30 B/s, 1 objects/s recovering 2024-03-21T13:26:44.277 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:44 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:44.277 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:44 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:44.277 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:44 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:44.277 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:44 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-03-21T13:26:44.278 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:44 smithi097 ceph-mon[30040]: pgmap v216: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s; 30 B/s, 1 objects/s recovering 2024-03-21T13:26:44.278 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:44 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:44.278 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:44 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:44.278 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:44 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:44.278 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:44 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-03-21T13:26:44.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:44 smithi144 ceph-mon[33633]: pgmap v216: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s; 30 B/s, 1 objects/s recovering 2024-03-21T13:26:44.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:44 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:44.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:44 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:44.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:44 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' 2024-03-21T13:26:44.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:44 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-03-21T13:26:45.365 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:45 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:26:45.366 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:45 smithi097 ceph-mon[39416]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-03-21T13:26:45.366 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:45 smithi097 ceph-mon[39416]: mgrmap e17: y(active, since 5m), standbys: x 2024-03-21T13:26:45.366 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:45 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:26:45.366 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:45 smithi097 ceph-mon[30040]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-03-21T13:26:45.366 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:45 smithi097 ceph-mon[30040]: mgrmap e17: y(active, since 5m), standbys: x 2024-03-21T13:26:45.366 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:26:45 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ignoring --setuser ceph since I am not root 2024-03-21T13:26:45.366 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:26:45 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ignoring --setgroup ceph since I am not root 2024-03-21T13:26:45.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:45 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:26:45.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:45 smithi144 ceph-mon[33633]: from='mgr.14152 172.21.15.97:0/2612151641' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-03-21T13:26:45.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:45 smithi144 ceph-mon[33633]: mgrmap e17: y(active, since 5m), standbys: x 2024-03-21T13:26:45.621 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:26:45 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: ignoring --setuser ceph since I am not root 2024-03-21T13:26:45.621 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:26:45 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: ignoring --setgroup ceph since I am not root 2024-03-21T13:26:45.621 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:26:45 smithi144 ceph-mgr[35732]: -- 172.21.15.144:0/1308747216 <== mon.1 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564e209ea5a0 con 0x564e209bf000 2024-03-21T13:26:45.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.622+0000 7f420e3a2700 1 -- 172.21.15.97:0/4107974659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4208101b90 msgr2=0x7f4208105be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:45.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.622+0000 7f420e3a2700 1 --2- 172.21.15.97:0/4107974659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4208101b90 0x7f4208105be0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f42040099c0 tx=0x7f4204009cd0 comp rx=0 tx=0).stop 2024-03-21T13:26:45.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.622+0000 7f420e3a2700 1 -- 172.21.15.97:0/4107974659 shutdown_connections 2024-03-21T13:26:45.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.622+0000 7f420e3a2700 1 --2- 172.21.15.97:0/4107974659 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f42081062c0 0x7f420810de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:45.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.622+0000 7f420e3a2700 1 --2- 172.21.15.97:0/4107974659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42081011e0 0x7f42081015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:45.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.622+0000 7f420e3a2700 1 --2- 172.21.15.97:0/4107974659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4208101b90 0x7f4208105be0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:45.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.622+0000 7f420e3a2700 1 -- 172.21.15.97:0/4107974659 >> 172.21.15.97:0/4107974659 conn(0x7f42080fca70 msgr2=0x7f42080fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:26:45.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.622+0000 7f420e3a2700 1 -- 172.21.15.97:0/4107974659 shutdown_connections 2024-03-21T13:26:45.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.622+0000 7f420e3a2700 1 -- 172.21.15.97:0/4107974659 wait complete. 2024-03-21T13:26:45.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.623+0000 7f420e3a2700 1 Processor -- start 2024-03-21T13:26:45.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.623+0000 7f420e3a2700 1 -- start start 2024-03-21T13:26:45.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.623+0000 7f420e3a2700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f42081011e0 0x7f420819a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:45.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.623+0000 7f420e3a2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4208101b90 0x7f420819a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:45.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.623+0000 7f420e3a2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42081062c0 0x7f420819ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:45.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.623+0000 7f420e3a2700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4208110970 con 0x7f42081062c0 2024-03-21T13:26:45.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.623+0000 7f420e3a2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42081107f0 con 0x7f4208101b90 2024-03-21T13:26:45.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.623+0000 7f420e3a2700 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4208110af0 con 0x7f42081011e0 2024-03-21T13:26:45.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.623+0000 7f420d3a0700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f42081011e0 0x7f420819a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:45.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.623+0000 7f420d3a0700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f42081011e0 0x7f420819a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.97:34502/0 (socket says 172.21.15.97:34502) 2024-03-21T13:26:45.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.623+0000 7f420d3a0700 1 -- 172.21.15.97:0/1536444855 learned_addr learned my addr 172.21.15.97:0/1536444855 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:26:45.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.623+0000 7f420dba1700 1 --2- 172.21.15.97:0/1536444855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42081062c0 0x7f420819ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:45.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.623+0000 7f420cb9f700 1 --2- 172.21.15.97:0/1536444855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4208101b90 0x7f420819a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:45.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.623+0000 7f420d3a0700 1 -- 172.21.15.97:0/1536444855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4208101b90 msgr2=0x7f420819a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:45.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.623+0000 7f420d3a0700 1 --2- 172.21.15.97:0/1536444855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4208101b90 0x7f420819a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:45.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.623+0000 7f420d3a0700 1 -- 172.21.15.97:0/1536444855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42081062c0 msgr2=0x7f420819ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:45.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.623+0000 7f420d3a0700 1 --2- 172.21.15.97:0/1536444855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42081062c0 0x7f420819ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:45.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.623+0000 7f420d3a0700 1 -- 172.21.15.97:0/1536444855 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f420819f150 con 0x7f42081011e0 2024-03-21T13:26:45.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.624+0000 7f420d3a0700 1 --2- 172.21.15.97:0/1536444855 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f42081011e0 0x7f420819a140 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f41f800e9a0 tx=0x7f41f800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:26:45.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.624+0000 7f41fe7fc700 1 -- 172.21.15.97:0/1536444855 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f800c9c0 con 0x7f42081011e0 2024-03-21T13:26:45.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.624+0000 7f41fe7fc700 1 -- 172.21.15.97:0/1536444855 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f41f800cb60 con 0x7f42081011e0 2024-03-21T13:26:45.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.624+0000 7f41fe7fc700 1 -- 172.21.15.97:0/1536444855 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f8018680 con 0x7f42081011e0 2024-03-21T13:26:45.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.624+0000 7f420e3a2700 1 -- 172.21.15.97:0/1536444855 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f420819f440 con 0x7f42081011e0 2024-03-21T13:26:45.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.624+0000 7f420e3a2700 1 -- 172.21.15.97:0/1536444855 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42081a6cc0 con 0x7f42081011e0 2024-03-21T13:26:45.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.625+0000 7f420e3a2700 1 -- 172.21.15.97:0/1536444855 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4208068e50 con 0x7f42081011e0 2024-03-21T13:26:45.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.625+0000 7f41fe7fc700 1 -- 172.21.15.97:0/1536444855 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 17) v1 ==== 92160+0+0 (secure 0 0 0) 0x7f41f8010430 con 0x7f42081011e0 2024-03-21T13:26:45.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.626+0000 7f41fe7fc700 1 --2- 172.21.15.97:0/1536444855 >> v2:172.21.15.97:6800/1943212787 conn(0x7f41f406e2f0 0x7f41f40707b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:45.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.626+0000 7f420cb9f700 1 -- 172.21.15.97:0/1536444855 >> v2:172.21.15.97:6800/1943212787 conn(0x7f41f406e2f0 msgr2=0x7f41f40707b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/1943212787 2024-03-21T13:26:45.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.626+0000 7f41fe7fc700 1 -- 172.21.15.97:0/1536444855 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f41f808e4d0 con 0x7f42081011e0 2024-03-21T13:26:45.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.626+0000 7f420cb9f700 1 --2- 172.21.15.97:0/1536444855 >> v2:172.21.15.97:6800/1943212787 conn(0x7f41f406e2f0 0x7f41f40707b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T13:26:45.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.630+0000 7f41fe7fc700 1 -- 172.21.15.97:0/1536444855 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f41f805bca0 con 0x7f42081011e0 2024-03-21T13:26:45.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.827+0000 7f420cb9f700 1 -- 172.21.15.97:0/1536444855 >> v2:172.21.15.97:6800/1943212787 conn(0x7f41f406e2f0 msgr2=0x7f41f40707b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/1943212787 2024-03-21T13:26:45.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.827+0000 7f420cb9f700 1 --2- 172.21.15.97:0/1536444855 >> v2:172.21.15.97:6800/1943212787 conn(0x7f41f406e2f0 0x7f41f40707b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-21T13:26:45.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.886+0000 7f420e3a2700 1 -- 172.21.15.97:0/1536444855 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f42081a6ef0 con 0x7f42081011e0 2024-03-21T13:26:45.888 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:26:45 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:26:45.705+0000 7f7647a22200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-21T13:26:45.888 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:26:45 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:26:45.799+0000 7f7647a22200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-21T13:26:45.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.891+0000 7f41fe7fc700 1 -- 172.21.15.97:0/1536444855 <== mon.2 v2:172.21.15.97:3301/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+176190 (secure 0 0 0) 0x7f41f805b5a0 con 0x7f42081011e0 2024-03-21T13:26:45.894 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:26:45.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.895+0000 7f420e3a2700 1 -- 172.21.15.97:0/1536444855 >> v2:172.21.15.97:6800/1943212787 conn(0x7f41f406e2f0 msgr2=0x7f41f40707b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T13:26:45.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.895+0000 7f420e3a2700 1 --2- 172.21.15.97:0/1536444855 >> v2:172.21.15.97:6800/1943212787 conn(0x7f41f406e2f0 0x7f41f40707b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:45.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.895+0000 7f420e3a2700 1 -- 172.21.15.97:0/1536444855 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f42081011e0 msgr2=0x7f420819a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:45.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.895+0000 7f420e3a2700 1 --2- 172.21.15.97:0/1536444855 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f42081011e0 0x7f420819a140 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f41f800e9a0 tx=0x7f41f800ecb0 comp rx=0 tx=0).stop 2024-03-21T13:26:45.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.895+0000 7f420e3a2700 1 -- 172.21.15.97:0/1536444855 shutdown_connections 2024-03-21T13:26:45.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.895+0000 7f420e3a2700 1 --2- 172.21.15.97:0/1536444855 >> v2:172.21.15.97:6800/1943212787 conn(0x7f41f406e2f0 0x7f41f40707b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:45.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.895+0000 7f420e3a2700 1 --2- 172.21.15.97:0/1536444855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42081062c0 0x7f420819ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:45.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.895+0000 7f420e3a2700 1 --2- 172.21.15.97:0/1536444855 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f42081011e0 0x7f420819a140 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:45.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.895+0000 7f420e3a2700 1 --2- 172.21.15.97:0/1536444855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4208101b90 0x7f420819a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:45.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.895+0000 7f420e3a2700 1 -- 172.21.15.97:0/1536444855 >> 172.21.15.97:0/1536444855 conn(0x7f42080fca70 msgr2=0x7f420806a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:26:45.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.895+0000 7f420e3a2700 1 -- 172.21.15.97:0/1536444855 shutdown_connections 2024-03-21T13:26:45.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:45.895+0000 7f420e3a2700 1 -- 172.21.15.97:0/1536444855 wait complete. 2024-03-21T13:26:46.121 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:26:45 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: 2024-03-21T13:26:45.684+0000 7f4ef19ba200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-21T13:26:46.121 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:26:45 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: 2024-03-21T13:26:45.777+0000 7f4ef19ba200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-21T13:26:46.457 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:46 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1536444855' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-21T13:26:46.457 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:46 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1536444855' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-21T13:26:46.509 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:46 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1536444855' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-21T13:26:46.510 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:26:46 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: 2024-03-21T13:26:46.418+0000 7f4ef19ba200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-21T13:26:46.631 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1943212787}]},"active_addr":"172.21.15.97:6800/1943212787","active_change":"2024-03-21T13:21:31.070233+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24112,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.97:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"cephadm","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":3797862919}]},{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":2302971796}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":571106904}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":4167763682}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":1125441995}]}]} 2024-03-21T13:26:46.634 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-21T13:26:46.635 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-21T13:26:46.635 DEBUG:teuthology.orchestra.run.smithi097:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-21T13:26:46.803 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:26:46 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:26:46.456+0000 7f7647a22200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-21T13:26:46.803 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:26:46 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:26:46.550+0000 7f7647a22200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-21T13:26:46.803 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:26:46 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:26:46.705+0000 7f7647a22200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-21T13:26:46.830 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:26:46 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: 2024-03-21T13:26:46.509+0000 7f4ef19ba200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-21T13:26:46.830 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:26:46 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: 2024-03-21T13:26:46.652+0000 7f4ef19ba200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-21T13:26:46.830 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:26:46 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: 2024-03-21T13:26:46.746+0000 7f4ef19ba200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-21T13:26:46.830 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:26:46 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: 2024-03-21T13:26:46.829+0000 7f4ef19ba200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-21T13:26:46.884 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T13:26:47.119 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:26:46 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:26:46.804+0000 7f7647a22200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-21T13:26:47.119 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:26:46 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:26:46.899+0000 7f7647a22200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-21T13:26:47.371 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:26:47 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: 2024-03-21T13:26:47.095+0000 7f4ef19ba200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-21T13:26:47.372 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:26:47 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: 2024-03-21T13:26:47.205+0000 7f4ef19ba200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-21T13:26:47.372 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:26:47 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: 2024-03-21T13:26:47.294+0000 7f4ef19ba200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-21T13:26:47.385 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:26:47 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:26:47.176+0000 7f7647a22200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-21T13:26:47.385 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:26:47 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:26:47.291+0000 7f7647a22200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-21T13:26:47.659 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:26:47 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:26:47.385+0000 7f7647a22200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-21T13:26:47.659 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:26:47 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:26:47.480+0000 7f7647a22200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-21T13:26:47.871 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:26:47 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: 2024-03-21T13:26:47.390+0000 7f4ef19ba200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-21T13:26:47.871 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:26:47 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: 2024-03-21T13:26:47.569+0000 7f4ef19ba200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-21T13:26:47.934 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:26:47 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:26:47.658+0000 7f7647a22200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-21T13:26:48.768 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:26:48 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:26:48.668+0000 7f7647a22200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-21T13:26:48.871 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:26:48 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: 2024-03-21T13:26:48.526+0000 7f4ef19ba200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-21T13:26:48.871 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:26:48 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: 2024-03-21T13:26:48.654+0000 7f4ef19ba200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-21T13:26:48.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:48.873+0000 7fe9f42ca700 1 -- 172.21.15.97:0/3190588275 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe9ec101210 msgr2=0x7fe9ec10d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:48.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:48.873+0000 7fe9f42ca700 1 --2- 172.21.15.97:0/3190588275 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe9ec101210 0x7fe9ec10d720 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe9e400b0a0 tx=0x7fe9e400b3b0 comp rx=0 tx=0).stop 2024-03-21T13:26:48.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:48.873+0000 7fe9f42ca700 1 -- 172.21.15.97:0/3190588275 shutdown_connections 2024-03-21T13:26:48.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:48.873+0000 7fe9f42ca700 1 --2- 172.21.15.97:0/3190588275 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe9ec101210 0x7fe9ec10d720 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:48.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:48.873+0000 7fe9f42ca700 1 --2- 172.21.15.97:0/3190588275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9ec106850 0x7fe9ec106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:48.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:48.873+0000 7fe9f42ca700 1 --2- 172.21.15.97:0/3190588275 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9ec100850 0x7fe9ec100cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:48.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:48.873+0000 7fe9f42ca700 1 -- 172.21.15.97:0/3190588275 >> 172.21.15.97:0/3190588275 conn(0x7fe9ec077690 msgr2=0x7fe9ec0feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:26:48.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:48.874+0000 7fe9f42ca700 1 -- 172.21.15.97:0/3190588275 shutdown_connections 2024-03-21T13:26:48.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:48.874+0000 7fe9f42ca700 1 -- 172.21.15.97:0/3190588275 wait complete. 2024-03-21T13:26:48.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:48.874+0000 7fe9f42ca700 1 Processor -- start 2024-03-21T13:26:48.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:48.874+0000 7fe9f42ca700 1 -- start start 2024-03-21T13:26:48.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:48.874+0000 7fe9f42ca700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9ec100850 0x7fe9ec198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:48.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:48.874+0000 7fe9f42ca700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9ec101210 0x7fe9ec198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:48.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:48.874+0000 7fe9f42ca700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe9ec106850 0x7fe9ec19c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:48.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:48.874+0000 7fe9f42ca700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ec10fa60 con 0x7fe9ec101210 2024-03-21T13:26:48.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:48.874+0000 7fe9f42ca700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ec10f8e0 con 0x7fe9ec100850 2024-03-21T13:26:48.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:48.874+0000 7fe9f42ca700 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ec10fbe0 con 0x7fe9ec106850 2024-03-21T13:26:48.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:48.875+0000 7fe9f2066700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9ec100850 0x7fe9ec198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:48.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:48.875+0000 7fe9f2066700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9ec100850 0x7fe9ec198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.97:49846/0 (socket says 172.21.15.97:49846) 2024-03-21T13:26:48.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:48.875+0000 7fe9f1865700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9ec101210 0x7fe9ec198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:48.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:48.875+0000 7fe9f2066700 1 -- 172.21.15.97:0/2649636241 learned_addr learned my addr 172.21.15.97:0/2649636241 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:26:48.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:48.875+0000 7fe9f2867700 1 --2- 172.21.15.97:0/2649636241 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe9ec106850 0x7fe9ec19c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:48.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:48.875+0000 7fe9f2066700 1 -- 172.21.15.97:0/2649636241 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe9ec106850 msgr2=0x7fe9ec19c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:48.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:48.875+0000 7fe9f2066700 1 --2- 172.21.15.97:0/2649636241 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe9ec106850 0x7fe9ec19c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:48.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:48.875+0000 7fe9f2066700 1 -- 172.21.15.97:0/2649636241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9ec101210 msgr2=0x7fe9ec198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:48.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:48.875+0000 7fe9f2066700 1 --2- 172.21.15.97:0/2649636241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9ec101210 0x7fe9ec198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:48.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:48.875+0000 7fe9f2066700 1 -- 172.21.15.97:0/2649636241 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9ec19d000 con 0x7fe9ec100850 2024-03-21T13:26:48.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:48.875+0000 7fe9f2066700 1 --2- 172.21.15.97:0/2649636241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9ec100850 0x7fe9ec198040 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe9e800eab0 tx=0x7fe9e800edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:26:48.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:48.875+0000 7fe9e37fe700 1 -- 172.21.15.97:0/2649636241 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e800ccd0 con 0x7fe9ec100850 2024-03-21T13:26:48.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:48.875+0000 7fe9f42ca700 1 -- 172.21.15.97:0/2649636241 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9ec19d2f0 con 0x7fe9ec100850 2024-03-21T13:26:48.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:48.875+0000 7fe9e37fe700 1 -- 172.21.15.97:0/2649636241 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fe9e8016890 con 0x7fe9ec100850 2024-03-21T13:26:48.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:48.875+0000 7fe9e37fe700 1 -- 172.21.15.97:0/2649636241 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e80157a0 con 0x7fe9ec100850 2024-03-21T13:26:48.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:48.875+0000 7fe9f42ca700 1 -- 172.21.15.97:0/2649636241 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9ec1a4b70 con 0x7fe9ec100850 2024-03-21T13:26:48.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:48.876+0000 7fe9f42ca700 1 -- 172.21.15.97:0/2649636241 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9ec101f90 con 0x7fe9ec100850 2024-03-21T13:26:48.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:48.877+0000 7fe9e37fe700 1 -- 172.21.15.97:0/2649636241 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) v1 ==== 92160+0+0 (secure 0 0 0) 0x7fe9e80163e0 con 0x7fe9ec100850 2024-03-21T13:26:48.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:48.877+0000 7fe9e37fe700 1 --2- 172.21.15.97:0/2649636241 >> v2:172.21.15.97:6800/1943212787 conn(0x7fe9d806e2f0 0x7fe9d80707b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:48.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:48.877+0000 7fe9f1865700 1 -- 172.21.15.97:0/2649636241 >> v2:172.21.15.97:6800/1943212787 conn(0x7fe9d806e2f0 msgr2=0x7fe9d80707b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/1943212787 2024-03-21T13:26:48.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:48.878+0000 7fe9e37fe700 1 -- 172.21.15.97:0/2649636241 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6001+0+0 (secure 0 0 0) 0x7fe9e808e5f0 con 0x7fe9ec100850 2024-03-21T13:26:48.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:48.878+0000 7fe9f1865700 1 --2- 172.21.15.97:0/2649636241 >> v2:172.21.15.97:6800/1943212787 conn(0x7fe9d806e2f0 0x7fe9d80707b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T13:26:48.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:48.882+0000 7fe9e37fe700 1 -- 172.21.15.97:0/2649636241 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe9e805bc40 con 0x7fe9ec100850 2024-03-21T13:26:49.053 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:26:48 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:26:48.801+0000 7f7647a22200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-21T13:26:49.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:49.079+0000 7fe9f1865700 1 -- 172.21.15.97:0/2649636241 >> v2:172.21.15.97:6800/1943212787 conn(0x7fe9d806e2f0 msgr2=0x7fe9d80707b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/1943212787 2024-03-21T13:26:49.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:49.079+0000 7fe9f1865700 1 --2- 172.21.15.97:0/2649636241 >> v2:172.21.15.97:6800/1943212787 conn(0x7fe9d806e2f0 0x7fe9d80707b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-21T13:26:49.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:49.084+0000 7fe9f42ca700 1 -- 172.21.15.97:0/2649636241 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe9ec068e50 con 0x7fe9ec100850 2024-03-21T13:26:49.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:49.085+0000 7fe9e37fe700 1 -- 172.21.15.97:0/2649636241 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+20668 (secure 0 0 0) 0x7fe9e805b540 con 0x7fe9ec100850 2024-03-21T13:26:49.086 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:26:49.086 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":70,"fsid":"a5dd1f32-e785-11ee-95cd-87774f69a715","created":"2024-03-21T13:20:36.782185+0000","modified":"2024-03-21T13:26:32.404482+0000","last_up_change":"2024-03-21T13:25:57.160251+0000","last_in_change":"2024-03-21T13:25:39.485525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T13:23:51.415792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"datapool","create_time":"2024-03-21T13:26:09.509557+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":64,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-03-21T13:26:10.075152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-03-21T13:26:11.497489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-03-21T13:26:13.463469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-03-21T13:26:15.534713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da68a71b-2182-4471-a6f8-4aa689009b20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6801","nonce":2691001097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2691001097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2691001097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2691001097}]},"public_addr":"172.21.15.97:6801/2691001097","cluster_addr":"172.21.15.97:6802/2691001097","heartbeat_back_addr":"172.21.15.97:6804/2691001097","heartbeat_front_addr":"172.21.15.97:6803/2691001097","state":["exists","up"]},{"osd":1,"uuid":"19b99056-a65c-47ec-9392-cc0d40b3b399","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":287967678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":287967678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":287967678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6807","nonce":287967678}]},"public_addr":"172.21.15.97:6805/287967678","cluster_addr":"172.21.15.97:6806/287967678","heartbeat_back_addr":"172.21.15.97:6808/287967678","heartbeat_front_addr":"172.21.15.97:6807/287967678","state":["exists","up"]},{"osd":2,"uuid":"f4d2f5c6-ac47-4a1d-a409-2bb884b6772f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":2953470250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2953470250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2953470250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2953470250}]},"public_addr":"172.21.15.97:6809/2953470250","cluster_addr":"172.21.15.97:6810/2953470250","heartbeat_back_addr":"172.21.15.97:6812/2953470250","heartbeat_front_addr":"172.21.15.97:6811/2953470250","state":["exists","up"]},{"osd":3,"uuid":"3ecf8a17-0ada-41e4-9740-b9f48ee14c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1678932102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":1678932102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1678932102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":1678932102}]},"public_addr":"172.21.15.97:6813/1678932102","cluster_addr":"172.21.15.97:6814/1678932102","heartbeat_back_addr":"172.21.15.97:6816/1678932102","heartbeat_front_addr":"172.21.15.97:6815/1678932102","state":["exists","up"]},{"osd":4,"uuid":"52ca17a1-0407-4fb1-994b-a382d3c46879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":913882960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6801","nonce":913882960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6803","nonce":913882960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":913882960}]},"public_addr":"172.21.15.144:6800/913882960","cluster_addr":"172.21.15.144:6801/913882960","heartbeat_back_addr":"172.21.15.144:6803/913882960","heartbeat_front_addr":"172.21.15.144:6802/913882960","state":["exists","up"]},{"osd":5,"uuid":"f3f7f8a2-d40b-4866-8dad-310dca09941c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2553466741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6805","nonce":2553466741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6807","nonce":2553466741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2553466741}]},"public_addr":"172.21.15.144:6804/2553466741","cluster_addr":"172.21.15.144:6805/2553466741","heartbeat_back_addr":"172.21.15.144:6807/2553466741","heartbeat_front_addr":"172.21.15.144:6806/2553466741","state":["exists","up"]},{"osd":6,"uuid":"cfca8ffd-b02e-44cd-bd48-55fdb4bd6ae5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":4170414757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4170414757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6811","nonce":4170414757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":4170414757}]},"public_addr":"172.21.15.144:6808/4170414757","cluster_addr":"172.21.15.144:6809/4170414757","heartbeat_back_addr":"172.21.15.144:6811/4170414757","heartbeat_front_addr":"172.21.15.144:6810/4170414757","state":["exists","up"]},{"osd":7,"uuid":"997e5de8-48df-4348-ab04-22c7c143cf20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3269572470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":3269572470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6815","nonce":3269572470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3269572470}]},"public_addr":"172.21.15.144:6812/3269572470","cluster_addr":"172.21.15.144:6813/3269572470","heartbeat_back_addr":"172.21.15.144:6815/3269572470","heartbeat_front_addr":"172.21.15.144:6814/3269572470","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T13:22:56.859160+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T13:23:23.500177+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T13:23:49.429654+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T13:24:15.416871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T13:24:39.114777+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T13:25:04.388302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T13:25:30.020027+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T13:25:55.809198+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.1d","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/938326700":"2024-03-22T13:21:31.070127+0000","172.21.15.97:0/930977130":"2024-03-22T13:21:31.070127+0000","172.21.15.97:0/2037905570":"2024-03-22T13:21:04.170875+0000","172.21.15.97:0/865729378":"2024-03-22T13:21:04.170875+0000","172.21.15.97:6800/2447843320":"2024-03-22T13:21:04.170875+0000","172.21.15.97:0/1523063805":"2024-03-22T13:21:04.170875+0000","172.21.15.97:6800/4140723216":"2024-03-22T13:21:31.070127+0000","172.21.15.97:0/652843111":"2024-03-22T13:21:31.070127+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T13:26:49.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:49.089+0000 7fe9f42ca700 1 -- 172.21.15.97:0/2649636241 >> v2:172.21.15.97:6800/1943212787 conn(0x7fe9d806e2f0 msgr2=0x7fe9d80707b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T13:26:49.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:49.089+0000 7fe9f42ca700 1 --2- 172.21.15.97:0/2649636241 >> v2:172.21.15.97:6800/1943212787 conn(0x7fe9d806e2f0 0x7fe9d80707b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:49.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:49.089+0000 7fe9f42ca700 1 -- 172.21.15.97:0/2649636241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9ec100850 msgr2=0x7fe9ec198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:49.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:49.089+0000 7fe9f42ca700 1 --2- 172.21.15.97:0/2649636241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9ec100850 0x7fe9ec198040 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe9e800eab0 tx=0x7fe9e800edc0 comp rx=0 tx=0).stop 2024-03-21T13:26:49.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:49.089+0000 7fe9f42ca700 1 -- 172.21.15.97:0/2649636241 shutdown_connections 2024-03-21T13:26:49.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:49.089+0000 7fe9f42ca700 1 --2- 172.21.15.97:0/2649636241 >> v2:172.21.15.97:6800/1943212787 conn(0x7fe9d806e2f0 0x7fe9d80707b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:49.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:49.089+0000 7fe9f42ca700 1 --2- 172.21.15.97:0/2649636241 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe9ec106850 0x7fe9ec19c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:49.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:49.089+0000 7fe9f42ca700 1 --2- 172.21.15.97:0/2649636241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9ec100850 0x7fe9ec198040 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:49.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:49.089+0000 7fe9f42ca700 1 --2- 172.21.15.97:0/2649636241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9ec101210 0x7fe9ec198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:49.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:49.089+0000 7fe9f42ca700 1 -- 172.21.15.97:0/2649636241 >> 172.21.15.97:0/2649636241 conn(0x7fe9ec077690 msgr2=0x7fe9ec1036d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:26:49.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:49.089+0000 7fe9f42ca700 1 -- 172.21.15.97:0/2649636241 shutdown_connections 2024-03-21T13:26:49.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:49.089+0000 7fe9f42ca700 1 -- 172.21.15.97:0/2649636241 wait complete. 2024-03-21T13:26:49.277 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:49 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2649636241' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T13:26:49.278 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:26:48 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: 2024-03-21T13:26:48.993+0000 7f4ef19ba200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-21T13:26:49.278 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:26:49 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: 2024-03-21T13:26:49.082+0000 7f4ef19ba200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-21T13:26:49.413 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:49 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2649636241' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T13:26:49.413 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:49 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2649636241' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T13:26:49.413 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:26:49 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:26:49.159+0000 7f7647a22200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-21T13:26:49.413 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:26:49 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:26:49.250+0000 7f7647a22200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-21T13:26:49.621 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:26:49 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: 2024-03-21T13:26:49.277+0000 7f4ef19ba200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-21T13:26:49.621 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:26:49 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: 2024-03-21T13:26:49.463+0000 7f4ef19ba200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-21T13:26:49.734 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:26:49 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:26:49.455+0000 7f7647a22200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-21T13:26:49.734 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:26:49 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:26:49.644+0000 7f7647a22200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-21T13:26:49.763 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-21T13:26:49.763 DEBUG:teuthology.orchestra.run.smithi097:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-21T13:26:50.008 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:26:49 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: 2024-03-21T13:26:49.744+0000 7f4ef19ba200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-21T13:26:50.014 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T13:26:50.053 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:26:49 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:26:49.946+0000 7f7647a22200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-21T13:26:50.371 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:26:50 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: 2024-03-21T13:26:50.007+0000 7f4ef19ba200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-21T13:26:50.371 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:26:50 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: 2024-03-21T13:26:50.244+0000 7f4ef19ba200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-21T13:26:50.442 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:26:50 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:26:50.220+0000 7f7647a22200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-21T13:26:50.806 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:26:50 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:26:50.469+0000 7f7647a22200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-21T13:26:51.371 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:26:51 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: 2024-03-21T13:26:51.052+0000 7f4ef19ba200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-21T13:26:51.468 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:26:51 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:26:51.305+0000 7f7647a22200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-21T13:26:51.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.720+0000 7fd29386f700 1 -- 172.21.15.97:0/2379531151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd28c107200 msgr2=0x7fd28c1075e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:51.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.720+0000 7fd29386f700 1 --2- 172.21.15.97:0/2379531151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd28c107200 0x7fd28c1075e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd27c0098f0 tx=0x7fd27c009c00 comp rx=0 tx=0).stop 2024-03-21T13:26:51.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.720+0000 7fd29386f700 1 -- 172.21.15.97:0/2379531151 shutdown_connections 2024-03-21T13:26:51.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.720+0000 7fd29386f700 1 --2- 172.21.15.97:0/2379531151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd28c101140 0x7fd28c10f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:51.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.720+0000 7fd29386f700 1 --2- 172.21.15.97:0/2379531151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd28c107200 0x7fd28c1075e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:51.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.720+0000 7fd29386f700 1 --2- 172.21.15.97:0/2379531151 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd28c100780 0x7fd28c100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:51.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.720+0000 7fd29386f700 1 -- 172.21.15.97:0/2379531151 >> 172.21.15.97:0/2379531151 conn(0x7fd28c077690 msgr2=0x7fd28c0feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:26:51.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.720+0000 7fd29386f700 1 -- 172.21.15.97:0/2379531151 shutdown_connections 2024-03-21T13:26:51.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.720+0000 7fd29386f700 1 -- 172.21.15.97:0/2379531151 wait complete. 2024-03-21T13:26:51.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.721+0000 7fd29386f700 1 Processor -- start 2024-03-21T13:26:51.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.721+0000 7fd29386f700 1 -- start start 2024-03-21T13:26:51.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.721+0000 7fd29386f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd28c100780 0x7fd28c1980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:51.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.721+0000 7fd29386f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd28c101140 0x7fd28c198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:51.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.721+0000 7fd29386f700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd28c107200 0x7fd28c19c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:51.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.721+0000 7fd29386f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28c111c80 con 0x7fd28c100780 2024-03-21T13:26:51.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.721+0000 7fd29386f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28c111b00 con 0x7fd28c101140 2024-03-21T13:26:51.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.721+0000 7fd29386f700 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd28c111e00 con 0x7fd28c107200 2024-03-21T13:26:51.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.721+0000 7fd291e0c700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd28c107200 0x7fd28c19c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:51.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.721+0000 7fd291e0c700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd28c107200 0x7fd28c19c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.97:42942/0 (socket says 172.21.15.97:42942) 2024-03-21T13:26:51.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.721+0000 7fd291e0c700 1 -- 172.21.15.97:0/1530757963 learned_addr learned my addr 172.21.15.97:0/1530757963 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:26:51.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.721+0000 7fd29160b700 1 --2- 172.21.15.97:0/1530757963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd28c100780 0x7fd28c1980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:51.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.721+0000 7fd290e0a700 1 --2- 172.21.15.97:0/1530757963 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd28c101140 0x7fd28c198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:51.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.722+0000 7fd290e0a700 1 -- 172.21.15.97:0/1530757963 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd28c107200 msgr2=0x7fd28c19c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:51.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.722+0000 7fd290e0a700 1 --2- 172.21.15.97:0/1530757963 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd28c107200 0x7fd28c19c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:51.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.722+0000 7fd290e0a700 1 -- 172.21.15.97:0/1530757963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd28c100780 msgr2=0x7fd28c1980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:51.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.722+0000 7fd290e0a700 1 --2- 172.21.15.97:0/1530757963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd28c100780 0x7fd28c1980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:51.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.722+0000 7fd290e0a700 1 -- 172.21.15.97:0/1530757963 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd28c19d0f0 con 0x7fd28c101140 2024-03-21T13:26:51.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.722+0000 7fd29160b700 1 --2- 172.21.15.97:0/1530757963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd28c100780 0x7fd28c1980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T13:26:51.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.722+0000 7fd291e0c700 1 --2- 172.21.15.97:0/1530757963 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd28c107200 0x7fd28c19c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T13:26:51.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.722+0000 7fd290e0a700 1 --2- 172.21.15.97:0/1530757963 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd28c101140 0x7fd28c198620 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd28800eb90 tx=0x7fd28800eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:26:51.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.722+0000 7fd2827fc700 1 -- 172.21.15.97:0/1530757963 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28800cc90 con 0x7fd28c101140 2024-03-21T13:26:51.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.722+0000 7fd29386f700 1 -- 172.21.15.97:0/1530757963 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd28c19d3e0 con 0x7fd28c101140 2024-03-21T13:26:51.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.722+0000 7fd2827fc700 1 -- 172.21.15.97:0/1530757963 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fd28800ce30 con 0x7fd28c101140 2024-03-21T13:26:51.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.723+0000 7fd2827fc700 1 -- 172.21.15.97:0/1530757963 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd288010430 con 0x7fd28c101140 2024-03-21T13:26:51.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.723+0000 7fd29386f700 1 -- 172.21.15.97:0/1530757963 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd28c101890 con 0x7fd28c101140 2024-03-21T13:26:51.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.723+0000 7fd2827fc700 1 -- 172.21.15.97:0/1530757963 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) v1 ==== 92160+0+0 (secure 0 0 0) 0x7fd288035b50 con 0x7fd28c101140 2024-03-21T13:26:51.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.724+0000 7fd2827fc700 1 --2- 172.21.15.97:0/1530757963 >> v2:172.21.15.97:6800/1943212787 conn(0x7fd27806e2f0 0x7fd2780707b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:51.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.724+0000 7fd29160b700 1 -- 172.21.15.97:0/1530757963 >> v2:172.21.15.97:6800/1943212787 conn(0x7fd27806e2f0 msgr2=0x7fd2780707b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/1943212787 2024-03-21T13:26:51.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.724+0000 7fd29386f700 1 -- 172.21.15.97:0/1530757963 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd28c068e50 con 0x7fd28c101140 2024-03-21T13:26:51.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.724+0000 7fd2827fc700 1 -- 172.21.15.97:0/1530757963 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6001+0+0 (secure 0 0 0) 0x7fd28808df10 con 0x7fd28c101140 2024-03-21T13:26:51.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.724+0000 7fd29160b700 1 --2- 172.21.15.97:0/1530757963 >> v2:172.21.15.97:6800/1943212787 conn(0x7fd27806e2f0 0x7fd2780707b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T13:26:51.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.729+0000 7fd2827fc700 1 -- 172.21.15.97:0/1530757963 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd288019e20 con 0x7fd28c101140 2024-03-21T13:26:51.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.925+0000 7fd29160b700 1 -- 172.21.15.97:0/1530757963 >> v2:172.21.15.97:6800/1943212787 conn(0x7fd27806e2f0 msgr2=0x7fd2780707b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/1943212787 2024-03-21T13:26:51.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.925+0000 7fd29160b700 1 --2- 172.21.15.97:0/1530757963 >> v2:172.21.15.97:6800/1943212787 conn(0x7fd27806e2f0 0x7fd2780707b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-21T13:26:51.945 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:26:51 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: 2024-03-21T13:26:51.628+0000 7f4ef19ba200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-21T13:26:51.945 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:26:51 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: 2024-03-21T13:26:51.725+0000 7f4ef19ba200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-21T13:26:51.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.950+0000 7fd29386f700 1 -- 172.21.15.97:0/1530757963 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd28c101b50 con 0x7fd28c101140 2024-03-21T13:26:51.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.952+0000 7fd2827fc700 1 -- 172.21.15.97:0/1530757963 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+20668 (secure 0 0 0) 0x7fd28800d340 con 0x7fd28c101140 2024-03-21T13:26:51.952 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:26:51.952 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":70,"fsid":"a5dd1f32-e785-11ee-95cd-87774f69a715","created":"2024-03-21T13:20:36.782185+0000","modified":"2024-03-21T13:26:32.404482+0000","last_up_change":"2024-03-21T13:25:57.160251+0000","last_in_change":"2024-03-21T13:25:39.485525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T13:23:51.415792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"datapool","create_time":"2024-03-21T13:26:09.509557+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":64,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-03-21T13:26:10.075152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-03-21T13:26:11.497489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-03-21T13:26:13.463469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-03-21T13:26:15.534713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da68a71b-2182-4471-a6f8-4aa689009b20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6801","nonce":2691001097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2691001097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2691001097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6803","nonce":2691001097}]},"public_addr":"172.21.15.97:6801/2691001097","cluster_addr":"172.21.15.97:6802/2691001097","heartbeat_back_addr":"172.21.15.97:6804/2691001097","heartbeat_front_addr":"172.21.15.97:6803/2691001097","state":["exists","up"]},{"osd":1,"uuid":"19b99056-a65c-47ec-9392-cc0d40b3b399","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6805","nonce":287967678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":287967678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":287967678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6807","nonce":287967678}]},"public_addr":"172.21.15.97:6805/287967678","cluster_addr":"172.21.15.97:6806/287967678","heartbeat_back_addr":"172.21.15.97:6808/287967678","heartbeat_front_addr":"172.21.15.97:6807/287967678","state":["exists","up"]},{"osd":2,"uuid":"f4d2f5c6-ac47-4a1d-a409-2bb884b6772f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":2953470250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2953470250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2953470250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2953470250}]},"public_addr":"172.21.15.97:6809/2953470250","cluster_addr":"172.21.15.97:6810/2953470250","heartbeat_back_addr":"172.21.15.97:6812/2953470250","heartbeat_front_addr":"172.21.15.97:6811/2953470250","state":["exists","up"]},{"osd":3,"uuid":"3ecf8a17-0ada-41e4-9740-b9f48ee14c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1678932102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":1678932102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1678932102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":1678932102}]},"public_addr":"172.21.15.97:6813/1678932102","cluster_addr":"172.21.15.97:6814/1678932102","heartbeat_back_addr":"172.21.15.97:6816/1678932102","heartbeat_front_addr":"172.21.15.97:6815/1678932102","state":["exists","up"]},{"osd":4,"uuid":"52ca17a1-0407-4fb1-994b-a382d3c46879","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":913882960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6801","nonce":913882960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6803","nonce":913882960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":913882960}]},"public_addr":"172.21.15.144:6800/913882960","cluster_addr":"172.21.15.144:6801/913882960","heartbeat_back_addr":"172.21.15.144:6803/913882960","heartbeat_front_addr":"172.21.15.144:6802/913882960","state":["exists","up"]},{"osd":5,"uuid":"f3f7f8a2-d40b-4866-8dad-310dca09941c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2553466741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6805","nonce":2553466741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6807","nonce":2553466741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2553466741}]},"public_addr":"172.21.15.144:6804/2553466741","cluster_addr":"172.21.15.144:6805/2553466741","heartbeat_back_addr":"172.21.15.144:6807/2553466741","heartbeat_front_addr":"172.21.15.144:6806/2553466741","state":["exists","up"]},{"osd":6,"uuid":"cfca8ffd-b02e-44cd-bd48-55fdb4bd6ae5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":4170414757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4170414757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6811","nonce":4170414757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":4170414757}]},"public_addr":"172.21.15.144:6808/4170414757","cluster_addr":"172.21.15.144:6809/4170414757","heartbeat_back_addr":"172.21.15.144:6811/4170414757","heartbeat_front_addr":"172.21.15.144:6810/4170414757","state":["exists","up"]},{"osd":7,"uuid":"997e5de8-48df-4348-ab04-22c7c143cf20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3269572470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":3269572470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6815","nonce":3269572470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3269572470}]},"public_addr":"172.21.15.144:6812/3269572470","cluster_addr":"172.21.15.144:6813/3269572470","heartbeat_back_addr":"172.21.15.144:6815/3269572470","heartbeat_front_addr":"172.21.15.144:6814/3269572470","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T13:22:56.859160+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T13:23:23.500177+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T13:23:49.429654+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T13:24:15.416871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T13:24:39.114777+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T13:25:04.388302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T13:25:30.020027+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T13:25:55.809198+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.1d","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/938326700":"2024-03-22T13:21:31.070127+0000","172.21.15.97:0/930977130":"2024-03-22T13:21:31.070127+0000","172.21.15.97:0/2037905570":"2024-03-22T13:21:04.170875+0000","172.21.15.97:0/865729378":"2024-03-22T13:21:04.170875+0000","172.21.15.97:6800/2447843320":"2024-03-22T13:21:04.170875+0000","172.21.15.97:0/1523063805":"2024-03-22T13:21:04.170875+0000","172.21.15.97:6800/4140723216":"2024-03-22T13:21:31.070127+0000","172.21.15.97:0/652843111":"2024-03-22T13:21:31.070127+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T13:26:51.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.954+0000 7fd29386f700 1 -- 172.21.15.97:0/1530757963 >> v2:172.21.15.97:6800/1943212787 conn(0x7fd27806e2f0 msgr2=0x7fd2780707b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T13:26:51.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.954+0000 7fd29386f700 1 --2- 172.21.15.97:0/1530757963 >> v2:172.21.15.97:6800/1943212787 conn(0x7fd27806e2f0 0x7fd2780707b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:51.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.954+0000 7fd29386f700 1 -- 172.21.15.97:0/1530757963 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd28c101140 msgr2=0x7fd28c198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:51.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.954+0000 7fd29386f700 1 --2- 172.21.15.97:0/1530757963 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd28c101140 0x7fd28c198620 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd28800eb90 tx=0x7fd28800eea0 comp rx=0 tx=0).stop 2024-03-21T13:26:51.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.954+0000 7fd29386f700 1 -- 172.21.15.97:0/1530757963 shutdown_connections 2024-03-21T13:26:51.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.954+0000 7fd29386f700 1 --2- 172.21.15.97:0/1530757963 >> v2:172.21.15.97:6800/1943212787 conn(0x7fd27806e2f0 0x7fd2780707b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:51.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.954+0000 7fd29386f700 1 --2- 172.21.15.97:0/1530757963 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd28c107200 0x7fd28c19c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:51.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.954+0000 7fd29386f700 1 --2- 172.21.15.97:0/1530757963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd28c100780 0x7fd28c1980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:51.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.954+0000 7fd29386f700 1 --2- 172.21.15.97:0/1530757963 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd28c101140 0x7fd28c198620 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:51.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.954+0000 7fd29386f700 1 -- 172.21.15.97:0/1530757963 >> 172.21.15.97:0/1530757963 conn(0x7fd28c077690 msgr2=0x7fd28c104cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:26:51.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.954+0000 7fd29386f700 1 -- 172.21.15.97:0/1530757963 shutdown_connections 2024-03-21T13:26:51.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:51.954+0000 7fd29386f700 1 -- 172.21.15.97:0/1530757963 wait complete. 2024-03-21T13:26:52.004 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:26:51 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:26:51.914+0000 7f7647a22200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-21T13:26:52.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:26:52 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:26:52.024+0000 7f7647a22200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-21T13:26:52.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:26:52 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:26:52.250+0000 7f7647a22200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-21T13:26:52.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:52 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1530757963' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T13:26:52.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:52 smithi097 ceph-mon[30040]: Standby manager daemon x restarted 2024-03-21T13:26:52.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:52 smithi097 ceph-mon[30040]: Standby manager daemon x started 2024-03-21T13:26:52.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:52 smithi097 ceph-mon[30040]: from='mgr.? 172.21.15.144:0/1134505927' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-21T13:26:52.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:52 smithi097 ceph-mon[30040]: from='mgr.? 172.21.15.144:0/1134505927' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T13:26:52.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:52 smithi097 ceph-mon[30040]: from='mgr.? 172.21.15.144:0/1134505927' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-21T13:26:52.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:52 smithi097 ceph-mon[30040]: from='mgr.? 172.21.15.144:0/1134505927' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T13:26:52.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:52 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1530757963' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T13:26:52.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:52 smithi097 ceph-mon[39416]: Standby manager daemon x restarted 2024-03-21T13:26:52.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:52 smithi097 ceph-mon[39416]: Standby manager daemon x started 2024-03-21T13:26:52.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:52 smithi097 ceph-mon[39416]: from='mgr.? 172.21.15.144:0/1134505927' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-21T13:26:52.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:52 smithi097 ceph-mon[39416]: from='mgr.? 172.21.15.144:0/1134505927' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T13:26:52.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:52 smithi097 ceph-mon[39416]: from='mgr.? 172.21.15.144:0/1134505927' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-21T13:26:52.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:52 smithi097 ceph-mon[39416]: from='mgr.? 172.21.15.144:0/1134505927' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T13:26:52.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:52 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1530757963' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T13:26:52.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:52 smithi144 ceph-mon[33633]: Standby manager daemon x restarted 2024-03-21T13:26:52.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:52 smithi144 ceph-mon[33633]: Standby manager daemon x started 2024-03-21T13:26:52.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:52 smithi144 ceph-mon[33633]: from='mgr.? 172.21.15.144:0/1134505927' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-21T13:26:52.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:52 smithi144 ceph-mon[33633]: from='mgr.? 172.21.15.144:0/1134505927' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T13:26:52.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:52 smithi144 ceph-mon[33633]: from='mgr.? 172.21.15.144:0/1134505927' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-21T13:26:52.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:52 smithi144 ceph-mon[33633]: from='mgr.? 172.21.15.144:0/1134505927' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T13:26:52.372 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:26:51 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: 2024-03-21T13:26:51.944+0000 7f4ef19ba200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-21T13:26:52.372 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:26:51 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: [21/Mar/2024:13:26:51] ENGINE Bus STARTING 2024-03-21T13:26:52.372 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:26:52 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: CherryPy Checker: 2024-03-21T13:26:52.372 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:26:52 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: The Application mounted at '' has an empty config. 2024-03-21T13:26:52.372 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:26:52 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: 2024-03-21T13:26:52.372 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:26:52 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: [21/Mar/2024:13:26:52] ENGINE Serving on http://:::9283 2024-03-21T13:26:52.372 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 13:26:52 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x[35709]: [21/Mar/2024:13:26:52] ENGINE Bus STARTED 2024-03-21T13:26:52.622 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:26:52 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: [21/Mar/2024:13:26:52] ENGINE Bus STARTING 2024-03-21T13:26:52.622 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:26:52 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: CherryPy Checker: 2024-03-21T13:26:52.622 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:26:52 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: The Application mounted at '' has an empty config. 2024-03-21T13:26:52.622 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:26:52 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T13:26:52.622 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:26:52 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: [21/Mar/2024:13:26:52] ENGINE Serving on http://:::9283 2024-03-21T13:26:52.622 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:26:52 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: [21/Mar/2024:13:26:52] ENGINE Bus STARTED 2024-03-21T13:26:52.656 DEBUG:teuthology.orchestra.run.smithi097:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-21T13:26:52.657 DEBUG:teuthology.orchestra.run.smithi097:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-21T13:26:52.657 DEBUG:teuthology.orchestra.run.smithi097:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-21T13:26:52.658 DEBUG:teuthology.orchestra.run.smithi097:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-21T13:26:52.658 DEBUG:teuthology.orchestra.run.smithi097:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-21T13:26:52.658 DEBUG:teuthology.orchestra.run.smithi097:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-21T13:26:52.659 DEBUG:teuthology.orchestra.run.smithi097:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-03-21T13:26:52.659 DEBUG:teuthology.orchestra.run.smithi097:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-03-21T13:26:53.165 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T13:26:53.180 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T13:26:53.185 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T13:26:53.190 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T13:26:53.190 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T13:26:53.196 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T13:26:53.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[30040]: mgrmap e18: y(active, since 5m), standbys: x 2024-03-21T13:26:53.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[30040]: Active manager daemon y restarted 2024-03-21T13:26:53.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[30040]: Activating manager daemon y 2024-03-21T13:26:53.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[30040]: osdmap e71: 8 total, 8 up, 8 in 2024-03-21T13:26:53.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[30040]: mgrmap e19: y(active, starting, since 0.0086204s), standbys: x 2024-03-21T13:26:53.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T13:26:53.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T13:26:53.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T13:26:53.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-21T13:26:53.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-21T13:26:53.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T13:26:53.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T13:26:53.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T13:26:53.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T13:26:53.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T13:26:53.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T13:26:53.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T13:26:53.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T13:26:53.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-21T13:26:53.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-21T13:26:53.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-21T13:26:53.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[30040]: Manager daemon y is now available 2024-03-21T13:26:53.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:26:53.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:26:53.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:26:53.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-21T13:26:53.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-21T13:26:53.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-21T13:26:53.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-21T13:26:53.306 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[39416]: mgrmap e18: y(active, since 5m), standbys: x 2024-03-21T13:26:53.306 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[39416]: Active manager daemon y restarted 2024-03-21T13:26:53.306 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[39416]: Activating manager daemon y 2024-03-21T13:26:53.306 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[39416]: osdmap e71: 8 total, 8 up, 8 in 2024-03-21T13:26:53.306 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[39416]: mgrmap e19: y(active, starting, since 0.0086204s), standbys: x 2024-03-21T13:26:53.306 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T13:26:53.306 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T13:26:53.306 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T13:26:53.306 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-21T13:26:53.306 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-21T13:26:53.306 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T13:26:53.306 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T13:26:53.307 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T13:26:53.307 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T13:26:53.307 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T13:26:53.307 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T13:26:53.307 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T13:26:53.307 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T13:26:53.307 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-21T13:26:53.307 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-21T13:26:53.307 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-21T13:26:53.307 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[39416]: Manager daemon y is now available 2024-03-21T13:26:53.307 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:26:53.307 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:26:53.307 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:26:53.307 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-21T13:26:53.308 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-21T13:26:53.308 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-21T13:26:53.308 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:53 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-21T13:26:53.308 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:53 smithi144 ceph-mon[33633]: mgrmap e18: y(active, since 5m), standbys: x 2024-03-21T13:26:53.308 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:53 smithi144 ceph-mon[33633]: Active manager daemon y restarted 2024-03-21T13:26:53.308 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:53 smithi144 ceph-mon[33633]: Activating manager daemon y 2024-03-21T13:26:53.308 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:53 smithi144 ceph-mon[33633]: osdmap e71: 8 total, 8 up, 8 in 2024-03-21T13:26:53.308 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:53 smithi144 ceph-mon[33633]: mgrmap e19: y(active, starting, since 0.0086204s), standbys: x 2024-03-21T13:26:53.308 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:53 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T13:26:53.308 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:53 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T13:26:53.309 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:53 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T13:26:53.309 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:53 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-21T13:26:53.309 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:53 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-21T13:26:53.309 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:53 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T13:26:53.309 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:53 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T13:26:53.309 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:53 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T13:26:53.309 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:53 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T13:26:53.309 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:53 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T13:26:53.309 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:53 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T13:26:53.309 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:53 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T13:26:53.309 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:53 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T13:26:53.309 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:53 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-21T13:26:53.309 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:53 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-21T13:26:53.309 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:53 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-21T13:26:53.310 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:53 smithi144 ceph-mon[33633]: Manager daemon y is now available 2024-03-21T13:26:53.310 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:53 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:26:53.310 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:53 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:26:53.310 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:53 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:26:53.310 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:53 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-21T13:26:53.310 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:53 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-21T13:26:53.310 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:53 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-21T13:26:53.310 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:53 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-21T13:26:53.564 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:26:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:26:54.391 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:54 smithi097 ceph-mon[39416]: [21/Mar/2024:13:26:53] ENGINE Bus STARTING 2024-03-21T13:26:54.391 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:54 smithi097 ceph-mon[39416]: mgrmap e20: y(active, since 1.01254s), standbys: x 2024-03-21T13:26:54.391 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:54 smithi097 ceph-mon[39416]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:26:54.391 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:54 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:26:54.391 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:54 smithi097 ceph-mon[39416]: [21/Mar/2024:13:26:53] ENGINE Serving on http://172.21.15.97:8765 2024-03-21T13:26:54.391 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:54 smithi097 ceph-mon[39416]: [21/Mar/2024:13:26:53] ENGINE Serving on https://172.21.15.97:7150 2024-03-21T13:26:54.391 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:54 smithi097 ceph-mon[39416]: [21/Mar/2024:13:26:53] ENGINE Bus STARTED 2024-03-21T13:26:54.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:54 smithi097 ceph-mon[30040]: [21/Mar/2024:13:26:53] ENGINE Bus STARTING 2024-03-21T13:26:54.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:54 smithi097 ceph-mon[30040]: mgrmap e20: y(active, since 1.01254s), standbys: x 2024-03-21T13:26:54.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:54 smithi097 ceph-mon[30040]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:26:54.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:54 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:26:54.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:54 smithi097 ceph-mon[30040]: [21/Mar/2024:13:26:53] ENGINE Serving on http://172.21.15.97:8765 2024-03-21T13:26:54.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:54 smithi097 ceph-mon[30040]: [21/Mar/2024:13:26:53] ENGINE Serving on https://172.21.15.97:7150 2024-03-21T13:26:54.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:54 smithi097 ceph-mon[30040]: [21/Mar/2024:13:26:53] ENGINE Bus STARTED 2024-03-21T13:26:54.618 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:54 smithi144 ceph-mon[33633]: [21/Mar/2024:13:26:53] ENGINE Bus STARTING 2024-03-21T13:26:54.618 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:54 smithi144 ceph-mon[33633]: mgrmap e20: y(active, since 1.01254s), standbys: x 2024-03-21T13:26:54.618 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:54 smithi144 ceph-mon[33633]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:26:54.618 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:54 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:26:54.618 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:54 smithi144 ceph-mon[33633]: [21/Mar/2024:13:26:53] ENGINE Serving on http://172.21.15.97:8765 2024-03-21T13:26:54.618 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:54 smithi144 ceph-mon[33633]: [21/Mar/2024:13:26:53] ENGINE Serving on https://172.21.15.97:7150 2024-03-21T13:26:54.619 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:54 smithi144 ceph-mon[33633]: [21/Mar/2024:13:26:53] ENGINE Bus STARTED 2024-03-21T13:26:55.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:55 smithi097 ceph-mon[39416]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:26:55.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:55 smithi097 ceph-mon[30040]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:26:55.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:55 smithi144 ceph-mon[33633]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:26:56.371 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:56 smithi097 ceph-mon[30040]: mgrmap e21: y(active, since 3s), standbys: x 2024-03-21T13:26:56.371 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:56 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:26:56.371 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:56 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:26:56.371 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:56 smithi097 ceph-mon[39416]: mgrmap e21: y(active, since 3s), standbys: x 2024-03-21T13:26:56.371 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:56 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:26:56.371 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:56 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:26:56.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:56 smithi144 ceph-mon[33633]: mgrmap e21: y(active, since 3s), standbys: x 2024-03-21T13:26:56.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:56 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:26:56.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:56 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:26:56.685 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T13:26:56.685 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T13:26:58.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:57 smithi097 ceph-mon[39416]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:26:58.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:57 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:26:58.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:57 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:26:58.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:57 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:26:58.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:57 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:26:58.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:57 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:26:58.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:57 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:26:58.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:57 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:26:58.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:57 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:26:58.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:57 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:26:58.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:57 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:26:58.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:57 smithi097 ceph-mon[39416]: Adjusting osd_memory_target on smithi144 to 3760M 2024-03-21T13:26:58.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:57 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:26:58.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:57 smithi097 ceph-mon[39416]: mgrmap e22: y(active, since 5s), standbys: x 2024-03-21T13:26:58.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:57 smithi097 ceph-mon[30040]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:26:58.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:57 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:26:58.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:57 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:26:58.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:57 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:26:58.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:57 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:26:58.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:57 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:26:58.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:57 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:26:58.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:57 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:26:58.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:57 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:26:58.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:57 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:26:58.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:57 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:26:58.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:57 smithi097 ceph-mon[30040]: Adjusting osd_memory_target on smithi144 to 3760M 2024-03-21T13:26:58.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:57 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:26:58.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:57 smithi097 ceph-mon[30040]: mgrmap e22: y(active, since 5s), standbys: x 2024-03-21T13:26:58.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:57 smithi144 ceph-mon[33633]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:26:58.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:57 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:26:58.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:57 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:26:58.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:57 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:26:58.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:57 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:26:58.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:57 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:26:58.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:57 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:26:58.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:57 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:26:58.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:57 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:26:58.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:57 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:26:58.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:57 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:26:58.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:57 smithi144 ceph-mon[33633]: Adjusting osd_memory_target on smithi144 to 3760M 2024-03-21T13:26:58.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:57 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:26:58.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:57 smithi144 ceph-mon[33633]: mgrmap e22: y(active, since 5s), standbys: x 2024-03-21T13:26:58.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.608+0000 7f2be89e2700 1 -- 172.21.15.97:0/2622259062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2be41081c0 msgr2=0x7f2be41085a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:58.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.608+0000 7f2be89e2700 1 --2- 172.21.15.97:0/2622259062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2be41081c0 0x7f2be41085a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4009920 tx=0x7f2bd4009c30 comp rx=0 tx=0).stop 2024-03-21T13:26:58.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.608+0000 7f2be89e2700 1 -- 172.21.15.97:0/2622259062 shutdown_connections 2024-03-21T13:26:58.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.608+0000 7f2be89e2700 1 --2- 172.21.15.97:0/2622259062 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2be4102ba0 0x7f2be410efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:58.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.608+0000 7f2be89e2700 1 --2- 172.21.15.97:0/2622259062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2be41081c0 0x7f2be41085a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:58.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.608+0000 7f2be89e2700 1 --2- 172.21.15.97:0/2622259062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2be41021e0 0x7f2be4102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:58.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.608+0000 7f2be89e2700 1 -- 172.21.15.97:0/2622259062 >> 172.21.15.97:0/2622259062 conn(0x7f2be40792c0 msgr2=0x7f2be40796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:26:58.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.608+0000 7f2be89e2700 1 -- 172.21.15.97:0/2622259062 shutdown_connections 2024-03-21T13:26:58.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.608+0000 7f2be89e2700 1 -- 172.21.15.97:0/2622259062 wait complete. 2024-03-21T13:26:58.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.608+0000 7f2be89e2700 1 Processor -- start 2024-03-21T13:26:58.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.608+0000 7f2be89e2700 1 -- start start 2024-03-21T13:26:58.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.609+0000 7f2be89e2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2be41021e0 0x7f2be4074490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:58.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.609+0000 7f2be89e2700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2be4102ba0 0x7f2be40749d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:58.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.609+0000 7f2be89e2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2be41081c0 0x7f2be4074f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:58.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.609+0000 7f2be89e2700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be4111af0 con 0x7f2be41081c0 2024-03-21T13:26:58.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.609+0000 7f2be89e2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be4111970 con 0x7f2be41021e0 2024-03-21T13:26:58.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.609+0000 7f2be89e2700 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2be4111c70 con 0x7f2be4102ba0 2024-03-21T13:26:58.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.609+0000 7f2be1d9b700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2be4102ba0 0x7f2be40749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:58.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.609+0000 7f2be1d9b700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2be4102ba0 0x7f2be40749d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.97:43048/0 (socket says 172.21.15.97:43048) 2024-03-21T13:26:58.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.609+0000 7f2be1d9b700 1 -- 172.21.15.97:0/1379681922 learned_addr learned my addr 172.21.15.97:0/1379681922 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:26:58.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.609+0000 7f2be2d9d700 1 --2- 172.21.15.97:0/1379681922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2be41081c0 0x7f2be4074f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:58.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.609+0000 7f2be259c700 1 --2- 172.21.15.97:0/1379681922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2be41021e0 0x7f2be4074490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:58.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.609+0000 7f2be1d9b700 1 -- 172.21.15.97:0/1379681922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2be41021e0 msgr2=0x7f2be4074490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:58.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.609+0000 7f2be1d9b700 1 --2- 172.21.15.97:0/1379681922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2be41021e0 0x7f2be4074490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:58.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.609+0000 7f2be1d9b700 1 -- 172.21.15.97:0/1379681922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2be41081c0 msgr2=0x7f2be4074f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:58.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.609+0000 7f2be1d9b700 1 --2- 172.21.15.97:0/1379681922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2be41081c0 0x7f2be4074f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:58.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.609+0000 7f2be1d9b700 1 -- 172.21.15.97:0/1379681922 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be41a64e0 con 0x7f2be4102ba0 2024-03-21T13:26:58.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.609+0000 7f2be1d9b700 1 --2- 172.21.15.97:0/1379681922 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2be4102ba0 0x7f2be40749d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2bd800c9b0 tx=0x7f2bd800ccc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:26:58.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.609+0000 7f2bd37fe700 1 -- 172.21.15.97:0/1379681922 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8007b00 con 0x7f2be4102ba0 2024-03-21T13:26:58.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.609+0000 7f2bd37fe700 1 -- 172.21.15.97:0/1379681922 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f2bd800ce50 con 0x7f2be4102ba0 2024-03-21T13:26:58.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.609+0000 7f2be89e2700 1 -- 172.21.15.97:0/1379681922 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2be41a67d0 con 0x7f2be4102ba0 2024-03-21T13:26:58.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.609+0000 7f2bd37fe700 1 -- 172.21.15.97:0/1379681922 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8018660 con 0x7f2be4102ba0 2024-03-21T13:26:58.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.610+0000 7f2be89e2700 1 -- 172.21.15.97:0/1379681922 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2be41a6d10 con 0x7f2be4102ba0 2024-03-21T13:26:58.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.610+0000 7f2be89e2700 1 -- 172.21.15.97:0/1379681922 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2be4107760 con 0x7f2be4102ba0 2024-03-21T13:26:58.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.611+0000 7f2bd37fe700 1 -- 172.21.15.97:0/1379681922 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 22) v1 ==== 92152+0+0 (secure 0 0 0) 0x7f2bd8004830 con 0x7f2be4102ba0 2024-03-21T13:26:58.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.611+0000 7f2bd37fe700 1 --2- 172.21.15.97:0/1379681922 >> v2:172.21.15.97:6800/12494470 conn(0x7f2bcc06e300 0x7f2bcc0707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:58.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.611+0000 7f2be259c700 1 --2- 172.21.15.97:0/1379681922 >> v2:172.21.15.97:6800/12494470 conn(0x7f2bcc06e300 0x7f2bcc0707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:58.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.611+0000 7f2bd37fe700 1 -- 172.21.15.97:0/1379681922 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f2bd808e3e0 con 0x7f2be4102ba0 2024-03-21T13:26:58.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.611+0000 7f2bd37fe700 1 --2- 172.21.15.97:0/1379681922 >> v2:172.21.15.97:6809/2953470250 conn(0x7f2bcc077ba0 0x7f2bcc07a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:26:58.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.611+0000 7f2bd37fe700 1 -- 172.21.15.97:0/1379681922 --> v2:172.21.15.97:6809/2953470250 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2bd800ee50 con 0x7f2bcc077ba0 2024-03-21T13:26:58.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.611+0000 7f2bd37fe700 1 -- 172.21.15.97:0/1379681922 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2bd8095050 con 0x7f2be4102ba0 2024-03-21T13:26:58.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.612+0000 7f2be2d9d700 1 --2- 172.21.15.97:0/1379681922 >> v2:172.21.15.97:6809/2953470250 conn(0x7f2bcc077ba0 0x7f2bcc07a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:26:58.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.612+0000 7f2be259c700 1 --2- 172.21.15.97:0/1379681922 >> v2:172.21.15.97:6800/12494470 conn(0x7f2bcc06e300 0x7f2bcc0707c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4009680 tx=0x7f2bd4019040 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:26:58.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.612+0000 7f2be2d9d700 1 --2- 172.21.15.97:0/1379681922 >> v2:172.21.15.97:6809/2953470250 conn(0x7f2bcc077ba0 0x7f2bcc07a040 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:26:58.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.613+0000 7f2bd37fe700 1 -- 172.21.15.97:0/1379681922 <== osd.2 v2:172.21.15.97:6809/2953470250 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f2bd800ee50 con 0x7f2bcc077ba0 2024-03-21T13:26:58.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.629+0000 7f2be89e2700 1 -- 172.21.15.97:0/1379681922 --> v2:172.21.15.97:6809/2953470250 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2be4103870 con 0x7f2bcc077ba0 2024-03-21T13:26:58.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.629+0000 7f2bd37fe700 1 -- 172.21.15.97:0/1379681922 <== osd.2 v2:172.21.15.97:6809/2953470250 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2be4103870 con 0x7f2bcc077ba0 2024-03-21T13:26:58.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.629+0000 7f2be89e2700 1 -- 172.21.15.97:0/1379681922 >> v2:172.21.15.97:6809/2953470250 conn(0x7f2bcc077ba0 msgr2=0x7f2bcc07a040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:58.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.629+0000 7f2be89e2700 1 --2- 172.21.15.97:0/1379681922 >> v2:172.21.15.97:6809/2953470250 conn(0x7f2bcc077ba0 0x7f2bcc07a040 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:26:58.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.629+0000 7f2be89e2700 1 -- 172.21.15.97:0/1379681922 >> v2:172.21.15.97:6800/12494470 conn(0x7f2bcc06e300 msgr2=0x7f2bcc0707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:58.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.629+0000 7f2be89e2700 1 --2- 172.21.15.97:0/1379681922 >> v2:172.21.15.97:6800/12494470 conn(0x7f2bcc06e300 0x7f2bcc0707c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4009680 tx=0x7f2bd4019040 comp rx=0 tx=0).stop 2024-03-21T13:26:58.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.629+0000 7f2be89e2700 1 -- 172.21.15.97:0/1379681922 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2be4102ba0 msgr2=0x7f2be40749d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:58.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.629+0000 7f2be89e2700 1 --2- 172.21.15.97:0/1379681922 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2be4102ba0 0x7f2be40749d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2bd800c9b0 tx=0x7f2bd800ccc0 comp rx=0 tx=0).stop 2024-03-21T13:26:58.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.629+0000 7f2be2d9d700 1 -- 172.21.15.97:0/1379681922 reap_dead start 2024-03-21T13:26:58.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.630+0000 7f2be89e2700 1 -- 172.21.15.97:0/1379681922 shutdown_connections 2024-03-21T13:26:58.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.630+0000 7f2be89e2700 1 -- 172.21.15.97:0/1379681922 >> 172.21.15.97:0/1379681922 conn(0x7f2be40792c0 msgr2=0x7f2be4105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:26:58.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.630+0000 7f2be89e2700 1 -- 172.21.15.97:0/1379681922 shutdown_connections 2024-03-21T13:26:58.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:26:58.630+0000 7f2be89e2700 1 -- 172.21.15.97:0/1379681922 wait complete. 2024-03-21T13:27:00.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:26:59 smithi097 ceph-mon[39416]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:27:00.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:26:59 smithi097 ceph-mon[30040]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:27:00.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:26:59 smithi144 ceph-mon[33633]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:27:01.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.394+0000 7fdbfb94e700 1 -- 172.21.15.97:0/1706687444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbf41070d0 msgr2=0x7fdbf41074b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:01.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.394+0000 7fdbfb94e700 1 --2- 172.21.15.97:0/1706687444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbf41070d0 0x7fdbf41074b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdbf00098f0 tx=0x7fdbf0009c00 comp rx=0 tx=0).stop 2024-03-21T13:27:01.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.394+0000 7fdbfb94e700 1 -- 172.21.15.97:0/1706687444 shutdown_connections 2024-03-21T13:27:01.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.394+0000 7fdbfb94e700 1 --2- 172.21.15.97:0/1706687444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbf4101a30 0x7fdbf410df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:01.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.394+0000 7fdbfb94e700 1 --2- 172.21.15.97:0/1706687444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbf41070d0 0x7fdbf41074b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:01.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.394+0000 7fdbfb94e700 1 --2- 172.21.15.97:0/1706687444 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdbf4101090 0x7fdbf41014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:01.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.394+0000 7fdbfb94e700 1 -- 172.21.15.97:0/1706687444 >> 172.21.15.97:0/1706687444 conn(0x7fdbf4077fc0 msgr2=0x7fdbf40783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:27:01.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.395+0000 7fdbfb94e700 1 -- 172.21.15.97:0/1706687444 shutdown_connections 2024-03-21T13:27:01.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.395+0000 7fdbfb94e700 1 -- 172.21.15.97:0/1706687444 wait complete. 2024-03-21T13:27:01.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.395+0000 7fdbfb94e700 1 Processor -- start 2024-03-21T13:27:01.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.395+0000 7fdbfb94e700 1 -- start start 2024-03-21T13:27:01.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.395+0000 7fdbfb94e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbf4101090 0x7fdbf4198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:01.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.395+0000 7fdbfb94e700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdbf4101a30 0x7fdbf4198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:01.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.395+0000 7fdbfb94e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbf41070d0 0x7fdbf419c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:01.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.395+0000 7fdbfb94e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf41109e0 con 0x7fdbf4101090 2024-03-21T13:27:01.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.395+0000 7fdbfb94e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf4110860 con 0x7fdbf41070d0 2024-03-21T13:27:01.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.395+0000 7fdbfb94e700 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf4110b60 con 0x7fdbf4101a30 2024-03-21T13:27:01.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.395+0000 7fdbf96ea700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbf4101090 0x7fdbf4198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:01.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.395+0000 7fdbf8ee9700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdbf4101a30 0x7fdbf4198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:01.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.395+0000 7fdbf96ea700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbf4101090 0x7fdbf4198150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46202/0 (socket says 172.21.15.97:46202) 2024-03-21T13:27:01.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.395+0000 7fdbf96ea700 1 -- 172.21.15.97:0/1595627088 learned_addr learned my addr 172.21.15.97:0/1595627088 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:27:01.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.396+0000 7fdbf9eeb700 1 --2- 172.21.15.97:0/1595627088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbf41070d0 0x7fdbf419c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:01.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.396+0000 7fdbf8ee9700 1 -- 172.21.15.97:0/1595627088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbf41070d0 msgr2=0x7fdbf419c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:01.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.396+0000 7fdbf8ee9700 1 --2- 172.21.15.97:0/1595627088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbf41070d0 0x7fdbf419c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:01.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.396+0000 7fdbf8ee9700 1 -- 172.21.15.97:0/1595627088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbf4101090 msgr2=0x7fdbf4198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:01.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.396+0000 7fdbf8ee9700 1 --2- 172.21.15.97:0/1595627088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbf4101090 0x7fdbf4198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:01.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.396+0000 7fdbf8ee9700 1 -- 172.21.15.97:0/1595627088 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbf419d110 con 0x7fdbf4101a30 2024-03-21T13:27:01.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.396+0000 7fdbf9eeb700 1 --2- 172.21.15.97:0/1595627088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbf41070d0 0x7fdbf419c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T13:27:01.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.396+0000 7fdbf96ea700 1 --2- 172.21.15.97:0/1595627088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbf4101090 0x7fdbf4198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T13:27:01.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.396+0000 7fdbf8ee9700 1 --2- 172.21.15.97:0/1595627088 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdbf4101a30 0x7fdbf4198690 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdbe400e9c0 tx=0x7fdbe400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:27:01.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.396+0000 7fdbea7fc700 1 -- 172.21.15.97:0/1595627088 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe4004220 con 0x7fdbf4101a30 2024-03-21T13:27:01.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.396+0000 7fdbfb94e700 1 -- 172.21.15.97:0/1595627088 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbf419d400 con 0x7fdbf4101a30 2024-03-21T13:27:01.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.396+0000 7fdbea7fc700 1 -- 172.21.15.97:0/1595627088 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fdbe40043c0 con 0x7fdbf4101a30 2024-03-21T13:27:01.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.396+0000 7fdbea7fc700 1 -- 172.21.15.97:0/1595627088 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe400cda0 con 0x7fdbf4101a30 2024-03-21T13:27:01.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.396+0000 7fdbfb94e700 1 -- 172.21.15.97:0/1595627088 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbf41a4c80 con 0x7fdbf4101a30 2024-03-21T13:27:01.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.397+0000 7fdbfb94e700 1 -- 172.21.15.97:0/1595627088 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdbf406a820 con 0x7fdbf4101a30 2024-03-21T13:27:01.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.397+0000 7fdbea7fc700 1 -- 172.21.15.97:0/1595627088 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 22) v1 ==== 92152+0+0 (secure 0 0 0) 0x7fdbe4004750 con 0x7fdbf4101a30 2024-03-21T13:27:01.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.398+0000 7fdbea7fc700 1 --2- 172.21.15.97:0/1595627088 >> v2:172.21.15.97:6800/12494470 conn(0x7fdbe006e300 0x7fdbe00707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:01.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.398+0000 7fdbf96ea700 1 --2- 172.21.15.97:0/1595627088 >> v2:172.21.15.97:6800/12494470 conn(0x7fdbe006e300 0x7fdbe00707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:01.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.398+0000 7fdbea7fc700 1 -- 172.21.15.97:0/1595627088 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6259+0+0 (secure 0 0 0) 0x7fdbe408e180 con 0x7fdbf4101a30 2024-03-21T13:27:01.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.398+0000 7fdbea7fc700 1 --2- 172.21.15.97:0/1595627088 >> v2:172.21.15.97:6801/2691001097 conn(0x7fdbe0077ba0 0x7fdbe007a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:01.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.398+0000 7fdbea7fc700 1 -- 172.21.15.97:0/1595627088 --> v2:172.21.15.97:6801/2691001097 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdbe4020e50 con 0x7fdbe0077ba0 2024-03-21T13:27:01.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.398+0000 7fdbea7fc700 1 -- 172.21.15.97:0/1595627088 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdbe4095050 con 0x7fdbf4101a30 2024-03-21T13:27:01.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.398+0000 7fdbf9eeb700 1 --2- 172.21.15.97:0/1595627088 >> v2:172.21.15.97:6801/2691001097 conn(0x7fdbe0077ba0 0x7fdbe007a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:01.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.398+0000 7fdbf96ea700 1 --2- 172.21.15.97:0/1595627088 >> v2:172.21.15.97:6800/12494470 conn(0x7fdbe006e300 0x7fdbe00707c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0009680 tx=0x7fdbf0019040 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:27:01.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.398+0000 7fdbf9eeb700 1 --2- 172.21.15.97:0/1595627088 >> v2:172.21.15.97:6801/2691001097 conn(0x7fdbe0077ba0 0x7fdbe007a040 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:27:01.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.399+0000 7fdbea7fc700 1 -- 172.21.15.97:0/1595627088 <== osd.0 v2:172.21.15.97:6801/2691001097 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fdbe4020e50 con 0x7fdbe0077ba0 2024-03-21T13:27:01.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.415+0000 7fdbfb94e700 1 -- 172.21.15.97:0/1595627088 --> v2:172.21.15.97:6801/2691001097 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdbf4102780 con 0x7fdbe0077ba0 2024-03-21T13:27:01.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.416+0000 7fdbea7fc700 1 -- 172.21.15.97:0/1595627088 <== osd.0 v2:172.21.15.97:6801/2691001097 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fdbf4102780 con 0x7fdbe0077ba0 2024-03-21T13:27:01.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.416+0000 7fdbfb94e700 1 -- 172.21.15.97:0/1595627088 >> v2:172.21.15.97:6801/2691001097 conn(0x7fdbe0077ba0 msgr2=0x7fdbe007a040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:01.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.416+0000 7fdbfb94e700 1 --2- 172.21.15.97:0/1595627088 >> v2:172.21.15.97:6801/2691001097 conn(0x7fdbe0077ba0 0x7fdbe007a040 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:01.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.416+0000 7fdbfb94e700 1 -- 172.21.15.97:0/1595627088 >> v2:172.21.15.97:6800/12494470 conn(0x7fdbe006e300 msgr2=0x7fdbe00707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:01.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.416+0000 7fdbfb94e700 1 --2- 172.21.15.97:0/1595627088 >> v2:172.21.15.97:6800/12494470 conn(0x7fdbe006e300 0x7fdbe00707c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0009680 tx=0x7fdbf0019040 comp rx=0 tx=0).stop 2024-03-21T13:27:01.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.416+0000 7fdbfb94e700 1 -- 172.21.15.97:0/1595627088 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdbf4101a30 msgr2=0x7fdbf4198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:01.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.416+0000 7fdbfb94e700 1 --2- 172.21.15.97:0/1595627088 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdbf4101a30 0x7fdbf4198690 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdbe400e9c0 tx=0x7fdbe400ecd0 comp rx=0 tx=0).stop 2024-03-21T13:27:01.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.416+0000 7fdbf9eeb700 1 -- 172.21.15.97:0/1595627088 reap_dead start 2024-03-21T13:27:01.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.416+0000 7fdbfb94e700 1 -- 172.21.15.97:0/1595627088 shutdown_connections 2024-03-21T13:27:01.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.416+0000 7fdbfb94e700 1 -- 172.21.15.97:0/1595627088 >> 172.21.15.97:0/1595627088 conn(0x7fdbf4077fc0 msgr2=0x7fdbf4069e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:27:01.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.416+0000 7fdbfb94e700 1 -- 172.21.15.97:0/1595627088 shutdown_connections 2024-03-21T13:27:01.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.416+0000 7fdbfb94e700 1 -- 172.21.15.97:0/1595627088 wait complete. 2024-03-21T13:27:01.952 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:01 smithi097 ceph-mon[39416]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-03-21T13:27:01.952 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:01 smithi097 ceph-mon[30040]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-03-21T13:27:02.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:01 smithi144 ceph-mon[33633]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-03-21T13:27:02.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.293+0000 7f8c6294b700 1 -- 172.21.15.97:0/1844886089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c5c107120 msgr2=0x7f8c5c107500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:02.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.293+0000 7f8c6294b700 1 --2- 172.21.15.97:0/1844886089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c5c107120 0x7f8c5c107500 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c009990 tx=0x7f8c4c009ca0 comp rx=0 tx=0).stop 2024-03-21T13:27:02.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.293+0000 7f8c6294b700 1 -- 172.21.15.97:0/1844886089 shutdown_connections 2024-03-21T13:27:02.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.293+0000 7f8c6294b700 1 --2- 172.21.15.97:0/1844886089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c5c101a30 0x7f8c5c10ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:02.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.293+0000 7f8c6294b700 1 --2- 172.21.15.97:0/1844886089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c5c107120 0x7f8c5c107500 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:02.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.293+0000 7f8c6294b700 1 --2- 172.21.15.97:0/1844886089 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8c5c101070 0x7f8c5c1014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:02.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.293+0000 7f8c6294b700 1 -- 172.21.15.97:0/1844886089 >> 172.21.15.97:0/1844886089 conn(0x7f8c5c0780a0 msgr2=0x7f8c5c0784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:27:02.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.293+0000 7f8c6294b700 1 -- 172.21.15.97:0/1844886089 shutdown_connections 2024-03-21T13:27:02.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.294+0000 7f8c6294b700 1 -- 172.21.15.97:0/1844886089 wait complete. 2024-03-21T13:27:02.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.294+0000 7f8c6294b700 1 Processor -- start 2024-03-21T13:27:02.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.294+0000 7f8c6294b700 1 -- start start 2024-03-21T13:27:02.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.295+0000 7f8c6294b700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8c5c101070 0x7f8c5c197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:02.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.295+0000 7f8c6294b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c5c101a30 0x7f8c5c1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:02.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.295+0000 7f8c6294b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c5c107120 0x7f8c5c19c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:02.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.295+0000 7f8c6294b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5c1109f0 con 0x7f8c5c107120 2024-03-21T13:27:02.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.295+0000 7f8c6294b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5c110870 con 0x7f8c5c101a30 2024-03-21T13:27:02.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.295+0000 7f8c6294b700 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5c110b70 con 0x7f8c5c101070 2024-03-21T13:27:02.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.295+0000 7f8c5bfff700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8c5c101070 0x7f8c5c197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:02.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.295+0000 7f8c60ee8700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c5c107120 0x7f8c5c19c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:02.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.295+0000 7f8c60ee8700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c5c107120 0x7f8c5c19c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46188/0 (socket says 172.21.15.97:46188) 2024-03-21T13:27:02.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.295+0000 7f8c60ee8700 1 -- 172.21.15.97:0/340049344 learned_addr learned my addr 172.21.15.97:0/340049344 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:27:02.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.295+0000 7f8c5b7fe700 1 --2- 172.21.15.97:0/340049344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c5c101a30 0x7f8c5c1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:02.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.295+0000 7f8c60ee8700 1 -- 172.21.15.97:0/340049344 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8c5c101070 msgr2=0x7f8c5c197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:02.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.295+0000 7f8c60ee8700 1 --2- 172.21.15.97:0/340049344 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8c5c101070 0x7f8c5c197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:02.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.295+0000 7f8c60ee8700 1 -- 172.21.15.97:0/340049344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c5c101a30 msgr2=0x7f8c5c1984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:02.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.295+0000 7f8c60ee8700 1 --2- 172.21.15.97:0/340049344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c5c101a30 0x7f8c5c1984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:02.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.295+0000 7f8c60ee8700 1 -- 172.21.15.97:0/340049344 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c5c19cf70 con 0x7f8c5c107120 2024-03-21T13:27:02.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.295+0000 7f8c5bfff700 1 --2- 172.21.15.97:0/340049344 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8c5c101070 0x7f8c5c197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T13:27:02.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.295+0000 7f8c60ee8700 1 --2- 172.21.15.97:0/340049344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c5c107120 0x7f8c5c19c830 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8c54007ee0 tx=0x7f8c5400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:27:02.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.296+0000 7f8c597fa700 1 -- 172.21.15.97:0/340049344 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5400ddd0 con 0x7f8c5c107120 2024-03-21T13:27:02.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.296+0000 7f8c6294b700 1 -- 172.21.15.97:0/340049344 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c5c19d260 con 0x7f8c5c107120 2024-03-21T13:27:02.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.296+0000 7f8c597fa700 1 -- 172.21.15.97:0/340049344 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f8c54004020 con 0x7f8c5c107120 2024-03-21T13:27:02.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.296+0000 7f8c597fa700 1 -- 172.21.15.97:0/340049344 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c54015620 con 0x7f8c5c107120 2024-03-21T13:27:02.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.296+0000 7f8c6294b700 1 -- 172.21.15.97:0/340049344 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c5c1a4ae0 con 0x7f8c5c107120 2024-03-21T13:27:02.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.297+0000 7f8c6294b700 1 -- 172.21.15.97:0/340049344 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8c5c06b290 con 0x7f8c5c107120 2024-03-21T13:27:02.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.297+0000 7f8c597fa700 1 -- 172.21.15.97:0/340049344 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 22) v1 ==== 92152+0+0 (secure 0 0 0) 0x7f8c54003a00 con 0x7f8c5c107120 2024-03-21T13:27:02.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.297+0000 7f8c597fa700 1 --2- 172.21.15.97:0/340049344 >> v2:172.21.15.97:6800/12494470 conn(0x7f8c4406e300 0x7f8c440707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:02.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.298+0000 7f8c597fa700 1 -- 172.21.15.97:0/340049344 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f8c5408dde0 con 0x7f8c5c107120 2024-03-21T13:27:02.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.298+0000 7f8c5bfff700 1 --2- 172.21.15.97:0/340049344 >> v2:172.21.15.97:6800/12494470 conn(0x7f8c4406e300 0x7f8c440707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:02.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.298+0000 7f8c597fa700 1 --2- 172.21.15.97:0/340049344 >> v2:172.21.15.144:6812/3269572470 conn(0x7f8c44077ba0 0x7f8c4407a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:02.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.298+0000 7f8c597fa700 1 -- 172.21.15.97:0/340049344 --> v2:172.21.15.144:6812/3269572470 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8c54020e50 con 0x7f8c44077ba0 2024-03-21T13:27:02.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.298+0000 7f8c597fa700 1 -- 172.21.15.97:0/340049344 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c540904e0 con 0x7f8c5c107120 2024-03-21T13:27:02.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.298+0000 7f8c5bfff700 1 --2- 172.21.15.97:0/340049344 >> v2:172.21.15.97:6800/12494470 conn(0x7f8c4406e300 0x7f8c440707c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c00b2e0 tx=0x7f8c4c01a040 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:27:02.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.298+0000 7f8c5b7fe700 1 --2- 172.21.15.97:0/340049344 >> v2:172.21.15.144:6812/3269572470 conn(0x7f8c44077ba0 0x7f8c4407a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:02.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.298+0000 7f8c5b7fe700 1 --2- 172.21.15.97:0/340049344 >> v2:172.21.15.144:6812/3269572470 conn(0x7f8c44077ba0 0x7f8c4407a040 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:27:02.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.299+0000 7f8c597fa700 1 -- 172.21.15.97:0/340049344 <== osd.7 v2:172.21.15.144:6812/3269572470 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f8c54020e50 con 0x7f8c44077ba0 2024-03-21T13:27:02.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.316+0000 7f8c6294b700 1 -- 172.21.15.97:0/340049344 --> v2:172.21.15.144:6812/3269572470 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8c5c102760 con 0x7f8c44077ba0 2024-03-21T13:27:02.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.316+0000 7f8c597fa700 1 -- 172.21.15.97:0/340049344 <== osd.7 v2:172.21.15.144:6812/3269572470 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f8c5c102760 con 0x7f8c44077ba0 2024-03-21T13:27:02.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.316+0000 7f8c6294b700 1 -- 172.21.15.97:0/340049344 >> v2:172.21.15.144:6812/3269572470 conn(0x7f8c44077ba0 msgr2=0x7f8c4407a040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:02.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.316+0000 7f8c6294b700 1 --2- 172.21.15.97:0/340049344 >> v2:172.21.15.144:6812/3269572470 conn(0x7f8c44077ba0 0x7f8c4407a040 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:02.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.317+0000 7f8c6294b700 1 -- 172.21.15.97:0/340049344 >> v2:172.21.15.97:6800/12494470 conn(0x7f8c4406e300 msgr2=0x7f8c440707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:02.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.317+0000 7f8c6294b700 1 --2- 172.21.15.97:0/340049344 >> v2:172.21.15.97:6800/12494470 conn(0x7f8c4406e300 0x7f8c440707c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c00b2e0 tx=0x7f8c4c01a040 comp rx=0 tx=0).stop 2024-03-21T13:27:02.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.317+0000 7f8c6294b700 1 -- 172.21.15.97:0/340049344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c5c107120 msgr2=0x7f8c5c19c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:02.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.317+0000 7f8c6294b700 1 --2- 172.21.15.97:0/340049344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c5c107120 0x7f8c5c19c830 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8c54007ee0 tx=0x7f8c5400d3b0 comp rx=0 tx=0).stop 2024-03-21T13:27:02.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.317+0000 7f8c60ee8700 1 -- 172.21.15.97:0/340049344 reap_dead start 2024-03-21T13:27:02.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.317+0000 7f8c6294b700 1 -- 172.21.15.97:0/340049344 shutdown_connections 2024-03-21T13:27:02.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.317+0000 7f8c6294b700 1 -- 172.21.15.97:0/340049344 >> 172.21.15.97:0/340049344 conn(0x7f8c5c0780a0 msgr2=0x7f8c5c06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:27:02.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.317+0000 7f8c6294b700 1 -- 172.21.15.97:0/340049344 shutdown_connections 2024-03-21T13:27:02.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:01.317+0000 7f8c6294b700 1 -- 172.21.15.97:0/340049344 wait complete. 2024-03-21T13:27:02.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:27:01 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:27:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:27:03.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.407+0000 7f3bf30eb700 1 -- 172.21.15.97:0/4044052491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bec101360 msgr2=0x7f3bec101740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:03.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.407+0000 7f3bf30eb700 1 --2- 172.21.15.97:0/4044052491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bec101360 0x7f3bec101740 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3be0009920 tx=0x7f3be0009c30 comp rx=0 tx=0).stop 2024-03-21T13:27:03.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.407+0000 7f3bf30eb700 1 -- 172.21.15.97:0/4044052491 shutdown_connections 2024-03-21T13:27:03.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.407+0000 7f3bf30eb700 1 --2- 172.21.15.97:0/4044052491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3bec106440 0x7f3bec10e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:03.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.407+0000 7f3bf30eb700 1 --2- 172.21.15.97:0/4044052491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bec101360 0x7f3bec101740 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:03.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.407+0000 7f3bf30eb700 1 --2- 172.21.15.97:0/4044052491 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3bec101d10 0x7f3bec105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:03.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.407+0000 7f3bf30eb700 1 -- 172.21.15.97:0/4044052491 >> 172.21.15.97:0/4044052491 conn(0x7f3bec0781c0 msgr2=0x7f3bec0785d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:27:03.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.407+0000 7f3bf30eb700 1 -- 172.21.15.97:0/4044052491 shutdown_connections 2024-03-21T13:27:03.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.407+0000 7f3bf30eb700 1 -- 172.21.15.97:0/4044052491 wait complete. 2024-03-21T13:27:03.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.408+0000 7f3bf30eb700 1 Processor -- start 2024-03-21T13:27:03.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.408+0000 7f3bf30eb700 1 -- start start 2024-03-21T13:27:03.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.408+0000 7f3bf30eb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bec101360 0x7f3bec1980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:03.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.408+0000 7f3bf30eb700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3bec101d10 0x7f3bec198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:03.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.408+0000 7f3bf30eb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3bec106440 0x7f3bec19c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:03.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.408+0000 7f3bf30eb700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec110ae0 con 0x7f3bec101360 2024-03-21T13:27:03.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.408+0000 7f3bf30eb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec110960 con 0x7f3bec106440 2024-03-21T13:27:03.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.408+0000 7f3bf30eb700 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec110c60 con 0x7f3bec101d10 2024-03-21T13:27:03.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.408+0000 7f3bebfff700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3bec101d10 0x7f3bec198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:03.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.408+0000 7f3bf0e87700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bec101360 0x7f3bec1980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:03.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.408+0000 7f3bebfff700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3bec101d10 0x7f3bec198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.97:54166/0 (socket says 172.21.15.97:54166) 2024-03-21T13:27:03.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.408+0000 7f3bf0e87700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bec101360 0x7f3bec1980e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46214/0 (socket says 172.21.15.97:46214) 2024-03-21T13:27:03.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.408+0000 7f3bebfff700 1 -- 172.21.15.97:0/362636430 learned_addr learned my addr 172.21.15.97:0/362636430 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:27:03.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.408+0000 7f3bf1688700 1 --2- 172.21.15.97:0/362636430 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3bec106440 0x7f3bec19c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:03.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.408+0000 7f3bebfff700 1 -- 172.21.15.97:0/362636430 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3bec106440 msgr2=0x7f3bec19c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:03.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.408+0000 7f3bebfff700 1 --2- 172.21.15.97:0/362636430 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3bec106440 0x7f3bec19c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:03.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.408+0000 7f3bebfff700 1 -- 172.21.15.97:0/362636430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bec101360 msgr2=0x7f3bec1980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:03.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.408+0000 7f3bebfff700 1 --2- 172.21.15.97:0/362636430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bec101360 0x7f3bec1980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:03.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.408+0000 7f3bebfff700 1 -- 172.21.15.97:0/362636430 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bec19d0f0 con 0x7f3bec101d10 2024-03-21T13:27:03.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.408+0000 7f3bf0e87700 1 --2- 172.21.15.97:0/362636430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bec101360 0x7f3bec1980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T13:27:03.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.408+0000 7f3bebfff700 1 --2- 172.21.15.97:0/362636430 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3bec101d10 0x7f3bec198620 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3bd800ec70 tx=0x7f3bd800ef80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:27:03.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.408+0000 7f3be9ffb700 1 -- 172.21.15.97:0/362636430 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd800cc90 con 0x7f3bec101d10 2024-03-21T13:27:03.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.409+0000 7f3bf30eb700 1 -- 172.21.15.97:0/362636430 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bec19d3e0 con 0x7f3bec101d10 2024-03-21T13:27:03.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.409+0000 7f3be9ffb700 1 -- 172.21.15.97:0/362636430 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f3bd800ce30 con 0x7f3bec101d10 2024-03-21T13:27:03.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.409+0000 7f3be9ffb700 1 -- 172.21.15.97:0/362636430 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd8010430 con 0x7f3bec101d10 2024-03-21T13:27:03.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.409+0000 7f3bf30eb700 1 -- 172.21.15.97:0/362636430 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bec1a4c60 con 0x7f3bec101d10 2024-03-21T13:27:03.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.409+0000 7f3bf30eb700 1 -- 172.21.15.97:0/362636430 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3bec105a30 con 0x7f3bec101d10 2024-03-21T13:27:03.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.410+0000 7f3be9ffb700 1 -- 172.21.15.97:0/362636430 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 22) v1 ==== 92152+0+0 (secure 0 0 0) 0x7f3bd8003f40 con 0x7f3bec101d10 2024-03-21T13:27:03.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.410+0000 7f3be9ffb700 1 --2- 172.21.15.97:0/362636430 >> v2:172.21.15.97:6800/12494470 conn(0x7f3bdc06e300 0x7f3bdc0707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:03.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.410+0000 7f3be9ffb700 1 -- 172.21.15.97:0/362636430 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f3bd808e700 con 0x7f3bec101d10 2024-03-21T13:27:03.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.410+0000 7f3bf0e87700 1 --2- 172.21.15.97:0/362636430 >> v2:172.21.15.97:6800/12494470 conn(0x7f3bdc06e300 0x7f3bdc0707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:03.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.410+0000 7f3be9ffb700 1 --2- 172.21.15.97:0/362636430 >> v2:172.21.15.97:6813/1678932102 conn(0x7f3bdc077ba0 0x7f3bdc07a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:03.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.410+0000 7f3be9ffb700 1 -- 172.21.15.97:0/362636430 --> v2:172.21.15.97:6813/1678932102 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3bd800be50 con 0x7f3bdc077ba0 2024-03-21T13:27:03.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.411+0000 7f3be9ffb700 1 -- 172.21.15.97:0/362636430 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3bd8095050 con 0x7f3bec101d10 2024-03-21T13:27:03.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.411+0000 7f3bf1688700 1 --2- 172.21.15.97:0/362636430 >> v2:172.21.15.97:6813/1678932102 conn(0x7f3bdc077ba0 0x7f3bdc07a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:03.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.411+0000 7f3bf0e87700 1 --2- 172.21.15.97:0/362636430 >> v2:172.21.15.97:6800/12494470 conn(0x7f3bdc06e300 0x7f3bdc0707c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3be0009680 tx=0x7f3be0019040 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:27:03.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.411+0000 7f3bf1688700 1 --2- 172.21.15.97:0/362636430 >> v2:172.21.15.97:6813/1678932102 conn(0x7f3bdc077ba0 0x7f3bdc07a040 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:27:03.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.412+0000 7f3be9ffb700 1 -- 172.21.15.97:0/362636430 <== osd.3 v2:172.21.15.97:6813/1678932102 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f3bd800be50 con 0x7f3bdc077ba0 2024-03-21T13:27:03.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.429+0000 7f3bf30eb700 1 -- 172.21.15.97:0/362636430 --> v2:172.21.15.97:6813/1678932102 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3bec102840 con 0x7f3bdc077ba0 2024-03-21T13:27:03.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.429+0000 7f3be9ffb700 1 -- 172.21.15.97:0/362636430 <== osd.3 v2:172.21.15.97:6813/1678932102 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f3bec102840 con 0x7f3bdc077ba0 2024-03-21T13:27:03.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.429+0000 7f3bf30eb700 1 -- 172.21.15.97:0/362636430 >> v2:172.21.15.97:6813/1678932102 conn(0x7f3bdc077ba0 msgr2=0x7f3bdc07a040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:03.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.429+0000 7f3bf30eb700 1 --2- 172.21.15.97:0/362636430 >> v2:172.21.15.97:6813/1678932102 conn(0x7f3bdc077ba0 0x7f3bdc07a040 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:03.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.429+0000 7f3bf30eb700 1 -- 172.21.15.97:0/362636430 >> v2:172.21.15.97:6800/12494470 conn(0x7f3bdc06e300 msgr2=0x7f3bdc0707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:03.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.429+0000 7f3bf30eb700 1 --2- 172.21.15.97:0/362636430 >> v2:172.21.15.97:6800/12494470 conn(0x7f3bdc06e300 0x7f3bdc0707c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3be0009680 tx=0x7f3be0019040 comp rx=0 tx=0).stop 2024-03-21T13:27:03.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.429+0000 7f3bf30eb700 1 -- 172.21.15.97:0/362636430 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3bec101d10 msgr2=0x7f3bec198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:03.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.429+0000 7f3bf30eb700 1 --2- 172.21.15.97:0/362636430 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3bec101d10 0x7f3bec198620 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3bd800ec70 tx=0x7f3bd800ef80 comp rx=0 tx=0).stop 2024-03-21T13:27:03.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.429+0000 7f3bf1688700 1 -- 172.21.15.97:0/362636430 reap_dead start 2024-03-21T13:27:03.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.429+0000 7f3bf30eb700 1 -- 172.21.15.97:0/362636430 shutdown_connections 2024-03-21T13:27:03.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.429+0000 7f3bf30eb700 1 -- 172.21.15.97:0/362636430 >> 172.21.15.97:0/362636430 conn(0x7f3bec0781c0 msgr2=0x7f3bec105050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:27:03.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.430+0000 7f3bf30eb700 1 -- 172.21.15.97:0/362636430 shutdown_connections 2024-03-21T13:27:03.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:03.430+0000 7f3bf30eb700 1 -- 172.21.15.97:0/362636430 wait complete. 2024-03-21T13:27:03.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:27:03 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:27:04.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:03 smithi097 ceph-mon[39416]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 24 KiB/s rd, 0 B/s wr, 13 op/s 2024-03-21T13:27:04.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:03 smithi097 ceph-mon[30040]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 24 KiB/s rd, 0 B/s wr, 13 op/s 2024-03-21T13:27:04.097 INFO:teuthology.orchestra.run.smithi097.stdout:34359738419 2024-03-21T13:27:04.097 DEBUG:teuthology.orchestra.run.smithi097:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-21T13:27:04.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:03 smithi144 ceph-mon[33633]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 24 KiB/s rd, 0 B/s wr, 13 op/s 2024-03-21T13:27:04.478 INFO:teuthology.orchestra.run.smithi097.stdout:107374182436 2024-03-21T13:27:04.479 DEBUG:teuthology.orchestra.run.smithi097:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-21T13:27:04.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.687+0000 7f6acaf5d700 1 -- 172.21.15.97:0/3483107162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ac41081c0 msgr2=0x7f6ac41085a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:04.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.687+0000 7f6acaf5d700 1 --2- 172.21.15.97:0/3483107162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ac41081c0 0x7f6ac41085a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6abc009fc0 tx=0x7f6abc00b330 comp rx=0 tx=0).stop 2024-03-21T13:27:04.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.687+0000 7f6acaf5d700 1 -- 172.21.15.97:0/3483107162 shutdown_connections 2024-03-21T13:27:04.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.687+0000 7f6acaf5d700 1 --2- 172.21.15.97:0/3483107162 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6ac4102ba0 0x7f6ac410efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:04.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.687+0000 7f6acaf5d700 1 --2- 172.21.15.97:0/3483107162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ac41081c0 0x7f6ac41085a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:04.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.687+0000 7f6acaf5d700 1 --2- 172.21.15.97:0/3483107162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ac41021e0 0x7f6ac4102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:04.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.687+0000 7f6acaf5d700 1 -- 172.21.15.97:0/3483107162 >> 172.21.15.97:0/3483107162 conn(0x7f6ac40792c0 msgr2=0x7f6ac40796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:27:04.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.687+0000 7f6acaf5d700 1 -- 172.21.15.97:0/3483107162 shutdown_connections 2024-03-21T13:27:04.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.687+0000 7f6acaf5d700 1 -- 172.21.15.97:0/3483107162 wait complete. 2024-03-21T13:27:04.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.688+0000 7f6acaf5d700 1 Processor -- start 2024-03-21T13:27:04.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.688+0000 7f6acaf5d700 1 -- start start 2024-03-21T13:27:04.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.688+0000 7f6acaf5d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ac41021e0 0x7f6ac4197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:04.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.688+0000 7f6acaf5d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ac4102ba0 0x7f6ac4198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:04.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.688+0000 7f6acaf5d700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6ac41081c0 0x7f6ac419c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:04.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.688+0000 7f6acaf5d700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac4111af0 con 0x7f6ac4102ba0 2024-03-21T13:27:04.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.688+0000 7f6acaf5d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac4111970 con 0x7f6ac41021e0 2024-03-21T13:27:04.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.688+0000 7f6acaf5d700 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac4111c70 con 0x7f6ac41081c0 2024-03-21T13:27:04.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.688+0000 7f6ac3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ac4102ba0 0x7f6ac4198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:04.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.688+0000 7f6ac3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ac4102ba0 0x7f6ac4198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46220/0 (socket says 172.21.15.97:46220) 2024-03-21T13:27:04.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.688+0000 7f6ac94fa700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6ac41081c0 0x7f6ac419c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:04.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.688+0000 7f6ac3fff700 1 -- 172.21.15.97:0/3874840657 learned_addr learned my addr 172.21.15.97:0/3874840657 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:27:04.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.688+0000 7f6ac8cf9700 1 --2- 172.21.15.97:0/3874840657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ac41021e0 0x7f6ac4197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:04.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.689+0000 7f6ac94fa700 1 -- 172.21.15.97:0/3874840657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ac41021e0 msgr2=0x7f6ac4197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:04.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.689+0000 7f6ac94fa700 1 --2- 172.21.15.97:0/3874840657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ac41021e0 0x7f6ac4197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:04.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.689+0000 7f6ac94fa700 1 -- 172.21.15.97:0/3874840657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ac4102ba0 msgr2=0x7f6ac4198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:04.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.689+0000 7f6ac94fa700 1 --2- 172.21.15.97:0/3874840657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ac4102ba0 0x7f6ac4198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:04.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.689+0000 7f6ac94fa700 1 -- 172.21.15.97:0/3874840657 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ac419d000 con 0x7f6ac41081c0 2024-03-21T13:27:04.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.689+0000 7f6ac3fff700 1 --2- 172.21.15.97:0/3874840657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ac4102ba0 0x7f6ac4198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T13:27:04.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.689+0000 7f6ac94fa700 1 --2- 172.21.15.97:0/3874840657 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6ac41081c0 0x7f6ac419c8c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6ab400eb10 tx=0x7f6ab400ee20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:27:04.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.689+0000 7f6ac1ffb700 1 -- 172.21.15.97:0/3874840657 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab400cc30 con 0x7f6ac41081c0 2024-03-21T13:27:04.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.689+0000 7f6ac1ffb700 1 -- 172.21.15.97:0/3874840657 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f6ab40044e0 con 0x7f6ac41081c0 2024-03-21T13:27:04.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.689+0000 7f6ac1ffb700 1 -- 172.21.15.97:0/3874840657 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab4010430 con 0x7f6ac41081c0 2024-03-21T13:27:04.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.689+0000 7f6acaf5d700 1 -- 172.21.15.97:0/3874840657 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ac419d2f0 con 0x7f6ac41081c0 2024-03-21T13:27:04.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.689+0000 7f6acaf5d700 1 -- 172.21.15.97:0/3874840657 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ac41a4b70 con 0x7f6ac41081c0 2024-03-21T13:27:04.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.690+0000 7f6acaf5d700 1 -- 172.21.15.97:0/3874840657 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6ac4107760 con 0x7f6ac41081c0 2024-03-21T13:27:04.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.690+0000 7f6ac1ffb700 1 -- 172.21.15.97:0/3874840657 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 22) v1 ==== 92152+0+0 (secure 0 0 0) 0x7f6ab4004000 con 0x7f6ac41081c0 2024-03-21T13:27:04.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.691+0000 7f6ac1ffb700 1 --2- 172.21.15.97:0/3874840657 >> v2:172.21.15.97:6800/12494470 conn(0x7f6aac06e300 0x7f6aac0707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:04.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.691+0000 7f6ac1ffb700 1 -- 172.21.15.97:0/3874840657 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f6ab408e5d0 con 0x7f6ac41081c0 2024-03-21T13:27:04.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.691+0000 7f6ac8cf9700 1 --2- 172.21.15.97:0/3874840657 >> v2:172.21.15.97:6800/12494470 conn(0x7f6aac06e300 0x7f6aac0707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:04.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.691+0000 7f6ac1ffb700 1 --2- 172.21.15.97:0/3874840657 >> v2:172.21.15.144:6800/913882960 conn(0x7f6aac077ba0 0x7f6aac07a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:04.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.691+0000 7f6ac1ffb700 1 -- 172.21.15.97:0/3874840657 --> v2:172.21.15.144:6800/913882960 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6aac07a6d0 con 0x7f6aac077ba0 2024-03-21T13:27:04.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.691+0000 7f6ac1ffb700 1 -- 172.21.15.97:0/3874840657 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ab4095050 con 0x7f6ac41081c0 2024-03-21T13:27:04.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.691+0000 7f6ac8cf9700 1 --2- 172.21.15.97:0/3874840657 >> v2:172.21.15.97:6800/12494470 conn(0x7f6aac06e300 0x7f6aac0707c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6abc009790 tx=0x7f6abc0096e0 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:27:04.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.691+0000 7f6ac3fff700 1 --2- 172.21.15.97:0/3874840657 >> v2:172.21.15.144:6800/913882960 conn(0x7f6aac077ba0 0x7f6aac07a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:04.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.692+0000 7f6ac3fff700 1 --2- 172.21.15.97:0/3874840657 >> v2:172.21.15.144:6800/913882960 conn(0x7f6aac077ba0 0x7f6aac07a040 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:27:04.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.693+0000 7f6ac1ffb700 1 -- 172.21.15.97:0/3874840657 <== osd.4 v2:172.21.15.144:6800/913882960 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f6aac07a6d0 con 0x7f6aac077ba0 2024-03-21T13:27:04.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.709+0000 7f6acaf5d700 1 -- 172.21.15.97:0/3874840657 --> v2:172.21.15.144:6800/913882960 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6ac4103870 con 0x7f6aac077ba0 2024-03-21T13:27:04.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.709+0000 7f6ac1ffb700 1 -- 172.21.15.97:0/3874840657 <== osd.4 v2:172.21.15.144:6800/913882960 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f6ac4103870 con 0x7f6aac077ba0 2024-03-21T13:27:04.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.710+0000 7f6acaf5d700 1 -- 172.21.15.97:0/3874840657 >> v2:172.21.15.144:6800/913882960 conn(0x7f6aac077ba0 msgr2=0x7f6aac07a040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:04.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.710+0000 7f6acaf5d700 1 --2- 172.21.15.97:0/3874840657 >> v2:172.21.15.144:6800/913882960 conn(0x7f6aac077ba0 0x7f6aac07a040 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:04.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.710+0000 7f6acaf5d700 1 -- 172.21.15.97:0/3874840657 >> v2:172.21.15.97:6800/12494470 conn(0x7f6aac06e300 msgr2=0x7f6aac0707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:04.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.710+0000 7f6acaf5d700 1 --2- 172.21.15.97:0/3874840657 >> v2:172.21.15.97:6800/12494470 conn(0x7f6aac06e300 0x7f6aac0707c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6abc009790 tx=0x7f6abc0096e0 comp rx=0 tx=0).stop 2024-03-21T13:27:04.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.710+0000 7f6acaf5d700 1 -- 172.21.15.97:0/3874840657 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6ac41081c0 msgr2=0x7f6ac419c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:04.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.710+0000 7f6acaf5d700 1 --2- 172.21.15.97:0/3874840657 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6ac41081c0 0x7f6ac419c8c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6ab400eb10 tx=0x7f6ab400ee20 comp rx=0 tx=0).stop 2024-03-21T13:27:04.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.710+0000 7f6ac94fa700 1 -- 172.21.15.97:0/3874840657 reap_dead start 2024-03-21T13:27:04.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.710+0000 7f6acaf5d700 1 -- 172.21.15.97:0/3874840657 shutdown_connections 2024-03-21T13:27:04.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.710+0000 7f6acaf5d700 1 -- 172.21.15.97:0/3874840657 >> 172.21.15.97:0/3874840657 conn(0x7f6ac40792c0 msgr2=0x7f6ac4105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:27:04.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.710+0000 7f6acaf5d700 1 -- 172.21.15.97:0/3874840657 shutdown_connections 2024-03-21T13:27:04.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:04.710+0000 7f6acaf5d700 1 -- 172.21.15.97:0/3874840657 wait complete. 2024-03-21T13:27:05.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:04 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:27:05.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:04 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:27:05.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:04 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:27:05.443 INFO:teuthology.orchestra.run.smithi097.stdout:231928233998 2024-03-21T13:27:05.444 DEBUG:teuthology.orchestra.run.smithi097:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-03-21T13:27:06.006 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:05 smithi097 ceph-mon[39416]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-21T13:27:06.006 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:05 smithi097 ceph-mon[30040]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-21T13:27:06.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:05 smithi144 ceph-mon[33633]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-21T13:27:06.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.108+0000 7ff8be13d700 1 -- 172.21.15.97:0/1969064785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8b8101490 msgr2=0x7ff8b810d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:06.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.108+0000 7ff8be13d700 1 --2- 172.21.15.97:0/1969064785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8b8101490 0x7ff8b810d6d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8009990 tx=0x7ff8a8009ca0 comp rx=0 tx=0).stop 2024-03-21T13:27:06.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.108+0000 7ff8be13d700 1 -- 172.21.15.97:0/1969064785 shutdown_connections 2024-03-21T13:27:06.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.108+0000 7ff8be13d700 1 --2- 172.21.15.97:0/1969064785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8b8101490 0x7ff8b810d6d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:06.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.108+0000 7ff8be13d700 1 --2- 172.21.15.97:0/1969064785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8b81068b0 0x7ff8b8106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:06.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.108+0000 7ff8be13d700 1 --2- 172.21.15.97:0/1969064785 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff8b8100ad0 0x7ff8b8100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:06.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.108+0000 7ff8be13d700 1 -- 172.21.15.97:0/1969064785 >> 172.21.15.97:0/1969064785 conn(0x7ff8b80763d0 msgr2=0x7ff8b80767e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:27:06.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.108+0000 7ff8be13d700 1 -- 172.21.15.97:0/1969064785 shutdown_connections 2024-03-21T13:27:06.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.108+0000 7ff8be13d700 1 -- 172.21.15.97:0/1969064785 wait complete. 2024-03-21T13:27:06.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.109+0000 7ff8be13d700 1 Processor -- start 2024-03-21T13:27:06.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.109+0000 7ff8be13d700 1 -- start start 2024-03-21T13:27:06.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.109+0000 7ff8be13d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8b8100ad0 0x7ff8b819dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:06.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.109+0000 7ff8be13d700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff8b8101490 0x7ff8b819e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:06.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.109+0000 7ff8be13d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8b81068b0 0x7ff8b8111020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:06.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.109+0000 7ff8be13d700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b8110290 con 0x7ff8b8100ad0 2024-03-21T13:27:06.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.109+0000 7ff8be13d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b8110110 con 0x7ff8b81068b0 2024-03-21T13:27:06.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.109+0000 7ff8be13d700 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b8110410 con 0x7ff8b8101490 2024-03-21T13:27:06.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.109+0000 7ff8b6ffd700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff8b8101490 0x7ff8b819e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:06.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.109+0000 7ff8b6ffd700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff8b8101490 0x7ff8b819e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.97:54200/0 (socket says 172.21.15.97:54200) 2024-03-21T13:27:06.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.109+0000 7ff8b77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8b8100ad0 0x7ff8b819dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:06.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.109+0000 7ff8b6ffd700 1 -- 172.21.15.97:0/7691382 learned_addr learned my addr 172.21.15.97:0/7691382 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:27:06.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.109+0000 7ff8b7fff700 1 --2- 172.21.15.97:0/7691382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8b81068b0 0x7ff8b8111020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:06.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.109+0000 7ff8b6ffd700 1 -- 172.21.15.97:0/7691382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8b81068b0 msgr2=0x7ff8b8111020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:06.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.109+0000 7ff8b6ffd700 1 --2- 172.21.15.97:0/7691382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8b81068b0 0x7ff8b8111020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:06.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.109+0000 7ff8b6ffd700 1 -- 172.21.15.97:0/7691382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8b8100ad0 msgr2=0x7ff8b819dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:06.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.109+0000 7ff8b6ffd700 1 --2- 172.21.15.97:0/7691382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8b8100ad0 0x7ff8b819dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:06.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.109+0000 7ff8b6ffd700 1 -- 172.21.15.97:0/7691382 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b81117c0 con 0x7ff8b8101490 2024-03-21T13:27:06.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.109+0000 7ff8b77fe700 1 --2- 172.21.15.97:0/7691382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8b8100ad0 0x7ff8b819dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T13:27:06.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.109+0000 7ff8b7fff700 1 --2- 172.21.15.97:0/7691382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8b81068b0 0x7ff8b8111020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T13:27:06.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.110+0000 7ff8b6ffd700 1 --2- 172.21.15.97:0/7691382 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff8b8101490 0x7ff8b819e280 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac00eb30 tx=0x7ff8ac00ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:27:06.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.110+0000 7ff8b4ff9700 1 -- 172.21.15.97:0/7691382 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac00cc90 con 0x7ff8b8101490 2024-03-21T13:27:06.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.110+0000 7ff8be13d700 1 -- 172.21.15.97:0/7691382 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8b8111ab0 con 0x7ff8b8101490 2024-03-21T13:27:06.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.110+0000 7ff8b4ff9700 1 -- 172.21.15.97:0/7691382 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7ff8ac00ce30 con 0x7ff8b8101490 2024-03-21T13:27:06.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.110+0000 7ff8be13d700 1 -- 172.21.15.97:0/7691382 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8b81a4c80 con 0x7ff8b8101490 2024-03-21T13:27:06.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.110+0000 7ff8b4ff9700 1 -- 172.21.15.97:0/7691382 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac018700 con 0x7ff8b8101490 2024-03-21T13:27:06.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.111+0000 7ff8be13d700 1 -- 172.21.15.97:0/7691382 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff8b80ff090 con 0x7ff8b8101490 2024-03-21T13:27:06.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.111+0000 7ff8b4ff9700 1 -- 172.21.15.97:0/7691382 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 22) v1 ==== 92152+0+0 (secure 0 0 0) 0x7ff8ac01f020 con 0x7ff8b8101490 2024-03-21T13:27:06.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.112+0000 7ff8b4ff9700 1 --2- 172.21.15.97:0/7691382 >> v2:172.21.15.97:6800/12494470 conn(0x7ff8a006e300 0x7ff8a00707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:06.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.112+0000 7ff8b4ff9700 1 -- 172.21.15.97:0/7691382 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6259+0+0 (secure 0 0 0) 0x7ff8ac08e420 con 0x7ff8b8101490 2024-03-21T13:27:06.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.112+0000 7ff8b77fe700 1 --2- 172.21.15.97:0/7691382 >> v2:172.21.15.97:6800/12494470 conn(0x7ff8a006e300 0x7ff8a00707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:06.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.112+0000 7ff8b4ff9700 1 --2- 172.21.15.97:0/7691382 >> v2:172.21.15.144:6804/2553466741 conn(0x7ff8a0077ba0 0x7ff8a007a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:06.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.112+0000 7ff8b4ff9700 1 -- 172.21.15.97:0/7691382 --> v2:172.21.15.144:6804/2553466741 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff8ac023e50 con 0x7ff8a0077ba0 2024-03-21T13:27:06.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.112+0000 7ff8b4ff9700 1 -- 172.21.15.97:0/7691382 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8ac095050 con 0x7ff8b8101490 2024-03-21T13:27:06.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.112+0000 7ff8b77fe700 1 --2- 172.21.15.97:0/7691382 >> v2:172.21.15.97:6800/12494470 conn(0x7ff8a006e300 0x7ff8a00707c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff8b000a7b0 tx=0x7ff8b000d040 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:27:06.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.112+0000 7ff8b7fff700 1 --2- 172.21.15.97:0/7691382 >> v2:172.21.15.144:6804/2553466741 conn(0x7ff8a0077ba0 0x7ff8a007a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:06.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.112+0000 7ff8b7fff700 1 --2- 172.21.15.97:0/7691382 >> v2:172.21.15.144:6804/2553466741 conn(0x7ff8a0077ba0 0x7ff8a007a040 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:27:06.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.113+0000 7ff8b4ff9700 1 -- 172.21.15.97:0/7691382 <== osd.5 v2:172.21.15.144:6804/2553466741 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ff8ac023e50 con 0x7ff8a0077ba0 2024-03-21T13:27:06.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.129+0000 7ff8be13d700 1 -- 172.21.15.97:0/7691382 --> v2:172.21.15.144:6804/2553466741 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff8b8101f40 con 0x7ff8a0077ba0 2024-03-21T13:27:06.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.130+0000 7ff8b4ff9700 1 -- 172.21.15.97:0/7691382 <== osd.5 v2:172.21.15.144:6804/2553466741 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ff8b8101f40 con 0x7ff8a0077ba0 2024-03-21T13:27:06.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.130+0000 7ff8be13d700 1 -- 172.21.15.97:0/7691382 >> v2:172.21.15.144:6804/2553466741 conn(0x7ff8a0077ba0 msgr2=0x7ff8a007a040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:06.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.130+0000 7ff8be13d700 1 --2- 172.21.15.97:0/7691382 >> v2:172.21.15.144:6804/2553466741 conn(0x7ff8a0077ba0 0x7ff8a007a040 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:06.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.130+0000 7ff8be13d700 1 -- 172.21.15.97:0/7691382 >> v2:172.21.15.97:6800/12494470 conn(0x7ff8a006e300 msgr2=0x7ff8a00707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:06.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.130+0000 7ff8be13d700 1 --2- 172.21.15.97:0/7691382 >> v2:172.21.15.97:6800/12494470 conn(0x7ff8a006e300 0x7ff8a00707c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff8b000a7b0 tx=0x7ff8b000d040 comp rx=0 tx=0).stop 2024-03-21T13:27:06.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.130+0000 7ff8be13d700 1 -- 172.21.15.97:0/7691382 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff8b8101490 msgr2=0x7ff8b819e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:06.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.130+0000 7ff8be13d700 1 --2- 172.21.15.97:0/7691382 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff8b8101490 0x7ff8b819e280 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac00eb30 tx=0x7ff8ac00ee40 comp rx=0 tx=0).stop 2024-03-21T13:27:06.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.130+0000 7ff8b7fff700 1 -- 172.21.15.97:0/7691382 reap_dead start 2024-03-21T13:27:06.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.130+0000 7ff8be13d700 1 -- 172.21.15.97:0/7691382 shutdown_connections 2024-03-21T13:27:06.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.130+0000 7ff8be13d700 1 -- 172.21.15.97:0/7691382 >> 172.21.15.97:0/7691382 conn(0x7ff8b80763d0 msgr2=0x7ff8b80fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:27:06.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.131+0000 7ff8be13d700 1 -- 172.21.15.97:0/7691382 shutdown_connections 2024-03-21T13:27:06.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:06.131+0000 7ff8be13d700 1 -- 172.21.15.97:0/7691382 wait complete. 2024-03-21T13:27:07.113 INFO:teuthology.orchestra.run.smithi097.stdout:128849018911 2024-03-21T13:27:07.114 DEBUG:teuthology.orchestra.run.smithi097:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-21T13:27:07.518 INFO:teuthology.orchestra.run.smithi097.stdout:77309411368 2024-03-21T13:27:07.518 DEBUG:teuthology.orchestra.run.smithi097:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-21T13:27:07.936 INFO:teuthology.orchestra.run.smithi097.stdout:163208757273 2024-03-21T13:27:07.936 DEBUG:teuthology.orchestra.run.smithi097:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-21T13:27:08.077 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:07 smithi097 ceph-mon[30040]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-21T13:27:08.077 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:07 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:27:08.078 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:07 smithi097 ceph-mon[39416]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-21T13:27:08.078 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:07 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:27:08.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:07 smithi144 ceph-mon[33633]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-21T13:27:08.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:07 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:27:08.144 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T13:27:08.147 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T13:27:08.410 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T13:27:08.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.667+0000 7f528a8ff700 1 -- 172.21.15.97:0/1702305949 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5284101a30 msgr2=0x7f528410df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:08.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.667+0000 7f528a8ff700 1 --2- 172.21.15.97:0/1702305949 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5284101a30 0x7f528410df10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f527c00b0a0 tx=0x7f527c00b3b0 comp rx=0 tx=0).stop 2024-03-21T13:27:08.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.667+0000 7f528a8ff700 1 -- 172.21.15.97:0/1702305949 shutdown_connections 2024-03-21T13:27:08.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.667+0000 7f528a8ff700 1 --2- 172.21.15.97:0/1702305949 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5284101a30 0x7f528410df10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:08.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.667+0000 7f528a8ff700 1 --2- 172.21.15.97:0/1702305949 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52841070d0 0x7f52841074b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:08.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.667+0000 7f528a8ff700 1 --2- 172.21.15.97:0/1702305949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5284101090 0x7f52841014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:08.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.667+0000 7f528a8ff700 1 -- 172.21.15.97:0/1702305949 >> 172.21.15.97:0/1702305949 conn(0x7f5284077fc0 msgr2=0x7f52840783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:27:08.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.667+0000 7f528a8ff700 1 -- 172.21.15.97:0/1702305949 shutdown_connections 2024-03-21T13:27:08.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.667+0000 7f528a8ff700 1 -- 172.21.15.97:0/1702305949 wait complete. 2024-03-21T13:27:08.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.668+0000 7f528a8ff700 1 Processor -- start 2024-03-21T13:27:08.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.668+0000 7f528a8ff700 1 -- start start 2024-03-21T13:27:08.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.668+0000 7f528a8ff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5284101090 0x7f5284196e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:08.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.668+0000 7f528a8ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5284101a30 0x7f5284197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:08.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.668+0000 7f528a8ff700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f52841070d0 0x7f528419b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:08.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.668+0000 7f528a8ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5284110a40 con 0x7f5284101a30 2024-03-21T13:27:08.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.668+0000 7f528a8ff700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52841108c0 con 0x7f5284101090 2024-03-21T13:27:08.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.668+0000 7f528a8ff700 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5284110bc0 con 0x7f52841070d0 2024-03-21T13:27:08.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.668+0000 7f52837fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5284101a30 0x7f5284197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:08.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.668+0000 7f52837fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5284101a30 0x7f5284197380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46254/0 (socket says 172.21.15.97:46254) 2024-03-21T13:27:08.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.668+0000 7f5288e9c700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f52841070d0 0x7f528419b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:08.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.668+0000 7f52837fe700 1 -- 172.21.15.97:0/3200163585 learned_addr learned my addr 172.21.15.97:0/3200163585 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:27:08.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.668+0000 7f5283fff700 1 --2- 172.21.15.97:0/3200163585 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5284101090 0x7f5284196e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:08.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.669+0000 7f52837fe700 1 -- 172.21.15.97:0/3200163585 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f52841070d0 msgr2=0x7f528419b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:08.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.669+0000 7f52837fe700 1 --2- 172.21.15.97:0/3200163585 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f52841070d0 0x7f528419b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:08.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.669+0000 7f52837fe700 1 -- 172.21.15.97:0/3200163585 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5284101090 msgr2=0x7f5284196e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:08.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.669+0000 7f52837fe700 1 --2- 172.21.15.97:0/3200163585 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5284101090 0x7f5284196e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:08.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.669+0000 7f52837fe700 1 -- 172.21.15.97:0/3200163585 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f528419be50 con 0x7f5284101a30 2024-03-21T13:27:08.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.669+0000 7f5288e9c700 1 --2- 172.21.15.97:0/3200163585 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f52841070d0 0x7f528419b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T13:27:08.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.669+0000 7f52837fe700 1 --2- 172.21.15.97:0/3200163585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5284101a30 0x7f5284197380 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f527400ec40 tx=0x7f527400ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:27:08.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.669+0000 7f52817fa700 1 -- 172.21.15.97:0/3200163585 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527400ba80 con 0x7f5284101a30 2024-03-21T13:27:08.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.669+0000 7f528a8ff700 1 -- 172.21.15.97:0/3200163585 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f528419c140 con 0x7f5284101a30 2024-03-21T13:27:08.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.669+0000 7f52817fa700 1 -- 172.21.15.97:0/3200163585 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f52740044d0 con 0x7f5284101a30 2024-03-21T13:27:08.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.669+0000 7f52817fa700 1 -- 172.21.15.97:0/3200163585 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5274003cf0 con 0x7f5284101a30 2024-03-21T13:27:08.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.669+0000 7f528a8ff700 1 -- 172.21.15.97:0/3200163585 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5284069e40 con 0x7f5284101a30 2024-03-21T13:27:08.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.670+0000 7f528a8ff700 1 -- 172.21.15.97:0/3200163585 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5284106670 con 0x7f5284101a30 2024-03-21T13:27:08.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.670+0000 7f52817fa700 1 -- 172.21.15.97:0/3200163585 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 22) v1 ==== 92152+0+0 (secure 0 0 0) 0x7f5274017410 con 0x7f5284101a30 2024-03-21T13:27:08.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.671+0000 7f52817fa700 1 --2- 172.21.15.97:0/3200163585 >> v2:172.21.15.97:6800/12494470 conn(0x7f526c06e300 0x7f526c0707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:08.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.671+0000 7f5283fff700 1 --2- 172.21.15.97:0/3200163585 >> v2:172.21.15.97:6800/12494470 conn(0x7f526c06e300 0x7f526c0707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:08.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.671+0000 7f52817fa700 1 -- 172.21.15.97:0/3200163585 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f527408f980 con 0x7f5284101a30 2024-03-21T13:27:08.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.671+0000 7f52817fa700 1 --2- 172.21.15.97:0/3200163585 >> v2:172.21.15.144:6808/4170414757 conn(0x7f526c077ba0 0x7f526c07a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:08.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.671+0000 7f52817fa700 1 -- 172.21.15.97:0/3200163585 --> v2:172.21.15.144:6808/4170414757 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5274009500 con 0x7f526c077ba0 2024-03-21T13:27:08.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.671+0000 7f52817fa700 1 -- 172.21.15.97:0/3200163585 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52740942c0 con 0x7f5284101a30 2024-03-21T13:27:08.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.671+0000 7f5283fff700 1 --2- 172.21.15.97:0/3200163585 >> v2:172.21.15.97:6800/12494470 conn(0x7f526c06e300 0x7f526c0707c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f527800abd0 tx=0x7f527800a380 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:27:08.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.671+0000 7f5288e9c700 1 --2- 172.21.15.97:0/3200163585 >> v2:172.21.15.144:6808/4170414757 conn(0x7f526c077ba0 0x7f526c07a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:08.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.672+0000 7f5288e9c700 1 --2- 172.21.15.97:0/3200163585 >> v2:172.21.15.144:6808/4170414757 conn(0x7f526c077ba0 0x7f526c07a040 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:27:08.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.673+0000 7f52817fa700 1 -- 172.21.15.97:0/3200163585 <== osd.6 v2:172.21.15.144:6808/4170414757 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f5274009500 con 0x7f526c077ba0 2024-03-21T13:27:08.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.688+0000 7f528a8ff700 1 -- 172.21.15.97:0/3200163585 --> v2:172.21.15.144:6808/4170414757 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5284102780 con 0x7f526c077ba0 2024-03-21T13:27:08.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.689+0000 7f52817fa700 1 -- 172.21.15.97:0/3200163585 <== osd.6 v2:172.21.15.144:6808/4170414757 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f5284102780 con 0x7f526c077ba0 2024-03-21T13:27:08.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.689+0000 7f528a8ff700 1 -- 172.21.15.97:0/3200163585 >> v2:172.21.15.144:6808/4170414757 conn(0x7f526c077ba0 msgr2=0x7f526c07a040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:08.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.689+0000 7f528a8ff700 1 --2- 172.21.15.97:0/3200163585 >> v2:172.21.15.144:6808/4170414757 conn(0x7f526c077ba0 0x7f526c07a040 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:08.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.689+0000 7f528a8ff700 1 -- 172.21.15.97:0/3200163585 >> v2:172.21.15.97:6800/12494470 conn(0x7f526c06e300 msgr2=0x7f526c0707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:08.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.689+0000 7f528a8ff700 1 --2- 172.21.15.97:0/3200163585 >> v2:172.21.15.97:6800/12494470 conn(0x7f526c06e300 0x7f526c0707c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f527800abd0 tx=0x7f527800a380 comp rx=0 tx=0).stop 2024-03-21T13:27:08.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.689+0000 7f528a8ff700 1 -- 172.21.15.97:0/3200163585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5284101a30 msgr2=0x7f5284197380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:08.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.689+0000 7f528a8ff700 1 --2- 172.21.15.97:0/3200163585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5284101a30 0x7f5284197380 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f527400ec40 tx=0x7f527400ef50 comp rx=0 tx=0).stop 2024-03-21T13:27:08.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.689+0000 7f5288e9c700 1 -- 172.21.15.97:0/3200163585 reap_dead start 2024-03-21T13:27:08.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.690+0000 7f528a8ff700 1 -- 172.21.15.97:0/3200163585 shutdown_connections 2024-03-21T13:27:08.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.690+0000 7f528a8ff700 1 -- 172.21.15.97:0/3200163585 >> 172.21.15.97:0/3200163585 conn(0x7f5284077fc0 msgr2=0x7f5284104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:27:08.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.690+0000 7f528a8ff700 1 -- 172.21.15.97:0/3200163585 shutdown_connections 2024-03-21T13:27:08.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:08.690+0000 7f528a8ff700 1 -- 172.21.15.97:0/3200163585 wait complete. 2024-03-21T13:27:09.313 INFO:teuthology.orchestra.run.smithi097.stdout:197568495637 2024-03-21T13:27:09.313 DEBUG:teuthology.orchestra.run.smithi097:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-03-21T13:27:09.886 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T13:27:10.089 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:09 smithi097 ceph-mon[39416]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-21T13:27:10.090 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:09 smithi097 ceph-mon[30040]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-21T13:27:10.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:09 smithi144 ceph-mon[33633]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-21T13:27:10.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.496+0000 7f86d22cf700 1 -- 172.21.15.97:0/2506162101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86cc1023d0 msgr2=0x7f86cc1027b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:10.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.496+0000 7f86d22cf700 1 --2- 172.21.15.97:0/2506162101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86cc1023d0 0x7f86cc1027b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f86bc0099c0 tx=0x7f86bc009cd0 comp rx=0 tx=0).stop 2024-03-21T13:27:10.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.496+0000 7f86d22cf700 1 -- 172.21.15.97:0/2506162101 shutdown_connections 2024-03-21T13:27:10.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.496+0000 7f86d22cf700 1 --2- 172.21.15.97:0/2506162101 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f86cc1074b0 0x7f86cc10f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:10.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.496+0000 7f86d22cf700 1 --2- 172.21.15.97:0/2506162101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86cc1023d0 0x7f86cc1027b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:10.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.496+0000 7f86d22cf700 1 --2- 172.21.15.97:0/2506162101 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86cc102d80 0x7f86cc106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:10.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.496+0000 7f86d22cf700 1 -- 172.21.15.97:0/2506162101 >> 172.21.15.97:0/2506162101 conn(0x7f86cc0fdc40 msgr2=0x7f86cc100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:27:10.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.496+0000 7f86d22cf700 1 -- 172.21.15.97:0/2506162101 shutdown_connections 2024-03-21T13:27:10.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.496+0000 7f86d22cf700 1 -- 172.21.15.97:0/2506162101 wait complete. 2024-03-21T13:27:10.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.497+0000 7f86d22cf700 1 Processor -- start 2024-03-21T13:27:10.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.497+0000 7f86d22cf700 1 -- start start 2024-03-21T13:27:10.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.497+0000 7f86d22cf700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f86cc1023d0 0x7f86cc1980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:10.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.497+0000 7f86d22cf700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86cc102d80 0x7f86cc1985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:10.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.497+0000 7f86d22cf700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86cc1074b0 0x7f86cc19c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:10.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.497+0000 7f86d22cf700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86cc111b40 con 0x7f86cc102d80 2024-03-21T13:27:10.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.497+0000 7f86d22cf700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86cc1119c0 con 0x7f86cc1074b0 2024-03-21T13:27:10.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.497+0000 7f86d22cf700 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86cc111cc0 con 0x7f86cc1023d0 2024-03-21T13:27:10.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.497+0000 7f86cb7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86cc102d80 0x7f86cc1985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:10.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.497+0000 7f86cbfff700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f86cc1023d0 0x7f86cc1980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:10.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.497+0000 7f86cb7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86cc102d80 0x7f86cc1985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42890/0 (socket says 172.21.15.97:42890) 2024-03-21T13:27:10.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.497+0000 7f86cb7fe700 1 -- 172.21.15.97:0/1046743361 learned_addr learned my addr 172.21.15.97:0/1046743361 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:27:10.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.497+0000 7f86d086c700 1 --2- 172.21.15.97:0/1046743361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86cc1074b0 0x7f86cc19c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:10.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.497+0000 7f86cb7fe700 1 -- 172.21.15.97:0/1046743361 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f86cc1023d0 msgr2=0x7f86cc1980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:10.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.497+0000 7f86cb7fe700 1 --2- 172.21.15.97:0/1046743361 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f86cc1023d0 0x7f86cc1980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:10.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.497+0000 7f86cb7fe700 1 -- 172.21.15.97:0/1046743361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86cc1074b0 msgr2=0x7f86cc19c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:10.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.497+0000 7f86cb7fe700 1 --2- 172.21.15.97:0/1046743361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86cc1074b0 0x7f86cc19c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:10.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.497+0000 7f86cb7fe700 1 -- 172.21.15.97:0/1046743361 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86cc19d0b0 con 0x7f86cc102d80 2024-03-21T13:27:10.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.497+0000 7f86d086c700 1 --2- 172.21.15.97:0/1046743361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86cc1074b0 0x7f86cc19c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T13:27:10.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.497+0000 7f86cbfff700 1 --2- 172.21.15.97:0/1046743361 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f86cc1023d0 0x7f86cc1980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T13:27:10.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.498+0000 7f86cb7fe700 1 --2- 172.21.15.97:0/1046743361 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86cc102d80 0x7f86cc1985e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f86c000d980 tx=0x7f86c000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:27:10.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.498+0000 7f86c97fa700 1 -- 172.21.15.97:0/1046743361 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c0017070 con 0x7f86cc102d80 2024-03-21T13:27:10.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.498+0000 7f86c97fa700 1 -- 172.21.15.97:0/1046743361 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f86c0009de0 con 0x7f86cc102d80 2024-03-21T13:27:10.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.498+0000 7f86d22cf700 1 -- 172.21.15.97:0/1046743361 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86cc19d3a0 con 0x7f86cc102d80 2024-03-21T13:27:10.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.498+0000 7f86c97fa700 1 -- 172.21.15.97:0/1046743361 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c000f3c0 con 0x7f86cc102d80 2024-03-21T13:27:10.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.498+0000 7f86d22cf700 1 -- 172.21.15.97:0/1046743361 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86cc1a4c20 con 0x7f86cc102d80 2024-03-21T13:27:10.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.499+0000 7f86d22cf700 1 -- 172.21.15.97:0/1046743361 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f86cc0fea00 con 0x7f86cc102d80 2024-03-21T13:27:10.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.499+0000 7f86c97fa700 1 -- 172.21.15.97:0/1046743361 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 22) v1 ==== 92152+0+0 (secure 0 0 0) 0x7f86c0009930 con 0x7f86cc102d80 2024-03-21T13:27:10.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.500+0000 7f86c97fa700 1 --2- 172.21.15.97:0/1046743361 >> v2:172.21.15.97:6800/12494470 conn(0x7f86b406e300 0x7f86b40707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:10.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.500+0000 7f86c97fa700 1 -- 172.21.15.97:0/1046743361 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f86c008dee0 con 0x7f86cc102d80 2024-03-21T13:27:10.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.500+0000 7f86cbfff700 1 --2- 172.21.15.97:0/1046743361 >> v2:172.21.15.97:6800/12494470 conn(0x7f86b406e300 0x7f86b40707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:10.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.500+0000 7f86c97fa700 1 --2- 172.21.15.97:0/1046743361 >> v2:172.21.15.97:6805/287967678 conn(0x7f86b4077ba0 0x7f86b407a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:10.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.500+0000 7f86c97fa700 1 -- 172.21.15.97:0/1046743361 --> v2:172.21.15.97:6805/287967678 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f86b407a6d0 con 0x7f86b4077ba0 2024-03-21T13:27:10.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.500+0000 7f86c97fa700 1 -- 172.21.15.97:0/1046743361 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86c00905e0 con 0x7f86cc102d80 2024-03-21T13:27:10.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.500+0000 7f86d086c700 1 --2- 172.21.15.97:0/1046743361 >> v2:172.21.15.97:6805/287967678 conn(0x7f86b4077ba0 0x7f86b407a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:10.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.500+0000 7f86cbfff700 1 --2- 172.21.15.97:0/1046743361 >> v2:172.21.15.97:6800/12494470 conn(0x7f86b406e300 0x7f86b40707c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f86bc000c00 tx=0x7f86bc005fb0 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:27:10.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.500+0000 7f86d086c700 1 --2- 172.21.15.97:0/1046743361 >> v2:172.21.15.97:6805/287967678 conn(0x7f86b4077ba0 0x7f86b407a040 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:27:10.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.501+0000 7f86c97fa700 1 -- 172.21.15.97:0/1046743361 <== osd.1 v2:172.21.15.97:6805/287967678 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f86b407a6d0 con 0x7f86b4077ba0 2024-03-21T13:27:10.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.520+0000 7f86d22cf700 1 -- 172.21.15.97:0/1046743361 --> v2:172.21.15.97:6805/287967678 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f86cc1038b0 con 0x7f86b4077ba0 2024-03-21T13:27:10.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.520+0000 7f86c97fa700 1 -- 172.21.15.97:0/1046743361 <== osd.1 v2:172.21.15.97:6805/287967678 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f86cc1038b0 con 0x7f86b4077ba0 2024-03-21T13:27:10.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.520+0000 7f86d22cf700 1 -- 172.21.15.97:0/1046743361 >> v2:172.21.15.97:6805/287967678 conn(0x7f86b4077ba0 msgr2=0x7f86b407a040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:10.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.520+0000 7f86d22cf700 1 --2- 172.21.15.97:0/1046743361 >> v2:172.21.15.97:6805/287967678 conn(0x7f86b4077ba0 0x7f86b407a040 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:10.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.521+0000 7f86d22cf700 1 -- 172.21.15.97:0/1046743361 >> v2:172.21.15.97:6800/12494470 conn(0x7f86b406e300 msgr2=0x7f86b40707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:10.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.521+0000 7f86d22cf700 1 --2- 172.21.15.97:0/1046743361 >> v2:172.21.15.97:6800/12494470 conn(0x7f86b406e300 0x7f86b40707c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f86bc000c00 tx=0x7f86bc005fb0 comp rx=0 tx=0).stop 2024-03-21T13:27:10.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.521+0000 7f86d22cf700 1 -- 172.21.15.97:0/1046743361 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86cc102d80 msgr2=0x7f86cc1985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:10.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.521+0000 7f86d22cf700 1 --2- 172.21.15.97:0/1046743361 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86cc102d80 0x7f86cc1985e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f86c000d980 tx=0x7f86c000dc90 comp rx=0 tx=0).stop 2024-03-21T13:27:10.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.521+0000 7f86d086c700 1 -- 172.21.15.97:0/1046743361 reap_dead start 2024-03-21T13:27:10.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.521+0000 7f86d22cf700 1 -- 172.21.15.97:0/1046743361 shutdown_connections 2024-03-21T13:27:10.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.521+0000 7f86d22cf700 1 -- 172.21.15.97:0/1046743361 >> 172.21.15.97:0/1046743361 conn(0x7f86cc0fdc40 msgr2=0x7f86cc0fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:27:10.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.521+0000 7f86d22cf700 1 -- 172.21.15.97:0/1046743361 shutdown_connections 2024-03-21T13:27:10.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:10.521+0000 7f86d22cf700 1 -- 172.21.15.97:0/1046743361 wait complete. 2024-03-21T13:27:12.111 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:11 smithi097 ceph-mon[39416]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-21T13:27:12.111 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:27:11 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:27:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:27:12.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:11 smithi097 ceph-mon[30040]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-21T13:27:12.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:11 smithi144 ceph-mon[33633]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-21T13:27:13.372 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:27:13 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:27:14.030 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:13 smithi097 ceph-mon[39416]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:14.031 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:13 smithi097 ceph-mon[30040]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:14.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:13 smithi144 ceph-mon[33633]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:15.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:14 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:27:15.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:14 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:27:15.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:14 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:27:15.222 INFO:teuthology.orchestra.run.smithi097.stdout:55834574895 2024-03-21T13:27:15.223 DEBUG:teuthology.orchestra.run.smithi097:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-21T13:27:15.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.435+0000 7fb2e9049700 1 -- 172.21.15.97:0/1070446815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2e4106370 msgr2=0x7fb2e410df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:15.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.435+0000 7fb2e9049700 1 --2- 172.21.15.97:0/1070446815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2e4106370 0x7fb2e410df40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc009fc0 tx=0x7fb2dc00b330 comp rx=0 tx=0).stop 2024-03-21T13:27:15.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.435+0000 7fb2e9049700 1 -- 172.21.15.97:0/1070446815 shutdown_connections 2024-03-21T13:27:15.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.435+0000 7fb2e9049700 1 --2- 172.21.15.97:0/1070446815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2e4106370 0x7fb2e410df40 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:15.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.435+0000 7fb2e9049700 1 --2- 172.21.15.97:0/1070446815 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb2e4101320 0x7fb2e4101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:15.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.435+0000 7fb2e9049700 1 --2- 172.21.15.97:0/1070446815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2e4101cd0 0x7fb2e4105c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:15.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.435+0000 7fb2e9049700 1 -- 172.21.15.97:0/1070446815 >> 172.21.15.97:0/1070446815 conn(0x7fb2e40770e0 msgr2=0x7fb2e40774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:27:15.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.435+0000 7fb2e9049700 1 -- 172.21.15.97:0/1070446815 shutdown_connections 2024-03-21T13:27:15.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.435+0000 7fb2e9049700 1 -- 172.21.15.97:0/1070446815 wait complete. 2024-03-21T13:27:15.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.436+0000 7fb2e9049700 1 Processor -- start 2024-03-21T13:27:15.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.436+0000 7fb2e9049700 1 -- start start 2024-03-21T13:27:15.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.437+0000 7fb2e9049700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2e4101320 0x7fb2e4198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:15.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.437+0000 7fb2e9049700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2e4101cd0 0x7fb2e4199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:15.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.437+0000 7fb2e9049700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb2e4106370 0x7fb2e41a3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:15.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.437+0000 7fb2e9049700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e4110a60 con 0x7fb2e4101320 2024-03-21T13:27:15.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.437+0000 7fb2e9049700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e41108e0 con 0x7fb2e4101cd0 2024-03-21T13:27:15.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.437+0000 7fb2e9049700 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e4110be0 con 0x7fb2e4106370 2024-03-21T13:27:15.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.437+0000 7fb2e2d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2e4101320 0x7fb2e4198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:15.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.437+0000 7fb2e359e700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb2e4106370 0x7fb2e41a3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:15.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.437+0000 7fb2e359e700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb2e4106370 0x7fb2e41a3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.97:41962/0 (socket says 172.21.15.97:41962) 2024-03-21T13:27:15.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.437+0000 7fb2e359e700 1 -- 172.21.15.97:0/761220572 learned_addr learned my addr 172.21.15.97:0/761220572 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:27:15.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.437+0000 7fb2e259c700 1 --2- 172.21.15.97:0/761220572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2e4101cd0 0x7fb2e4199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:15.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.437+0000 7fb2e2d9d700 1 -- 172.21.15.97:0/761220572 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb2e4106370 msgr2=0x7fb2e41a3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:15.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.437+0000 7fb2e2d9d700 1 --2- 172.21.15.97:0/761220572 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb2e4106370 0x7fb2e41a3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:15.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.437+0000 7fb2e2d9d700 1 -- 172.21.15.97:0/761220572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2e4101cd0 msgr2=0x7fb2e4199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:15.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.437+0000 7fb2e2d9d700 1 --2- 172.21.15.97:0/761220572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2e4101cd0 0x7fb2e4199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:15.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.437+0000 7fb2e2d9d700 1 -- 172.21.15.97:0/761220572 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2e41a4390 con 0x7fb2e4101320 2024-03-21T13:27:15.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.437+0000 7fb2e359e700 1 --2- 172.21.15.97:0/761220572 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb2e4106370 0x7fb2e41a3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T13:27:15.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.437+0000 7fb2e2d9d700 1 --2- 172.21.15.97:0/761220572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2e4101320 0x7fb2e4198dd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb2d400eb90 tx=0x7fb2d400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:27:15.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.438+0000 7fb2cbfff700 1 -- 172.21.15.97:0/761220572 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d400cc90 con 0x7fb2e4101320 2024-03-21T13:27:15.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.438+0000 7fb2cbfff700 1 -- 172.21.15.97:0/761220572 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fb2d400ce30 con 0x7fb2e4101320 2024-03-21T13:27:15.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.438+0000 7fb2e9049700 1 -- 172.21.15.97:0/761220572 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2e4196bc0 con 0x7fb2e4101320 2024-03-21T13:27:15.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.438+0000 7fb2cbfff700 1 -- 172.21.15.97:0/761220572 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d4010430 con 0x7fb2e4101320 2024-03-21T13:27:15.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.438+0000 7fb2e9049700 1 -- 172.21.15.97:0/761220572 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2e4197100 con 0x7fb2e4101320 2024-03-21T13:27:15.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.439+0000 7fb2e9049700 1 -- 172.21.15.97:0/761220572 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2e4068e50 con 0x7fb2e4101320 2024-03-21T13:27:15.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.440+0000 7fb2cbfff700 1 -- 172.21.15.97:0/761220572 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 22) v1 ==== 92152+0+0 (secure 0 0 0) 0x7fb2d4003ec0 con 0x7fb2e4101320 2024-03-21T13:27:15.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.440+0000 7fb2cbfff700 1 --2- 172.21.15.97:0/761220572 >> v2:172.21.15.97:6800/12494470 conn(0x7fb2cc06e300 0x7fb2cc0707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:15.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.440+0000 7fb2cbfff700 1 -- 172.21.15.97:0/761220572 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6259+0+0 (secure 0 0 0) 0x7fb2d408e620 con 0x7fb2e4101320 2024-03-21T13:27:15.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.440+0000 7fb2e259c700 1 --2- 172.21.15.97:0/761220572 >> v2:172.21.15.97:6800/12494470 conn(0x7fb2cc06e300 0x7fb2cc0707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:15.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.441+0000 7fb2e259c700 1 --2- 172.21.15.97:0/761220572 >> v2:172.21.15.97:6800/12494470 conn(0x7fb2cc06e300 0x7fb2cc0707c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4100cd0 tx=0x7fb2d800b410 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:27:15.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.446+0000 7fb2cbfff700 1 -- 172.21.15.97:0/761220572 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb2d405bdf0 con 0x7fb2e4101320 2024-03-21T13:27:15.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.654+0000 7fb2e9049700 1 -- 172.21.15.97:0/761220572 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fb2e4197360 con 0x7fb2e4101320 2024-03-21T13:27:15.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.654+0000 7fb2cbfff700 1 -- 172.21.15.97:0/761220572 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb2d405f840 con 0x7fb2e4101320 2024-03-21T13:27:15.655 INFO:teuthology.orchestra.run.smithi097.stdout:231928234001 2024-03-21T13:27:15.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.657+0000 7fb2e9049700 1 -- 172.21.15.97:0/761220572 >> v2:172.21.15.97:6800/12494470 conn(0x7fb2cc06e300 msgr2=0x7fb2cc0707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:15.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.657+0000 7fb2e9049700 1 --2- 172.21.15.97:0/761220572 >> v2:172.21.15.97:6800/12494470 conn(0x7fb2cc06e300 0x7fb2cc0707c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4100cd0 tx=0x7fb2d800b410 comp rx=0 tx=0).stop 2024-03-21T13:27:15.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.657+0000 7fb2e9049700 1 -- 172.21.15.97:0/761220572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2e4101320 msgr2=0x7fb2e4198dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:15.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.657+0000 7fb2e9049700 1 --2- 172.21.15.97:0/761220572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2e4101320 0x7fb2e4198dd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb2d400eb90 tx=0x7fb2d400eea0 comp rx=0 tx=0).stop 2024-03-21T13:27:15.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.657+0000 7fb2e9049700 1 -- 172.21.15.97:0/761220572 shutdown_connections 2024-03-21T13:27:15.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.657+0000 7fb2e9049700 1 --2- 172.21.15.97:0/761220572 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb2e4106370 0x7fb2e41a3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:15.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.657+0000 7fb2e9049700 1 --2- 172.21.15.97:0/761220572 >> v2:172.21.15.97:6800/12494470 conn(0x7fb2cc06e300 0x7fb2cc0707c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:15.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.657+0000 7fb2e9049700 1 --2- 172.21.15.97:0/761220572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2e4101320 0x7fb2e4198dd0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:15.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.657+0000 7fb2e9049700 1 --2- 172.21.15.97:0/761220572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2e4101cd0 0x7fb2e4199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:15.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.657+0000 7fb2e9049700 1 -- 172.21.15.97:0/761220572 >> 172.21.15.97:0/761220572 conn(0x7fb2e40770e0 msgr2=0x7fb2e40fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:27:15.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.657+0000 7fb2e9049700 1 -- 172.21.15.97:0/761220572 shutdown_connections 2024-03-21T13:27:15.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:15.657+0000 7fb2e9049700 1 -- 172.21.15.97:0/761220572 wait complete. 2024-03-21T13:27:15.961 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:15 smithi097 ceph-mon[39416]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:27:15.961 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:15 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/761220572' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-21T13:27:15.961 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:15 smithi097 ceph-mon[30040]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:27:15.961 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:15 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/761220572' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-21T13:27:16.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.063+0000 7fd591f04700 1 -- 172.21.15.97:0/2796871299 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd58c1074b0 msgr2=0x7fd58c10f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:16.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.063+0000 7fd591f04700 1 --2- 172.21.15.97:0/2796871299 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd58c1074b0 0x7fd58c10f080 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd58400b0d0 tx=0x7fd58400b3e0 comp rx=0 tx=0).stop 2024-03-21T13:27:16.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.063+0000 7fd591f04700 1 -- 172.21.15.97:0/2796871299 shutdown_connections 2024-03-21T13:27:16.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.063+0000 7fd591f04700 1 --2- 172.21.15.97:0/2796871299 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd58c1074b0 0x7fd58c10f080 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:16.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.063+0000 7fd591f04700 1 --2- 172.21.15.97:0/2796871299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd58c1023d0 0x7fd58c1027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:16.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.063+0000 7fd591f04700 1 --2- 172.21.15.97:0/2796871299 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd58c102d80 0x7fd58c106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:16.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.063+0000 7fd591f04700 1 -- 172.21.15.97:0/2796871299 >> 172.21.15.97:0/2796871299 conn(0x7fd58c0fdc40 msgr2=0x7fd58c100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:27:16.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.063+0000 7fd591f04700 1 -- 172.21.15.97:0/2796871299 shutdown_connections 2024-03-21T13:27:16.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.063+0000 7fd591f04700 1 -- 172.21.15.97:0/2796871299 wait complete. 2024-03-21T13:27:16.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.064+0000 7fd591f04700 1 Processor -- start 2024-03-21T13:27:16.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.064+0000 7fd591f04700 1 -- start start 2024-03-21T13:27:16.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.064+0000 7fd591f04700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd58c1023d0 0x7fd58c197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:16.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.064+0000 7fd591f04700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd58c102d80 0x7fd58c198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:16.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.064+0000 7fd591f04700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd58c1074b0 0x7fd58c19c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:16.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.064+0000 7fd591f04700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58c111bb0 con 0x7fd58c1023d0 2024-03-21T13:27:16.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.064+0000 7fd591f04700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58c111a30 con 0x7fd58c102d80 2024-03-21T13:27:16.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.064+0000 7fd591f04700 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd58c111d30 con 0x7fd58c1074b0 2024-03-21T13:27:16.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.064+0000 7fd58b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd58c1023d0 0x7fd58c197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:16.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.064+0000 7fd58b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd58c1023d0 0x7fd58c197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42934/0 (socket says 172.21.15.97:42934) 2024-03-21T13:27:16.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.064+0000 7fd58b7fe700 1 -- 172.21.15.97:0/3618201321 learned_addr learned my addr 172.21.15.97:0/3618201321 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:27:16.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.064+0000 7fd58bfff700 1 --2- 172.21.15.97:0/3618201321 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd58c1074b0 0x7fd58c19c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:16.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.064+0000 7fd582ffd700 1 --2- 172.21.15.97:0/3618201321 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd58c102d80 0x7fd58c198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:16.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.064+0000 7fd58b7fe700 1 -- 172.21.15.97:0/3618201321 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd58c1074b0 msgr2=0x7fd58c19c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:16.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.064+0000 7fd58b7fe700 1 --2- 172.21.15.97:0/3618201321 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd58c1074b0 0x7fd58c19c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:16.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.064+0000 7fd58b7fe700 1 -- 172.21.15.97:0/3618201321 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd58c102d80 msgr2=0x7fd58c198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:16.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.064+0000 7fd58b7fe700 1 --2- 172.21.15.97:0/3618201321 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd58c102d80 0x7fd58c198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:16.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.064+0000 7fd58b7fe700 1 -- 172.21.15.97:0/3618201321 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd58c19cfe0 con 0x7fd58c1023d0 2024-03-21T13:27:16.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.064+0000 7fd582ffd700 1 --2- 172.21.15.97:0/3618201321 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd58c102d80 0x7fd58c198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T13:27:16.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.064+0000 7fd58bfff700 1 --2- 172.21.15.97:0/3618201321 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd58c1074b0 0x7fd58c19c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T13:27:16.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.065+0000 7fd58b7fe700 1 --2- 172.21.15.97:0/3618201321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd58c1023d0 0x7fd58c197fd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd57c00d980 tx=0x7fd57c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:27:16.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.065+0000 7fd5897fa700 1 -- 172.21.15.97:0/3618201321 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57c017070 con 0x7fd58c1023d0 2024-03-21T13:27:16.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.065+0000 7fd591f04700 1 -- 172.21.15.97:0/3618201321 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd58c19d2d0 con 0x7fd58c1023d0 2024-03-21T13:27:16.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.065+0000 7fd5897fa700 1 -- 172.21.15.97:0/3618201321 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fd57c005020 con 0x7fd58c1023d0 2024-03-21T13:27:16.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.065+0000 7fd5897fa700 1 -- 172.21.15.97:0/3618201321 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57c009b20 con 0x7fd58c1023d0 2024-03-21T13:27:16.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.065+0000 7fd591f04700 1 -- 172.21.15.97:0/3618201321 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd58c1a4b50 con 0x7fd58c1023d0 2024-03-21T13:27:16.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.066+0000 7fd591f04700 1 -- 172.21.15.97:0/3618201321 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd58c068e50 con 0x7fd58c1023d0 2024-03-21T13:27:16.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.066+0000 7fd5897fa700 1 -- 172.21.15.97:0/3618201321 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 22) v1 ==== 92152+0+0 (secure 0 0 0) 0x7fd57c009cc0 con 0x7fd58c1023d0 2024-03-21T13:27:16.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.066+0000 7fd5897fa700 1 --2- 172.21.15.97:0/3618201321 >> v2:172.21.15.97:6800/12494470 conn(0x7fd57806e300 0x7fd5780707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:16.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.067+0000 7fd582ffd700 1 --2- 172.21.15.97:0/3618201321 >> v2:172.21.15.97:6800/12494470 conn(0x7fd57806e300 0x7fd5780707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:16.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.067+0000 7fd5897fa700 1 -- 172.21.15.97:0/3618201321 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6259+0+0 (secure 0 0 0) 0x7fd57c08e340 con 0x7fd58c1023d0 2024-03-21T13:27:16.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.067+0000 7fd582ffd700 1 --2- 172.21.15.97:0/3618201321 >> v2:172.21.15.97:6800/12494470 conn(0x7fd57806e300 0x7fd5780707c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd58c108810 tx=0x7fd574009260 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:27:16.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.071+0000 7fd5897fa700 1 -- 172.21.15.97:0/3618201321 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd57c020020 con 0x7fd58c1023d0 2024-03-21T13:27:16.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:15 smithi144 ceph-mon[33633]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:27:16.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:15 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/761220572' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-21T13:27:16.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.276+0000 7fd591f04700 1 -- 172.21.15.97:0/3618201321 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd58c1a4de0 con 0x7fd58c1023d0 2024-03-21T13:27:16.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.276+0000 7fd5897fa700 1 -- 172.21.15.97:0/3618201321 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd57c05ba90 con 0x7fd58c1023d0 2024-03-21T13:27:16.277 INFO:teuthology.orchestra.run.smithi097.stdout:34359738422 2024-03-21T13:27:16.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.278+0000 7fd591f04700 1 -- 172.21.15.97:0/3618201321 >> v2:172.21.15.97:6800/12494470 conn(0x7fd57806e300 msgr2=0x7fd5780707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:16.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.278+0000 7fd591f04700 1 --2- 172.21.15.97:0/3618201321 >> v2:172.21.15.97:6800/12494470 conn(0x7fd57806e300 0x7fd5780707c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd58c108810 tx=0x7fd574009260 comp rx=0 tx=0).stop 2024-03-21T13:27:16.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.278+0000 7fd591f04700 1 -- 172.21.15.97:0/3618201321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd58c1023d0 msgr2=0x7fd58c197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:16.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.278+0000 7fd591f04700 1 --2- 172.21.15.97:0/3618201321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd58c1023d0 0x7fd58c197fd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd57c00d980 tx=0x7fd57c00dc90 comp rx=0 tx=0).stop 2024-03-21T13:27:16.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.278+0000 7fd591f04700 1 -- 172.21.15.97:0/3618201321 shutdown_connections 2024-03-21T13:27:16.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.278+0000 7fd591f04700 1 --2- 172.21.15.97:0/3618201321 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd58c1074b0 0x7fd58c19c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:16.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.278+0000 7fd591f04700 1 --2- 172.21.15.97:0/3618201321 >> v2:172.21.15.97:6800/12494470 conn(0x7fd57806e300 0x7fd5780707c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:16.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.278+0000 7fd591f04700 1 --2- 172.21.15.97:0/3618201321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd58c1023d0 0x7fd58c197fd0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:16.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.278+0000 7fd591f04700 1 --2- 172.21.15.97:0/3618201321 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd58c102d80 0x7fd58c198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:16.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.278+0000 7fd591f04700 1 -- 172.21.15.97:0/3618201321 >> 172.21.15.97:0/3618201321 conn(0x7fd58c0fdc40 msgr2=0x7fd58c0fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:27:16.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.278+0000 7fd591f04700 1 -- 172.21.15.97:0/3618201321 shutdown_connections 2024-03-21T13:27:16.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:16.278+0000 7fd591f04700 1 -- 172.21.15.97:0/3618201321 wait complete. 2024-03-21T13:27:16.912 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:16 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3618201321' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-21T13:27:16.912 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:16 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3618201321' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-21T13:27:17.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.014+0000 7fa0a10ba700 1 -- 172.21.15.97:0/3431614537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa09c1070d0 msgr2=0x7fa09c1074b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:17.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.014+0000 7fa0a10ba700 1 --2- 172.21.15.97:0/3431614537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa09c1070d0 0x7fa09c1074b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa0900098f0 tx=0x7fa090009c00 comp rx=0 tx=0).stop 2024-03-21T13:27:17.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.014+0000 7fa0a10ba700 1 -- 172.21.15.97:0/3431614537 shutdown_connections 2024-03-21T13:27:17.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.014+0000 7fa0a10ba700 1 --2- 172.21.15.97:0/3431614537 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa09c101a30 0x7fa09c10df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:17.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.014+0000 7fa0a10ba700 1 --2- 172.21.15.97:0/3431614537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa09c1070d0 0x7fa09c1074b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:17.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.014+0000 7fa0a10ba700 1 --2- 172.21.15.97:0/3431614537 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa09c101090 0x7fa09c1014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:17.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.014+0000 7fa0a10ba700 1 -- 172.21.15.97:0/3431614537 >> 172.21.15.97:0/3431614537 conn(0x7fa09c077fc0 msgr2=0x7fa09c0783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:27:17.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.014+0000 7fa0a10ba700 1 -- 172.21.15.97:0/3431614537 shutdown_connections 2024-03-21T13:27:17.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.014+0000 7fa0a10ba700 1 -- 172.21.15.97:0/3431614537 wait complete. 2024-03-21T13:27:17.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.015+0000 7fa0a10ba700 1 Processor -- start 2024-03-21T13:27:17.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.015+0000 7fa0a10ba700 1 -- start start 2024-03-21T13:27:17.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.015+0000 7fa0a10ba700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa09c101090 0x7fa09c198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:17.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.015+0000 7fa0a10ba700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa09c101a30 0x7fa09c199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:17.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.015+0000 7fa0a10ba700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa09c1070d0 0x7fa09c1a3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:17.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.015+0000 7fa0a10ba700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09c1109e0 con 0x7fa09c1070d0 2024-03-21T13:27:17.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.015+0000 7fa0a10ba700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09c110860 con 0x7fa09c101090 2024-03-21T13:27:17.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.015+0000 7fa0a10ba700 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa09c110b60 con 0x7fa09c101a30 2024-03-21T13:27:17.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.015+0000 7fa09a59c700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa09c101a30 0x7fa09c199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:17.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.015+0000 7fa09b59e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa09c1070d0 0x7fa09c1a3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:17.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.015+0000 7fa09a59c700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa09c101a30 0x7fa09c199450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.97:42002/0 (socket says 172.21.15.97:42002) 2024-03-21T13:27:17.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.015+0000 7fa09a59c700 1 -- 172.21.15.97:0/1314751360 learned_addr learned my addr 172.21.15.97:0/1314751360 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:27:17.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.015+0000 7fa09ad9d700 1 --2- 172.21.15.97:0/1314751360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa09c101090 0x7fa09c198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:17.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.015+0000 7fa09b59e700 1 -- 172.21.15.97:0/1314751360 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa09c101a30 msgr2=0x7fa09c199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:17.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.015+0000 7fa09b59e700 1 --2- 172.21.15.97:0/1314751360 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa09c101a30 0x7fa09c199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:17.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.015+0000 7fa09b59e700 1 -- 172.21.15.97:0/1314751360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa09c101090 msgr2=0x7fa09c198f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:17.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.015+0000 7fa09b59e700 1 --2- 172.21.15.97:0/1314751360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa09c101090 0x7fa09c198f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:17.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.015+0000 7fa09b59e700 1 -- 172.21.15.97:0/1314751360 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa09c1a44d0 con 0x7fa09c1070d0 2024-03-21T13:27:17.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.016+0000 7fa09a59c700 1 --2- 172.21.15.97:0/1314751360 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa09c101a30 0x7fa09c199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T13:27:17.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.016+0000 7fa09b59e700 1 --2- 172.21.15.97:0/1314751360 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa09c1070d0 0x7fa09c1a3f90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa09400ab40 tx=0x7fa09400ae50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:27:17.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.016+0000 7fa083fff700 1 -- 172.21.15.97:0/1314751360 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa094017020 con 0x7fa09c1070d0 2024-03-21T13:27:17.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.016+0000 7fa0a10ba700 1 -- 172.21.15.97:0/1314751360 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa09c1a4700 con 0x7fa09c1070d0 2024-03-21T13:27:17.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.016+0000 7fa083fff700 1 -- 172.21.15.97:0/1314751360 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fa094004bb0 con 0x7fa09c1070d0 2024-03-21T13:27:17.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.016+0000 7fa083fff700 1 -- 172.21.15.97:0/1314751360 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa094005280 con 0x7fa09c1070d0 2024-03-21T13:27:17.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.016+0000 7fa0a10ba700 1 -- 172.21.15.97:0/1314751360 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa09c1a4bc0 con 0x7fa09c1070d0 2024-03-21T13:27:17.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.017+0000 7fa0a10ba700 1 -- 172.21.15.97:0/1314751360 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa09c102780 con 0x7fa09c1070d0 2024-03-21T13:27:17.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.017+0000 7fa083fff700 1 -- 172.21.15.97:0/1314751360 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 22) v1 ==== 92152+0+0 (secure 0 0 0) 0x7fa09400c640 con 0x7fa09c1070d0 2024-03-21T13:27:17.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.018+0000 7fa083fff700 1 --2- 172.21.15.97:0/1314751360 >> v2:172.21.15.97:6800/12494470 conn(0x7fa08406e300 0x7fa0840707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:17.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.018+0000 7fa083fff700 1 -- 172.21.15.97:0/1314751360 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6259+0+0 (secure 0 0 0) 0x7fa09408da00 con 0x7fa09c1070d0 2024-03-21T13:27:17.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.018+0000 7fa09ad9d700 1 --2- 172.21.15.97:0/1314751360 >> v2:172.21.15.97:6800/12494470 conn(0x7fa08406e300 0x7fa0840707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:17.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.018+0000 7fa09ad9d700 1 --2- 172.21.15.97:0/1314751360 >> v2:172.21.15.97:6800/12494470 conn(0x7fa08406e300 0x7fa0840707c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa090009fd0 tx=0x7fa090019040 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:27:17.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.023+0000 7fa083fff700 1 -- 172.21.15.97:0/1314751360 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa094021020 con 0x7fa09c1070d0 2024-03-21T13:27:17.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:16 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3618201321' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-21T13:27:17.231 INFO:teuthology.orchestra.run.smithi097.stdout:77309411371 2024-03-21T13:27:17.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.230+0000 7fa0a10ba700 1 -- 172.21.15.97:0/1314751360 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa09c06a430 con 0x7fa09c1070d0 2024-03-21T13:27:17.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.230+0000 7fa083fff700 1 -- 172.21.15.97:0/1314751360 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa09405b150 con 0x7fa09c1070d0 2024-03-21T13:27:17.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.232+0000 7fa0a10ba700 1 -- 172.21.15.97:0/1314751360 >> v2:172.21.15.97:6800/12494470 conn(0x7fa08406e300 msgr2=0x7fa0840707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:17.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.232+0000 7fa0a10ba700 1 --2- 172.21.15.97:0/1314751360 >> v2:172.21.15.97:6800/12494470 conn(0x7fa08406e300 0x7fa0840707c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa090009fd0 tx=0x7fa090019040 comp rx=0 tx=0).stop 2024-03-21T13:27:17.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.232+0000 7fa0a10ba700 1 -- 172.21.15.97:0/1314751360 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa09c1070d0 msgr2=0x7fa09c1a3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:17.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.232+0000 7fa0a10ba700 1 --2- 172.21.15.97:0/1314751360 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa09c1070d0 0x7fa09c1a3f90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa09400ab40 tx=0x7fa09400ae50 comp rx=0 tx=0).stop 2024-03-21T13:27:17.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.232+0000 7fa0a10ba700 1 -- 172.21.15.97:0/1314751360 shutdown_connections 2024-03-21T13:27:17.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.232+0000 7fa0a10ba700 1 --2- 172.21.15.97:0/1314751360 >> v2:172.21.15.97:6800/12494470 conn(0x7fa08406e300 0x7fa0840707c0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:17.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.232+0000 7fa0a10ba700 1 --2- 172.21.15.97:0/1314751360 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa09c1070d0 0x7fa09c1a3f90 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:17.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.232+0000 7fa0a10ba700 1 --2- 172.21.15.97:0/1314751360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa09c101090 0x7fa09c198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:17.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.232+0000 7fa0a10ba700 1 --2- 172.21.15.97:0/1314751360 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa09c101a30 0x7fa09c199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:17.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.232+0000 7fa0a10ba700 1 -- 172.21.15.97:0/1314751360 >> 172.21.15.97:0/1314751360 conn(0x7fa09c077fc0 msgr2=0x7fa09c104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:27:17.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.233+0000 7fa0a10ba700 1 -- 172.21.15.97:0/1314751360 shutdown_connections 2024-03-21T13:27:17.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:17.233+0000 7fa0a10ba700 1 -- 172.21.15.97:0/1314751360 wait complete. 2024-03-21T13:27:17.583 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928233998 got 231928234001 for osd.7 2024-03-21T13:27:17.583 DEBUG:teuthology.parallel:result is None 2024-03-21T13:27:17.968 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738419 got 34359738422 for osd.0 2024-03-21T13:27:17.969 DEBUG:teuthology.parallel:result is None 2024-03-21T13:27:18.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:17 smithi144 ceph-mon[33633]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:18.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:17 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1314751360' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-21T13:27:18.187 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T13:27:18.187 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T13:27:18.187 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T13:27:18.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:17 smithi097 ceph-mon[39416]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:18.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:17 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1314751360' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-21T13:27:18.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:17 smithi097 ceph-mon[30040]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:18.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:17 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1314751360' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-21T13:27:18.680 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T13:27:18.705 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411368 got 77309411371 for osd.2 2024-03-21T13:27:18.705 DEBUG:teuthology.parallel:result is None 2024-03-21T13:27:20.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:19 smithi144 ceph-mon[33633]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:27:20.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:19 smithi097 ceph-mon[39416]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:27:20.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:19 smithi097 ceph-mon[30040]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:27:21.950 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:21 smithi097 ceph-mon[30040]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:27:21.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:21 smithi097 ceph-mon[39416]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:27:22.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:21 smithi144 ceph-mon[33633]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:27:22.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:21.841+0000 7f2bf0476700 1 -- 172.21.15.97:0/2713737939 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2be806a930 msgr2=0x7f2be806ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:22.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:21.841+0000 7f2bf0476700 1 --2- 172.21.15.97:0/2713737939 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2be806a930 0x7f2be806ad10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2be4009990 tx=0x7f2be4009ca0 comp rx=0 tx=0).stop 2024-03-21T13:27:22.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:21.842+0000 7f2bf0476700 1 -- 172.21.15.97:0/2713737939 shutdown_connections 2024-03-21T13:27:22.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:21.842+0000 7f2bf0476700 1 --2- 172.21.15.97:0/2713737939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2be81074a0 0x7f2be810f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:22.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:21.842+0000 7f2bf0476700 1 --2- 172.21.15.97:0/2713737939 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2be806a930 0x7f2be806ad10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:22.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:21.842+0000 7f2bf0476700 1 --2- 172.21.15.97:0/2713737939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2be806b2e0 0x7f2be8106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:22.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:21.842+0000 7f2bf0476700 1 -- 172.21.15.97:0/2713737939 >> 172.21.15.97:0/2713737939 conn(0x7f2be80770e0 msgr2=0x7f2be80774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:27:22.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:21.842+0000 7f2bf0476700 1 -- 172.21.15.97:0/2713737939 shutdown_connections 2024-03-21T13:27:22.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:21.842+0000 7f2bf0476700 1 -- 172.21.15.97:0/2713737939 wait complete. 2024-03-21T13:27:22.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:21.842+0000 7f2bf0476700 1 Processor -- start 2024-03-21T13:27:22.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:21.842+0000 7f2bf0476700 1 -- start start 2024-03-21T13:27:22.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:21.842+0000 7f2bf0476700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2be806a930 0x7f2be8197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:22.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:21.842+0000 7f2bf0476700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2be806b2e0 0x7f2be8198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:22.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:21.842+0000 7f2bf0476700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2be81074a0 0x7f2be819c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:22.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:21.842+0000 7f2bf0476700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be8111b50 con 0x7f2be806a930 2024-03-21T13:27:22.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:21.842+0000 7f2bf0476700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be81119d0 con 0x7f2be81074a0 2024-03-21T13:27:22.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:21.842+0000 7f2bf0476700 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2be8111cd0 con 0x7f2be806b2e0 2024-03-21T13:27:22.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:21.843+0000 7f2bee212700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2be806a930 0x7f2be8197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:22.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:21.843+0000 7f2beda11700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2be806b2e0 0x7f2be8198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:22.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:21.843+0000 7f2bee212700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2be806a930 0x7f2be8197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57750/0 (socket says 172.21.15.97:57750) 2024-03-21T13:27:22.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:21.843+0000 7f2bee212700 1 -- 172.21.15.97:0/895287215 learned_addr learned my addr 172.21.15.97:0/895287215 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:27:22.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:21.843+0000 7f2beea13700 1 --2- 172.21.15.97:0/895287215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2be81074a0 0x7f2be819c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:22.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:21.843+0000 7f2bee212700 1 -- 172.21.15.97:0/895287215 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2be806b2e0 msgr2=0x7f2be8198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:22.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:21.843+0000 7f2bee212700 1 --2- 172.21.15.97:0/895287215 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2be806b2e0 0x7f2be8198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:22.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:21.843+0000 7f2bee212700 1 -- 172.21.15.97:0/895287215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2be81074a0 msgr2=0x7f2be819c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:22.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:21.843+0000 7f2bee212700 1 --2- 172.21.15.97:0/895287215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2be81074a0 0x7f2be819c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:22.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:21.843+0000 7f2bee212700 1 -- 172.21.15.97:0/895287215 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be819d000 con 0x7f2be806a930 2024-03-21T13:27:22.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:21.843+0000 7f2beda11700 1 --2- 172.21.15.97:0/895287215 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2be806b2e0 0x7f2be8198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T13:27:22.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:21.843+0000 7f2bee212700 1 --2- 172.21.15.97:0/895287215 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2be806a930 0x7f2be8197ff0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2be4005910 tx=0x7f2be4005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:27:22.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:21.843+0000 7f2bdf7fe700 1 -- 172.21.15.97:0/895287215 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be401cce0 con 0x7f2be806a930 2024-03-21T13:27:22.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:21.843+0000 7f2bf0476700 1 -- 172.21.15.97:0/895287215 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2be819d290 con 0x7f2be806a930 2024-03-21T13:27:22.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:21.843+0000 7f2bdf7fe700 1 -- 172.21.15.97:0/895287215 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f2be40054e0 con 0x7f2be806a930 2024-03-21T13:27:22.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:21.843+0000 7f2bdf7fe700 1 -- 172.21.15.97:0/895287215 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be4004dc0 con 0x7f2be806a930 2024-03-21T13:27:22.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:21.843+0000 7f2bf0476700 1 -- 172.21.15.97:0/895287215 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2be81010c0 con 0x7f2be806a930 2024-03-21T13:27:22.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:21.844+0000 7f2bf0476700 1 -- 172.21.15.97:0/895287215 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2be81038a0 con 0x7f2be806a930 2024-03-21T13:27:22.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:21.845+0000 7f2bdf7fe700 1 -- 172.21.15.97:0/895287215 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 22) v1 ==== 92152+0+0 (secure 0 0 0) 0x7f2be4009d10 con 0x7f2be806a930 2024-03-21T13:27:22.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:21.845+0000 7f2bdf7fe700 1 --2- 172.21.15.97:0/895287215 >> v2:172.21.15.97:6800/12494470 conn(0x7f2bd406e300 0x7f2bd40707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:22.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:21.845+0000 7f2bdf7fe700 1 -- 172.21.15.97:0/895287215 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f2be4093300 con 0x7f2be806a930 2024-03-21T13:27:22.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:21.845+0000 7f2beda11700 1 --2- 172.21.15.97:0/895287215 >> v2:172.21.15.97:6800/12494470 conn(0x7f2bd406e300 0x7f2bd40707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:22.174 INFO:teuthology.orchestra.run.smithi097.stdout:197568495639 2024-03-21T13:27:22.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:21.845+0000 7f2beda11700 1 --2- 172.21.15.97:0/895287215 >> v2:172.21.15.97:6800/12494470 conn(0x7f2bd406e300 0x7f2bd40707c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2be8100d10 tx=0x7f2bd8009260 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:27:22.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:21.850+0000 7f2bdf7fe700 1 -- 172.21.15.97:0/895287215 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2be4013070 con 0x7f2be806a930 2024-03-21T13:27:22.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:22.050+0000 7f2bf0476700 1 -- 172.21.15.97:0/895287215 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f2be8199350 con 0x7f2be806a930 2024-03-21T13:27:22.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:22.051+0000 7f2bdf7fe700 1 -- 172.21.15.97:0/895287215 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2be4060a50 con 0x7f2be806a930 2024-03-21T13:27:22.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:22.053+0000 7f2bf0476700 1 -- 172.21.15.97:0/895287215 >> v2:172.21.15.97:6800/12494470 conn(0x7f2bd406e300 msgr2=0x7f2bd40707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:22.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:22.053+0000 7f2bf0476700 1 --2- 172.21.15.97:0/895287215 >> v2:172.21.15.97:6800/12494470 conn(0x7f2bd406e300 0x7f2bd40707c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2be8100d10 tx=0x7f2bd8009260 comp rx=0 tx=0).stop 2024-03-21T13:27:22.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:22.053+0000 7f2bf0476700 1 -- 172.21.15.97:0/895287215 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2be806a930 msgr2=0x7f2be8197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:22.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:22.053+0000 7f2bf0476700 1 --2- 172.21.15.97:0/895287215 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2be806a930 0x7f2be8197ff0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2be4005910 tx=0x7f2be4005ef0 comp rx=0 tx=0).stop 2024-03-21T13:27:22.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:22.053+0000 7f2bf0476700 1 -- 172.21.15.97:0/895287215 shutdown_connections 2024-03-21T13:27:22.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:22.053+0000 7f2bf0476700 1 --2- 172.21.15.97:0/895287215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2be81074a0 0x7f2be819c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:22.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:22.053+0000 7f2bf0476700 1 --2- 172.21.15.97:0/895287215 >> v2:172.21.15.97:6800/12494470 conn(0x7f2bd406e300 0x7f2bd40707c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:22.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:22.053+0000 7f2bf0476700 1 --2- 172.21.15.97:0/895287215 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2be806a930 0x7f2be8197ff0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:22.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:22.053+0000 7f2bf0476700 1 --2- 172.21.15.97:0/895287215 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2be806b2e0 0x7f2be8198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:22.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:22.053+0000 7f2bf0476700 1 -- 172.21.15.97:0/895287215 >> 172.21.15.97:0/895287215 conn(0x7f2be80770e0 msgr2=0x7f2be80fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:27:22.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:22.053+0000 7f2bf0476700 1 -- 172.21.15.97:0/895287215 shutdown_connections 2024-03-21T13:27:22.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:22.053+0000 7f2bf0476700 1 -- 172.21.15.97:0/895287215 wait complete. 2024-03-21T13:27:22.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:27:21 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:27:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:27:23.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:22 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/895287215' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-21T13:27:23.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:22 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:27:23.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:22 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/895287215' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-21T13:27:23.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:22 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:27:23.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:22 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/895287215' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-21T13:27:23.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:22 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:27:23.237 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495637 got 197568495639 for osd.6 2024-03-21T13:27:23.237 DEBUG:teuthology.parallel:result is None 2024-03-21T13:27:23.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:27:23 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:27:24.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:23 smithi144 ceph-mon[33633]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:24.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:23 smithi097 ceph-mon[39416]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:24.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:23 smithi097 ceph-mon[30040]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:24.934 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:24 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:27:24.934 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:24 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:27:25.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:24 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:27:25.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.283+0000 7fc18ae68700 1 -- 172.21.15.97:0/3977631567 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc184101ca0 msgr2=0x7fc184102120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:25.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.283+0000 7fc18ae68700 1 --2- 172.21.15.97:0/3977631567 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc184101ca0 0x7fc184102120 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc1700099c0 tx=0x7fc170009cd0 comp rx=0 tx=0).stop 2024-03-21T13:27:25.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.283+0000 7fc18ae68700 1 -- 172.21.15.97:0/3977631567 shutdown_connections 2024-03-21T13:27:25.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.283+0000 7fc18ae68700 1 --2- 172.21.15.97:0/3977631567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc184102660 0x7fc18410dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:25.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.283+0000 7fc18ae68700 1 --2- 172.21.15.97:0/3977631567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc184106120 0x7fc184106500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:25.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.283+0000 7fc18ae68700 1 --2- 172.21.15.97:0/3977631567 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc184101ca0 0x7fc184102120 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:25.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.283+0000 7fc18ae68700 1 -- 172.21.15.97:0/3977631567 >> 172.21.15.97:0/3977631567 conn(0x7fc1840763e0 msgr2=0x7fc1840767f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:27:25.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.284+0000 7fc18ae68700 1 -- 172.21.15.97:0/3977631567 shutdown_connections 2024-03-21T13:27:25.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.284+0000 7fc18ae68700 1 -- 172.21.15.97:0/3977631567 wait complete. 2024-03-21T13:27:25.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.284+0000 7fc18ae68700 1 Processor -- start 2024-03-21T13:27:25.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.284+0000 7fc18ae68700 1 -- start start 2024-03-21T13:27:25.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.284+0000 7fc18ae68700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc184101ca0 0x7fc184102e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:25.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.284+0000 7fc18ae68700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc184102660 0x7fc184103360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:25.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.284+0000 7fc18ae68700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc184106120 0x7fc1841038a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:25.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.284+0000 7fc18ae68700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc184110b10 con 0x7fc184101ca0 2024-03-21T13:27:25.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.284+0000 7fc18ae68700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc184110990 con 0x7fc184102660 2024-03-21T13:27:25.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.284+0000 7fc18ae68700 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc184110c90 con 0x7fc184106120 2024-03-21T13:27:25.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.284+0000 7fc188c04700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc184101ca0 0x7fc184102e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:25.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.284+0000 7fc188c04700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc184101ca0 0x7fc184102e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57768/0 (socket says 172.21.15.97:57768) 2024-03-21T13:27:25.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.284+0000 7fc189405700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc184106120 0x7fc1841038a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:25.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.284+0000 7fc188c04700 1 -- 172.21.15.97:0/1431265094 learned_addr learned my addr 172.21.15.97:0/1431265094 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:27:25.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.285+0000 7fc183fff700 1 --2- 172.21.15.97:0/1431265094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc184102660 0x7fc184103360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:25.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.285+0000 7fc189405700 1 -- 172.21.15.97:0/1431265094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc184102660 msgr2=0x7fc184103360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:25.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.285+0000 7fc189405700 1 --2- 172.21.15.97:0/1431265094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc184102660 0x7fc184103360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:25.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.285+0000 7fc189405700 1 -- 172.21.15.97:0/1431265094 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc184101ca0 msgr2=0x7fc184102e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:25.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.285+0000 7fc189405700 1 --2- 172.21.15.97:0/1431265094 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc184101ca0 0x7fc184102e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:25.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.285+0000 7fc189405700 1 -- 172.21.15.97:0/1431265094 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1841a6540 con 0x7fc184106120 2024-03-21T13:27:25.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.285+0000 7fc188c04700 1 --2- 172.21.15.97:0/1431265094 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc184101ca0 0x7fc184102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T13:27:25.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.285+0000 7fc189405700 1 --2- 172.21.15.97:0/1431265094 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc184106120 0x7fc1841038a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc17c00ab40 tx=0x7fc17c00ae50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:27:25.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.285+0000 7fc181ffb700 1 -- 172.21.15.97:0/1431265094 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17c018070 con 0x7fc184106120 2024-03-21T13:27:25.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.285+0000 7fc181ffb700 1 -- 172.21.15.97:0/1431265094 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fc17c00c640 con 0x7fc184106120 2024-03-21T13:27:25.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.285+0000 7fc18ae68700 1 -- 172.21.15.97:0/1431265094 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1841a6830 con 0x7fc184106120 2024-03-21T13:27:25.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.285+0000 7fc181ffb700 1 -- 172.21.15.97:0/1431265094 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17c004290 con 0x7fc184106120 2024-03-21T13:27:25.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.285+0000 7fc18ae68700 1 -- 172.21.15.97:0/1431265094 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1841a6d40 con 0x7fc184106120 2024-03-21T13:27:25.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.286+0000 7fc18ae68700 1 -- 172.21.15.97:0/1431265094 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc184068e50 con 0x7fc184106120 2024-03-21T13:27:25.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.287+0000 7fc181ffb700 1 -- 172.21.15.97:0/1431265094 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 22) v1 ==== 92152+0+0 (secure 0 0 0) 0x7fc17c01b070 con 0x7fc184106120 2024-03-21T13:27:25.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.287+0000 7fc181ffb700 1 --2- 172.21.15.97:0/1431265094 >> v2:172.21.15.97:6800/12494470 conn(0x7fc17406e300 0x7fc1740707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:25.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.287+0000 7fc188c04700 1 --2- 172.21.15.97:0/1431265094 >> v2:172.21.15.97:6800/12494470 conn(0x7fc17406e300 0x7fc1740707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:25.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.287+0000 7fc181ffb700 1 -- 172.21.15.97:0/1431265094 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6259+0+0 (secure 0 0 0) 0x7fc17c08d3e0 con 0x7fc184106120 2024-03-21T13:27:25.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.287+0000 7fc188c04700 1 --2- 172.21.15.97:0/1431265094 >> v2:172.21.15.97:6800/12494470 conn(0x7fc17406e300 0x7fc1740707c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc1780103d0 tx=0x7fc178007480 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:27:25.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.292+0000 7fc181ffb700 1 -- 172.21.15.97:0/1431265094 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc17c05abb0 con 0x7fc184106120 2024-03-21T13:27:25.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.482+0000 7fade7eff700 1 -- 172.21.15.97:0/4113694131 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fade01024a0 msgr2=0x7fade0105d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:25.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.482+0000 7fade7eff700 1 --2- 172.21.15.97:0/4113694131 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fade01024a0 0x7fade0105d30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fadd0009920 tx=0x7fadd0009c30 comp rx=0 tx=0).stop 2024-03-21T13:27:25.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.482+0000 7fade7eff700 1 -- 172.21.15.97:0/4113694131 shutdown_connections 2024-03-21T13:27:25.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.482+0000 7fade7eff700 1 --2- 172.21.15.97:0/4113694131 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fade0106410 0x7fade010dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:25.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.482+0000 7fade7eff700 1 --2- 172.21.15.97:0/4113694131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fade0101af0 0x7fade0101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:25.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.482+0000 7fade7eff700 1 --2- 172.21.15.97:0/4113694131 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fade01024a0 0x7fade0105d30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:25.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.482+0000 7fade7eff700 1 -- 172.21.15.97:0/4113694131 >> 172.21.15.97:0/4113694131 conn(0x7fade0077690 msgr2=0x7fade00feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:27:25.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.482+0000 7fade7eff700 1 -- 172.21.15.97:0/4113694131 shutdown_connections 2024-03-21T13:27:25.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.482+0000 7fade7eff700 1 -- 172.21.15.97:0/4113694131 wait complete. 2024-03-21T13:27:25.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.483+0000 7fade7eff700 1 Processor -- start 2024-03-21T13:27:25.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.483+0000 7fade7eff700 1 -- start start 2024-03-21T13:27:25.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.483+0000 7fade7eff700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fade0101af0 0x7fade0197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:25.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.483+0000 7fade7eff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fade01024a0 0x7fade01976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:25.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.483+0000 7fade7eff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fade0106410 0x7fade019ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:25.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.483+0000 7fade7eff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade0110b10 con 0x7fade01024a0 2024-03-21T13:27:25.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.483+0000 7fade7eff700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade0110990 con 0x7fade0106410 2024-03-21T13:27:25.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.483+0000 7fade7eff700 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fade0110c90 con 0x7fade0101af0 2024-03-21T13:27:25.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.483+0000 7fade5c9b700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fade0101af0 0x7fade0197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:25.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.483+0000 7fade549a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fade01024a0 0x7fade01976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:25.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.483+0000 7fade5c9b700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fade0101af0 0x7fade0197160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.97:37948/0 (socket says 172.21.15.97:37948) 2024-03-21T13:27:25.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.483+0000 7fade5c9b700 1 -- 172.21.15.97:0/1589171391 learned_addr learned my addr 172.21.15.97:0/1589171391 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:27:25.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.483+0000 7fade649c700 1 --2- 172.21.15.97:0/1589171391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fade0106410 0x7fade019ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:25.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.483+0000 7fade5c9b700 1 -- 172.21.15.97:0/1589171391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fade0106410 msgr2=0x7fade019ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:25.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.483+0000 7fade5c9b700 1 --2- 172.21.15.97:0/1589171391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fade0106410 0x7fade019ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:25.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.483+0000 7fade5c9b700 1 -- 172.21.15.97:0/1589171391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fade01024a0 msgr2=0x7fade01976a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:25.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.483+0000 7fade5c9b700 1 --2- 172.21.15.97:0/1589171391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fade01024a0 0x7fade01976a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:25.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.483+0000 7fade5c9b700 1 -- 172.21.15.97:0/1589171391 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade019c170 con 0x7fade0101af0 2024-03-21T13:27:25.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.483+0000 7fade549a700 1 --2- 172.21.15.97:0/1589171391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fade01024a0 0x7fade01976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T13:27:25.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.483+0000 7fade649c700 1 --2- 172.21.15.97:0/1589171391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fade0106410 0x7fade019ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T13:27:25.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.484+0000 7fade5c9b700 1 --2- 172.21.15.97:0/1589171391 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fade0101af0 0x7fade0197160 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7faddc00d980 tx=0x7faddc00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:27:25.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.484+0000 7fadd6ffd700 1 -- 172.21.15.97:0/1589171391 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc017070 con 0x7fade0101af0 2024-03-21T13:27:25.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.484+0000 7fadd6ffd700 1 -- 172.21.15.97:0/1589171391 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7faddc005020 con 0x7fade0101af0 2024-03-21T13:27:25.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.484+0000 7fadd6ffd700 1 -- 172.21.15.97:0/1589171391 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc009b20 con 0x7fade0101af0 2024-03-21T13:27:25.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.484+0000 7fade7eff700 1 -- 172.21.15.97:0/1589171391 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fade019c460 con 0x7fade0101af0 2024-03-21T13:27:25.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.484+0000 7fade7eff700 1 -- 172.21.15.97:0/1589171391 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fade0100780 con 0x7fade0101af0 2024-03-21T13:27:25.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.485+0000 7fade7eff700 1 -- 172.21.15.97:0/1589171391 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fade0068e50 con 0x7fade0101af0 2024-03-21T13:27:25.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.485+0000 7fadd6ffd700 1 -- 172.21.15.97:0/1589171391 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 22) v1 ==== 92152+0+0 (secure 0 0 0) 0x7faddc009cc0 con 0x7fade0101af0 2024-03-21T13:27:25.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.485+0000 7fadd6ffd700 1 --2- 172.21.15.97:0/1589171391 >> v2:172.21.15.97:6800/12494470 conn(0x7fadcc06e300 0x7fadcc0707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:25.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.486+0000 7fade549a700 1 --2- 172.21.15.97:0/1589171391 >> v2:172.21.15.97:6800/12494470 conn(0x7fadcc06e300 0x7fadcc0707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:25.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.486+0000 7fadd6ffd700 1 -- 172.21.15.97:0/1589171391 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6259+0+0 (secure 0 0 0) 0x7faddc08e2d0 con 0x7fade0101af0 2024-03-21T13:27:25.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.486+0000 7fade549a700 1 --2- 172.21.15.97:0/1589171391 >> v2:172.21.15.97:6800/12494470 conn(0x7fadcc06e300 0x7fadcc0707c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fadd00098f0 tx=0x7fadd0019040 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:27:25.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.490+0000 7fadd6ffd700 1 -- 172.21.15.97:0/1589171391 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7faddc020020 con 0x7fade0101af0 2024-03-21T13:27:25.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.495+0000 7fc18ae68700 1 -- 172.21.15.97:0/1431265094 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc1841a6fa0 con 0x7fc184106120 2024-03-21T13:27:25.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.495+0000 7fc181ffb700 1 -- 172.21.15.97:0/1431265094 <== mon.2 v2:172.21.15.97:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc17c05e600 con 0x7fc184106120 2024-03-21T13:27:25.496 INFO:teuthology.orchestra.run.smithi097.stdout:55834574898 2024-03-21T13:27:25.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.498+0000 7fc18ae68700 1 -- 172.21.15.97:0/1431265094 >> v2:172.21.15.97:6800/12494470 conn(0x7fc17406e300 msgr2=0x7fc1740707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:25.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.498+0000 7fc18ae68700 1 --2- 172.21.15.97:0/1431265094 >> v2:172.21.15.97:6800/12494470 conn(0x7fc17406e300 0x7fc1740707c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc1780103d0 tx=0x7fc178007480 comp rx=0 tx=0).stop 2024-03-21T13:27:25.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.498+0000 7fc18ae68700 1 -- 172.21.15.97:0/1431265094 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc184106120 msgr2=0x7fc1841038a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:25.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.498+0000 7fc18ae68700 1 --2- 172.21.15.97:0/1431265094 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc184106120 0x7fc1841038a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc17c00ab40 tx=0x7fc17c00ae50 comp rx=0 tx=0).stop 2024-03-21T13:27:25.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.498+0000 7fc18ae68700 1 -- 172.21.15.97:0/1431265094 shutdown_connections 2024-03-21T13:27:25.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.498+0000 7fc18ae68700 1 --2- 172.21.15.97:0/1431265094 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc184106120 0x7fc1841038a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:25.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.498+0000 7fc18ae68700 1 --2- 172.21.15.97:0/1431265094 >> v2:172.21.15.97:6800/12494470 conn(0x7fc17406e300 0x7fc1740707c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:25.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.498+0000 7fc18ae68700 1 --2- 172.21.15.97:0/1431265094 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc184101ca0 0x7fc184102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:25.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.498+0000 7fc18ae68700 1 --2- 172.21.15.97:0/1431265094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc184102660 0x7fc184103360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:25.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.498+0000 7fc18ae68700 1 -- 172.21.15.97:0/1431265094 >> 172.21.15.97:0/1431265094 conn(0x7fc1840763e0 msgr2=0x7fc1841012a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:27:25.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.498+0000 7fc18ae68700 1 -- 172.21.15.97:0/1431265094 shutdown_connections 2024-03-21T13:27:25.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.498+0000 7fc18ae68700 1 -- 172.21.15.97:0/1431265094 wait complete. 2024-03-21T13:27:25.702 INFO:teuthology.orchestra.run.smithi097.stdout:128849018915 2024-03-21T13:27:25.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.701+0000 7fade7eff700 1 -- 172.21.15.97:0/1589171391 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fade01984c0 con 0x7fade0101af0 2024-03-21T13:27:25.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.701+0000 7fadd6ffd700 1 -- 172.21.15.97:0/1589171391 <== mon.2 v2:172.21.15.97:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faddc05baa0 con 0x7fade0101af0 2024-03-21T13:27:25.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.703+0000 7fade7eff700 1 -- 172.21.15.97:0/1589171391 >> v2:172.21.15.97:6800/12494470 conn(0x7fadcc06e300 msgr2=0x7fadcc0707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:25.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.703+0000 7fade7eff700 1 --2- 172.21.15.97:0/1589171391 >> v2:172.21.15.97:6800/12494470 conn(0x7fadcc06e300 0x7fadcc0707c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fadd00098f0 tx=0x7fadd0019040 comp rx=0 tx=0).stop 2024-03-21T13:27:25.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.703+0000 7fade7eff700 1 -- 172.21.15.97:0/1589171391 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fade0101af0 msgr2=0x7fade0197160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:25.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.704+0000 7fade7eff700 1 --2- 172.21.15.97:0/1589171391 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fade0101af0 0x7fade0197160 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7faddc00d980 tx=0x7faddc00dc90 comp rx=0 tx=0).stop 2024-03-21T13:27:25.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.704+0000 7fade7eff700 1 -- 172.21.15.97:0/1589171391 shutdown_connections 2024-03-21T13:27:25.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.704+0000 7fade7eff700 1 --2- 172.21.15.97:0/1589171391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fade0106410 0x7fade019ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:25.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.704+0000 7fade7eff700 1 --2- 172.21.15.97:0/1589171391 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fade0101af0 0x7fade0197160 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:25.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.704+0000 7fade7eff700 1 --2- 172.21.15.97:0/1589171391 >> v2:172.21.15.97:6800/12494470 conn(0x7fadcc06e300 0x7fadcc0707c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:25.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.704+0000 7fade7eff700 1 --2- 172.21.15.97:0/1589171391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fade01024a0 0x7fade01976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:25.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.704+0000 7fade7eff700 1 -- 172.21.15.97:0/1589171391 >> 172.21.15.97:0/1589171391 conn(0x7fade0077690 msgr2=0x7fade0103e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:27:25.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.704+0000 7fade7eff700 1 -- 172.21.15.97:0/1589171391 shutdown_connections 2024-03-21T13:27:25.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.704+0000 7fade7eff700 1 -- 172.21.15.97:0/1589171391 wait complete. 2024-03-21T13:27:25.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.803+0000 7f0aa8ebf700 1 -- 172.21.15.97:0/2799395649 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0aa4101cd0 msgr2=0x7f0aa4105cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:25.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.803+0000 7f0aa8ebf700 1 --2- 172.21.15.97:0/2799395649 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0aa4101cd0 0x7f0aa4105cd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0a980098f0 tx=0x7f0a98009c00 comp rx=0 tx=0).stop 2024-03-21T13:27:25.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.803+0000 7f0aa8ebf700 1 -- 172.21.15.97:0/2799395649 shutdown_connections 2024-03-21T13:27:25.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.803+0000 7f0aa8ebf700 1 --2- 172.21.15.97:0/2799395649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0aa41063b0 0x7f0aa410df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:25.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.803+0000 7f0aa8ebf700 1 --2- 172.21.15.97:0/2799395649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0aa4101320 0x7f0aa4101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:25.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.803+0000 7f0aa8ebf700 1 --2- 172.21.15.97:0/2799395649 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0aa4101cd0 0x7f0aa4105cd0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:25.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.803+0000 7f0aa8ebf700 1 -- 172.21.15.97:0/2799395649 >> 172.21.15.97:0/2799395649 conn(0x7f0aa40770b0 msgr2=0x7f0aa40774c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:27:25.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.803+0000 7f0aa8ebf700 1 -- 172.21.15.97:0/2799395649 shutdown_connections 2024-03-21T13:27:25.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.803+0000 7f0aa8ebf700 1 -- 172.21.15.97:0/2799395649 wait complete. 2024-03-21T13:27:25.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.804+0000 7f0aa8ebf700 1 Processor -- start 2024-03-21T13:27:25.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.804+0000 7f0aa8ebf700 1 -- start start 2024-03-21T13:27:25.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.804+0000 7f0aa8ebf700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0aa4101320 0x7f0aa4196fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:25.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.804+0000 7f0aa8ebf700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0aa4101cd0 0x7f0aa41974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:25.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.804+0000 7f0aa8ebf700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0aa41063b0 0x7f0aa419b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:25.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.804+0000 7f0aa8ebf700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa4110a40 con 0x7f0aa41063b0 2024-03-21T13:27:25.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.804+0000 7f0aa8ebf700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa41108c0 con 0x7f0aa4101cd0 2024-03-21T13:27:25.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.804+0000 7f0aa8ebf700 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa4110bc0 con 0x7f0aa4101320 2024-03-21T13:27:25.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.804+0000 7f0aa259c700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0aa4101320 0x7f0aa4196fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:25.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.804+0000 7f0aa2d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0aa41063b0 0x7f0aa419b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:25.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.804+0000 7f0aa259c700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0aa4101320 0x7f0aa4196fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.97:37954/0 (socket says 172.21.15.97:37954) 2024-03-21T13:27:25.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.805+0000 7f0aa2d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0aa41063b0 0x7f0aa419b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57806/0 (socket says 172.21.15.97:57806) 2024-03-21T13:27:25.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.805+0000 7f0aa259c700 1 -- 172.21.15.97:0/3823352432 learned_addr learned my addr 172.21.15.97:0/3823352432 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:27:25.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.805+0000 7f0aa1d9b700 1 --2- 172.21.15.97:0/3823352432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0aa4101cd0 0x7f0aa41974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:25.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.805+0000 7f0aa259c700 1 -- 172.21.15.97:0/3823352432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0aa4101cd0 msgr2=0x7f0aa41974e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:25.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.805+0000 7f0aa259c700 1 --2- 172.21.15.97:0/3823352432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0aa4101cd0 0x7f0aa41974e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:25.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.805+0000 7f0aa259c700 1 -- 172.21.15.97:0/3823352432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0aa41063b0 msgr2=0x7f0aa419b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:25.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.805+0000 7f0aa259c700 1 --2- 172.21.15.97:0/3823352432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0aa41063b0 0x7f0aa419b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:25.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.805+0000 7f0aa259c700 1 -- 172.21.15.97:0/3823352432 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa419bfb0 con 0x7f0aa4101320 2024-03-21T13:27:25.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.805+0000 7f0aa2d9d700 1 --2- 172.21.15.97:0/3823352432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0aa41063b0 0x7f0aa419b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T13:27:25.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.805+0000 7f0aa259c700 1 --2- 172.21.15.97:0/3823352432 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0aa4101320 0x7f0aa4196fa0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c007f10 tx=0x7f0a9c00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:27:25.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.805+0000 7f0a937fe700 1 -- 172.21.15.97:0/3823352432 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c004bb0 con 0x7f0aa4101320 2024-03-21T13:27:25.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.805+0000 7f0aa8ebf700 1 -- 172.21.15.97:0/3823352432 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0aa419c2a0 con 0x7f0aa4101320 2024-03-21T13:27:25.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.805+0000 7f0a937fe700 1 -- 172.21.15.97:0/3823352432 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f0a9c004d50 con 0x7f0aa4101320 2024-03-21T13:27:25.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.805+0000 7f0a937fe700 1 -- 172.21.15.97:0/3823352432 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c005980 con 0x7f0aa4101320 2024-03-21T13:27:25.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.805+0000 7f0aa8ebf700 1 -- 172.21.15.97:0/3823352432 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aa41a3b20 con 0x7f0aa4101320 2024-03-21T13:27:25.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.806+0000 7f0aa8ebf700 1 -- 172.21.15.97:0/3823352432 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0aa41027b0 con 0x7f0aa4101320 2024-03-21T13:27:25.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.807+0000 7f0a937fe700 1 -- 172.21.15.97:0/3823352432 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 22) v1 ==== 92152+0+0 (secure 0 0 0) 0x7f0a9c01a020 con 0x7f0aa4101320 2024-03-21T13:27:25.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.807+0000 7f0a937fe700 1 --2- 172.21.15.97:0/3823352432 >> v2:172.21.15.97:6800/12494470 conn(0x7f0a8c06e300 0x7f0a8c0707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:25.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.807+0000 7f0a937fe700 1 -- 172.21.15.97:0/3823352432 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f0a9c08db00 con 0x7f0aa4101320 2024-03-21T13:27:25.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.807+0000 7f0aa1d9b700 1 --2- 172.21.15.97:0/3823352432 >> v2:172.21.15.97:6800/12494470 conn(0x7f0a8c06e300 0x7f0a8c0707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:25.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.808+0000 7f0aa1d9b700 1 --2- 172.21.15.97:0/3823352432 >> v2:172.21.15.97:6800/12494470 conn(0x7f0a8c06e300 0x7f0a8c0707c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0a98009810 tx=0x7f0a98019040 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:27:25.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:25.812+0000 7f0a937fe700 1 -- 172.21.15.97:0/3823352432 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0a9c010070 con 0x7f0aa4101320 2024-03-21T13:27:25.982 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:25 smithi097 ceph-mon[39416]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:27:25.982 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:25 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1431265094' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-21T13:27:25.982 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:25 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1589171391' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-21T13:27:25.982 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:25 smithi097 ceph-mon[30040]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:27:25.982 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:25 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1431265094' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-21T13:27:25.983 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:25 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1589171391' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-21T13:27:26.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.001+0000 7fc015bb4700 1 -- 172.21.15.97:0/99475860 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc010102d80 msgr2=0x7fc010106dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:26.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.001+0000 7fc015bb4700 1 --2- 172.21.15.97:0/99475860 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc010102d80 0x7fc010106dd0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc0040099c0 tx=0x7fc004009cd0 comp rx=0 tx=0).stop 2024-03-21T13:27:26.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.001+0000 7fc015bb4700 1 -- 172.21.15.97:0/99475860 shutdown_connections 2024-03-21T13:27:26.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.001+0000 7fc015bb4700 1 --2- 172.21.15.97:0/99475860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0101074b0 0x7fc01010f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:26.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.001+0000 7fc015bb4700 1 --2- 172.21.15.97:0/99475860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0101023d0 0x7fc0101027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:26.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.001+0000 7fc015bb4700 1 --2- 172.21.15.97:0/99475860 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc010102d80 0x7fc010106dd0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:26.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.001+0000 7fc015bb4700 1 -- 172.21.15.97:0/99475860 >> 172.21.15.97:0/99475860 conn(0x7fc0100fdc40 msgr2=0x7fc010100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:27:26.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.001+0000 7fc015bb4700 1 -- 172.21.15.97:0/99475860 shutdown_connections 2024-03-21T13:27:26.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.001+0000 7fc015bb4700 1 -- 172.21.15.97:0/99475860 wait complete. 2024-03-21T13:27:26.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.002+0000 7fc015bb4700 1 Processor -- start 2024-03-21T13:27:26.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.002+0000 7fc015bb4700 1 -- start start 2024-03-21T13:27:26.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.002+0000 7fc015bb4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0101023d0 0x7fc010198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:26.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.002+0000 7fc015bb4700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc010102d80 0x7fc0101985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:26.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.002+0000 7fc015bb4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0101074b0 0x7fc01019c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:26.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.002+0000 7fc015bb4700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc010111ba0 con 0x7fc0101023d0 2024-03-21T13:27:26.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.002+0000 7fc015bb4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc010111a20 con 0x7fc0101074b0 2024-03-21T13:27:26.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.002+0000 7fc015bb4700 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc010111d20 con 0x7fc010102d80 2024-03-21T13:27:26.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.002+0000 7fc00effd700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc010102d80 0x7fc0101985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:26.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.002+0000 7fc00effd700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc010102d80 0x7fc0101985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.97:37970/0 (socket says 172.21.15.97:37970) 2024-03-21T13:27:26.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.002+0000 7fc00effd700 1 -- 172.21.15.97:0/1529644197 learned_addr learned my addr 172.21.15.97:0/1529644197 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:27:26.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.002+0000 7fc00f7fe700 1 --2- 172.21.15.97:0/1529644197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0101023d0 0x7fc010198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:26.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.003+0000 7fc00effd700 1 -- 172.21.15.97:0/1529644197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0101074b0 msgr2=0x7fc01019c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:26.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.003+0000 7fc00ffff700 1 --2- 172.21.15.97:0/1529644197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0101074b0 0x7fc01019c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:26.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.003+0000 7fc00effd700 1 --2- 172.21.15.97:0/1529644197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0101074b0 0x7fc01019c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:26.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.003+0000 7fc00effd700 1 -- 172.21.15.97:0/1529644197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0101023d0 msgr2=0x7fc010198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:26.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.003+0000 7fc00effd700 1 --2- 172.21.15.97:0/1529644197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0101023d0 0x7fc010198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:26.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.003+0000 7fc00effd700 1 -- 172.21.15.97:0/1529644197 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc01019d020 con 0x7fc010102d80 2024-03-21T13:27:26.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.003+0000 7fc00f7fe700 1 --2- 172.21.15.97:0/1529644197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0101023d0 0x7fc010198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T13:27:26.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.003+0000 7fc00effd700 1 --2- 172.21.15.97:0/1529644197 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc010102d80 0x7fc0101985a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc00400f670 tx=0x7fc00400f6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:27:26.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.003+0000 7fc00cff9700 1 -- 172.21.15.97:0/1529644197 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc004027070 con 0x7fc010102d80 2024-03-21T13:27:26.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.003+0000 7fc00cff9700 1 -- 172.21.15.97:0/1529644197 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fc00400fbc0 con 0x7fc010102d80 2024-03-21T13:27:26.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.003+0000 7fc00cff9700 1 -- 172.21.15.97:0/1529644197 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0040173f0 con 0x7fc010102d80 2024-03-21T13:27:26.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.003+0000 7fc015bb4700 1 -- 172.21.15.97:0/1529644197 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc01019d2b0 con 0x7fc010102d80 2024-03-21T13:27:26.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.003+0000 7fc015bb4700 1 -- 172.21.15.97:0/1529644197 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0101a4b90 con 0x7fc010102d80 2024-03-21T13:27:26.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.004+0000 7fc015bb4700 1 -- 172.21.15.97:0/1529644197 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc010068e50 con 0x7fc010102d80 2024-03-21T13:27:26.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.005+0000 7fc00cff9700 1 -- 172.21.15.97:0/1529644197 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 22) v1 ==== 92152+0+0 (secure 0 0 0) 0x7fc00402a070 con 0x7fc010102d80 2024-03-21T13:27:26.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.005+0000 7fc00cff9700 1 --2- 172.21.15.97:0/1529644197 >> v2:172.21.15.97:6800/12494470 conn(0x7fbff806e300 0x7fbff80707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:26.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.005+0000 7fc00cff9700 1 -- 172.21.15.97:0/1529644197 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6259+0+0 (secure 0 0 0) 0x7fc00408f160 con 0x7fc010102d80 2024-03-21T13:27:26.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.005+0000 7fc00f7fe700 1 --2- 172.21.15.97:0/1529644197 >> v2:172.21.15.97:6800/12494470 conn(0x7fbff806e300 0x7fbff80707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:26.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.006+0000 7fc00f7fe700 1 --2- 172.21.15.97:0/1529644197 >> v2:172.21.15.97:6800/12494470 conn(0x7fbff806e300 0x7fbff80707c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc000009680 tx=0x7fc000009260 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:27:26.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.010+0000 7fc00cff9700 1 -- 172.21.15.97:0/1529644197 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc004013070 con 0x7fc010102d80 2024-03-21T13:27:26.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.016+0000 7f0aa8ebf700 1 -- 172.21.15.97:0/3823352432 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0aa41a3db0 con 0x7f0aa4101320 2024-03-21T13:27:26.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.016+0000 7f0a937fe700 1 -- 172.21.15.97:0/3823352432 <== mon.2 v2:172.21.15.97:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0a9c05b250 con 0x7f0aa4101320 2024-03-21T13:27:26.016 INFO:teuthology.orchestra.run.smithi097.stdout:107374182440 2024-03-21T13:27:26.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.018+0000 7f0aa8ebf700 1 -- 172.21.15.97:0/3823352432 >> v2:172.21.15.97:6800/12494470 conn(0x7f0a8c06e300 msgr2=0x7f0a8c0707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:26.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.018+0000 7f0aa8ebf700 1 --2- 172.21.15.97:0/3823352432 >> v2:172.21.15.97:6800/12494470 conn(0x7f0a8c06e300 0x7f0a8c0707c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0a98009810 tx=0x7f0a98019040 comp rx=0 tx=0).stop 2024-03-21T13:27:26.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.018+0000 7f0aa8ebf700 1 -- 172.21.15.97:0/3823352432 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0aa4101320 msgr2=0x7f0aa4196fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:26.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.018+0000 7f0aa8ebf700 1 --2- 172.21.15.97:0/3823352432 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0aa4101320 0x7f0aa4196fa0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c007f10 tx=0x7f0a9c00d3b0 comp rx=0 tx=0).stop 2024-03-21T13:27:26.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.018+0000 7f0aa8ebf700 1 -- 172.21.15.97:0/3823352432 shutdown_connections 2024-03-21T13:27:26.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.018+0000 7f0aa8ebf700 1 --2- 172.21.15.97:0/3823352432 >> v2:172.21.15.97:6800/12494470 conn(0x7f0a8c06e300 0x7f0a8c0707c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:26.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.018+0000 7f0aa8ebf700 1 --2- 172.21.15.97:0/3823352432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0aa41063b0 0x7f0aa419b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:26.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.018+0000 7f0aa8ebf700 1 --2- 172.21.15.97:0/3823352432 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0aa4101320 0x7f0aa4196fa0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:26.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.018+0000 7f0aa8ebf700 1 --2- 172.21.15.97:0/3823352432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0aa4101cd0 0x7f0aa41974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:26.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.018+0000 7f0aa8ebf700 1 -- 172.21.15.97:0/3823352432 >> 172.21.15.97:0/3823352432 conn(0x7f0aa40770b0 msgr2=0x7f0aa40fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:27:26.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.018+0000 7f0aa8ebf700 1 -- 172.21.15.97:0/3823352432 shutdown_connections 2024-03-21T13:27:26.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.018+0000 7f0aa8ebf700 1 -- 172.21.15.97:0/3823352432 wait complete. 2024-03-21T13:27:26.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:25 smithi144 ceph-mon[33633]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:27:26.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:25 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1431265094' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-21T13:27:26.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:25 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1589171391' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-21T13:27:26.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.216+0000 7fc015bb4700 1 -- 172.21.15.97:0/1529644197 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fc0101a4d60 con 0x7fc010102d80 2024-03-21T13:27:26.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.217+0000 7fc00cff9700 1 -- 172.21.15.97:0/1529644197 <== mon.2 v2:172.21.15.97:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc00405c8b0 con 0x7fc010102d80 2024-03-21T13:27:26.217 INFO:teuthology.orchestra.run.smithi097.stdout:163208757277 2024-03-21T13:27:26.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.219+0000 7fc015bb4700 1 -- 172.21.15.97:0/1529644197 >> v2:172.21.15.97:6800/12494470 conn(0x7fbff806e300 msgr2=0x7fbff80707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:26.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.219+0000 7fc015bb4700 1 --2- 172.21.15.97:0/1529644197 >> v2:172.21.15.97:6800/12494470 conn(0x7fbff806e300 0x7fbff80707c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc000009680 tx=0x7fc000009260 comp rx=0 tx=0).stop 2024-03-21T13:27:26.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.219+0000 7fc015bb4700 1 -- 172.21.15.97:0/1529644197 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc010102d80 msgr2=0x7fc0101985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:26.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.219+0000 7fc015bb4700 1 --2- 172.21.15.97:0/1529644197 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc010102d80 0x7fc0101985a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc00400f670 tx=0x7fc00400f6a0 comp rx=0 tx=0).stop 2024-03-21T13:27:26.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.219+0000 7fc015bb4700 1 -- 172.21.15.97:0/1529644197 shutdown_connections 2024-03-21T13:27:26.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.219+0000 7fc015bb4700 1 --2- 172.21.15.97:0/1529644197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0101074b0 0x7fc01019c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:26.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.219+0000 7fc015bb4700 1 --2- 172.21.15.97:0/1529644197 >> v2:172.21.15.97:6800/12494470 conn(0x7fbff806e300 0x7fbff80707c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:26.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.219+0000 7fc015bb4700 1 --2- 172.21.15.97:0/1529644197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0101023d0 0x7fc010198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:26.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.219+0000 7fc015bb4700 1 --2- 172.21.15.97:0/1529644197 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc010102d80 0x7fc0101985a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:26.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.219+0000 7fc015bb4700 1 -- 172.21.15.97:0/1529644197 >> 172.21.15.97:0/1529644197 conn(0x7fc0100fdc40 msgr2=0x7fc0100fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:27:26.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.219+0000 7fc015bb4700 1 -- 172.21.15.97:0/1529644197 shutdown_connections 2024-03-21T13:27:26.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:26.219+0000 7fc015bb4700 1 -- 172.21.15.97:0/1529644197 wait complete. 2024-03-21T13:27:26.938 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182436 got 107374182440 for osd.3 2024-03-21T13:27:26.938 DEBUG:teuthology.parallel:result is None 2024-03-21T13:27:27.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:26 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3823352432' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-21T13:27:27.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:26 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1529644197' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-21T13:27:27.199 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:26 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3823352432' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-21T13:27:27.199 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:26 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1529644197' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-21T13:27:27.199 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:26 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3823352432' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-21T13:27:27.199 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:26 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1529644197' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-21T13:27:27.493 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574895 got 55834574898 for osd.1 2024-03-21T13:27:27.493 DEBUG:teuthology.parallel:result is None 2024-03-21T13:27:27.930 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757273 got 163208757277 for osd.5 2024-03-21T13:27:27.931 DEBUG:teuthology.parallel:result is None 2024-03-21T13:27:28.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:27 smithi144 ceph-mon[33633]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:28.214 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:27 smithi097 ceph-mon[39416]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:28.214 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:27 smithi097 ceph-mon[30040]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:28.537 INFO:tasks.cephadm.ceph_manager.ceph:need seq 128849018911 got 128849018915 for osd.4 2024-03-21T13:27:28.538 DEBUG:teuthology.parallel:result is None 2024-03-21T13:27:28.538 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-21T13:27:28.538 DEBUG:teuthology.orchestra.run.smithi097:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph pg dump --format=json 2024-03-21T13:27:28.797 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T13:27:29.947 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:29 smithi097 ceph-mon[39416]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:27:29.947 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:29 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:29.948 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:29 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:29.948 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:29 smithi097 ceph-mon[30040]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:27:29.948 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:29 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:29.948 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:29 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:30.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:29 smithi144 ceph-mon[33633]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:27:30.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:29 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:30.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:29 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:30.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.556+0000 7f4eace7d700 1 -- 172.21.15.97:0/819227728 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4ea8106850 msgr2=0x7f4ea8106c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:30.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.556+0000 7f4eace7d700 1 --2- 172.21.15.97:0/819227728 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4ea8106850 0x7f4ea8106c30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4e980098f0 tx=0x7f4e98009c00 comp rx=0 tx=0).stop 2024-03-21T13:27:30.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.556+0000 7f4eace7d700 1 -- 172.21.15.97:0/819227728 shutdown_connections 2024-03-21T13:27:30.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.556+0000 7f4eace7d700 1 --2- 172.21.15.97:0/819227728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ea8101210 0x7f4ea810d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:30.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.556+0000 7f4eace7d700 1 --2- 172.21.15.97:0/819227728 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4ea8106850 0x7f4ea8106c30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:30.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.556+0000 7f4eace7d700 1 --2- 172.21.15.97:0/819227728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea8100850 0x7f4ea8100cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:30.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.556+0000 7f4eace7d700 1 -- 172.21.15.97:0/819227728 >> 172.21.15.97:0/819227728 conn(0x7f4ea8077690 msgr2=0x7f4ea80feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:27:30.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.556+0000 7f4eace7d700 1 -- 172.21.15.97:0/819227728 shutdown_connections 2024-03-21T13:27:30.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.557+0000 7f4eace7d700 1 -- 172.21.15.97:0/819227728 wait complete. 2024-03-21T13:27:30.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.557+0000 7f4eace7d700 1 Processor -- start 2024-03-21T13:27:30.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.558+0000 7f4eace7d700 1 -- start start 2024-03-21T13:27:30.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.558+0000 7f4eace7d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea8100850 0x7f4ea8198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:30.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.558+0000 7f4eace7d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ea8101210 0x7f4ea8199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:30.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.558+0000 7f4eace7d700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4ea8106850 0x7f4ea81a3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:30.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.558+0000 7f4eace7d700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea81102b0 con 0x7f4ea8100850 2024-03-21T13:27:30.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.558+0000 7f4eace7d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea8110130 con 0x7f4ea8101210 2024-03-21T13:27:30.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.558+0000 7f4eace7d700 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea8110430 con 0x7f4ea8106850 2024-03-21T13:27:30.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.558+0000 7f4ea659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea8100850 0x7f4ea8198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:30.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.558+0000 7f4ea659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea8100850 0x7f4ea8198f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45670/0 (socket says 172.21.15.97:45670) 2024-03-21T13:27:30.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.558+0000 7f4ea659c700 1 -- 172.21.15.97:0/2217202171 learned_addr learned my addr 172.21.15.97:0/2217202171 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:27:30.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.558+0000 7f4ea6d9d700 1 --2- 172.21.15.97:0/2217202171 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4ea8106850 0x7f4ea81a3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:30.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.558+0000 7f4ea5d9b700 1 --2- 172.21.15.97:0/2217202171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ea8101210 0x7f4ea8199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:30.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.558+0000 7f4ea659c700 1 -- 172.21.15.97:0/2217202171 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4ea8106850 msgr2=0x7f4ea81a3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:30.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.558+0000 7f4ea659c700 1 --2- 172.21.15.97:0/2217202171 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4ea8106850 0x7f4ea81a3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:30.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.558+0000 7f4ea659c700 1 -- 172.21.15.97:0/2217202171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ea8101210 msgr2=0x7f4ea8199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:30.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.558+0000 7f4ea659c700 1 --2- 172.21.15.97:0/2217202171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ea8101210 0x7f4ea8199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:30.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.558+0000 7f4ea659c700 1 -- 172.21.15.97:0/2217202171 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea81a44e0 con 0x7f4ea8100850 2024-03-21T13:27:30.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.558+0000 7f4ea5d9b700 1 --2- 172.21.15.97:0/2217202171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ea8101210 0x7f4ea8199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T13:27:30.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.559+0000 7f4ea659c700 1 --2- 172.21.15.97:0/2217202171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea8100850 0x7f4ea8198f20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4e980059d0 tx=0x7f4e9800fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:27:30.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.559+0000 7f4e977fe700 1 -- 172.21.15.97:0/2217202171 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e98027070 con 0x7f4ea8100850 2024-03-21T13:27:30.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.559+0000 7f4eace7d700 1 -- 172.21.15.97:0/2217202171 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ea81a46b0 con 0x7f4ea8100850 2024-03-21T13:27:30.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.559+0000 7f4e977fe700 1 -- 172.21.15.97:0/2217202171 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f4e98005570 con 0x7f4ea8100850 2024-03-21T13:27:30.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.559+0000 7f4e977fe700 1 -- 172.21.15.97:0/2217202171 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e98004930 con 0x7f4ea8100850 2024-03-21T13:27:30.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.559+0000 7f4eace7d700 1 -- 172.21.15.97:0/2217202171 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ea81a4b00 con 0x7f4ea8100850 2024-03-21T13:27:30.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.560+0000 7f4eace7d700 1 -- 172.21.15.97:0/2217202171 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ea8101f90 con 0x7f4ea8100850 2024-03-21T13:27:30.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.560+0000 7f4e977fe700 1 -- 172.21.15.97:0/2217202171 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 22) v1 ==== 92152+0+0 (secure 0 0 0) 0x7f4e9802a070 con 0x7f4ea8100850 2024-03-21T13:27:30.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.561+0000 7f4e977fe700 1 --2- 172.21.15.97:0/2217202171 >> v2:172.21.15.97:6800/12494470 conn(0x7f4e9006e300 0x7f4e900707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:30.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.561+0000 7f4e977fe700 1 -- 172.21.15.97:0/2217202171 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f4e9808ee40 con 0x7f4ea8100850 2024-03-21T13:27:30.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.561+0000 7f4ea5d9b700 1 --2- 172.21.15.97:0/2217202171 >> v2:172.21.15.97:6800/12494470 conn(0x7f4e9006e300 0x7f4e900707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:30.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.561+0000 7f4ea5d9b700 1 --2- 172.21.15.97:0/2217202171 >> v2:172.21.15.97:6800/12494470 conn(0x7f4e9006e300 0x7f4e900707c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c009750 tx=0x7f4e9c009310 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:27:30.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.566+0000 7f4e977fe700 1 -- 172.21.15.97:0/2217202171 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4e98013030 con 0x7f4ea8100850 2024-03-21T13:27:30.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.829+0000 7f4eace7d700 1 -- 172.21.15.97:0/2217202171 --> v2:172.21.15.97:6800/12494470 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4ea80ff1c0 con 0x7f4e9006e300 2024-03-21T13:27:30.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.840+0000 7f4e977fe700 1 -- 172.21.15.97:0/2217202171 <== mgr.24482 v2:172.21.15.97:6800/12494470 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+346469 (secure 0 0 0) 0x7f4ea80ff1c0 con 0x7f4e9006e300 2024-03-21T13:27:30.845 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:27:30.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.843+0000 7f4eace7d700 1 -- 172.21.15.97:0/2217202171 >> v2:172.21.15.97:6800/12494470 conn(0x7f4e9006e300 msgr2=0x7f4e900707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:30.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.843+0000 7f4eace7d700 1 --2- 172.21.15.97:0/2217202171 >> v2:172.21.15.97:6800/12494470 conn(0x7f4e9006e300 0x7f4e900707c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c009750 tx=0x7f4e9c009310 comp rx=0 tx=0).stop 2024-03-21T13:27:30.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.843+0000 7f4eace7d700 1 -- 172.21.15.97:0/2217202171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea8100850 msgr2=0x7f4ea8198f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:30.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.843+0000 7f4eace7d700 1 --2- 172.21.15.97:0/2217202171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea8100850 0x7f4ea8198f20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4e980059d0 tx=0x7f4e9800fa70 comp rx=0 tx=0).stop 2024-03-21T13:27:30.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.843+0000 7f4eace7d700 1 -- 172.21.15.97:0/2217202171 shutdown_connections 2024-03-21T13:27:30.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.843+0000 7f4eace7d700 1 --2- 172.21.15.97:0/2217202171 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4ea8106850 0x7f4ea81a3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:30.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.843+0000 7f4eace7d700 1 --2- 172.21.15.97:0/2217202171 >> v2:172.21.15.97:6800/12494470 conn(0x7f4e9006e300 0x7f4e900707c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:30.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.843+0000 7f4eace7d700 1 --2- 172.21.15.97:0/2217202171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea8100850 0x7f4ea8198f20 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:30.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.843+0000 7f4eace7d700 1 --2- 172.21.15.97:0/2217202171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ea8101210 0x7f4ea8199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:30.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.843+0000 7f4eace7d700 1 -- 172.21.15.97:0/2217202171 >> 172.21.15.97:0/2217202171 conn(0x7f4ea8077690 msgr2=0x7f4ea80feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:27:30.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.843+0000 7f4eace7d700 1 -- 172.21.15.97:0/2217202171 shutdown_connections 2024-03-21T13:27:30.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:30.843+0000 7f4eace7d700 1 -- 172.21.15.97:0/2217202171 wait complete. 2024-03-21T13:27:30.849 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-03-21T13:27:31.175 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:31 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:31.176 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:31 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:31.176 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:31 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:27:31.176 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:31 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:27:31.176 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:31 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:27:31.176 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:31 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:27:31.176 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:31 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:27:31.176 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:31 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:27:31.176 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:31 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:27:31.176 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:31 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:27:31.176 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:31 smithi097 ceph-mon[39416]: Adjusting osd_memory_target on smithi097 to 3760M 2024-03-21T13:27:31.176 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:31 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:31.177 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:31 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:27:31.177 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:31 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:27:31.177 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:31 smithi097 ceph-mon[39416]: Updating smithi097:/etc/ceph/ceph.conf 2024-03-21T13:27:31.177 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:31 smithi097 ceph-mon[39416]: Updating smithi144:/etc/ceph/ceph.conf 2024-03-21T13:27:31.177 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:31 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:31.177 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:31 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:31.177 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:31 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:31.177 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:31 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:31.177 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:31 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:27:31.177 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:31 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:27:31.177 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:31 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:27:31.178 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:31 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:27:31.178 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:31 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:27:31.178 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:31 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:27:31.178 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:31 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:27:31.178 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:31 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:27:31.178 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:31 smithi097 ceph-mon[30040]: Adjusting osd_memory_target on smithi097 to 3760M 2024-03-21T13:27:31.178 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:31 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:31.178 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:31 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:27:31.178 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:31 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:27:31.178 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:31 smithi097 ceph-mon[30040]: Updating smithi097:/etc/ceph/ceph.conf 2024-03-21T13:27:31.178 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:31 smithi097 ceph-mon[30040]: Updating smithi144:/etc/ceph/ceph.conf 2024-03-21T13:27:31.178 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:31 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:31.178 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:31 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:31.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:31 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:31.373 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:31 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:31.373 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:31 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:27:31.373 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:31 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:27:31.373 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:31 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:27:31.373 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:31 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:27:31.374 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:31 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:27:31.374 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:31 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:27:31.374 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:31 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:27:31.374 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:31 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:27:31.374 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:31 smithi144 ceph-mon[33633]: Adjusting osd_memory_target on smithi097 to 3760M 2024-03-21T13:27:31.374 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:31 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:31.374 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:31 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:27:31.374 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:31 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:27:31.374 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:31 smithi144 ceph-mon[33633]: Updating smithi097:/etc/ceph/ceph.conf 2024-03-21T13:27:31.374 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:31 smithi144 ceph-mon[33633]: Updating smithi144:/etc/ceph/ceph.conf 2024-03-21T13:27:31.374 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:31 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:31.374 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:31 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:31.549 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-03-21T13:27:30.286894+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596392,"num_objects":198,"num_object_clones":0,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":198,"num_whiteouts":0,"num_read":1004,"num_read_kb":850,"num_write":615,"num_write_kb":2461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":1771434,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":600,"ondisk_log_size":600,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":248408,"kb_used_data":7292,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":749712808,"statfs":{"total":767960285184,"available":767705915392,"internally_reserved":0,"allocated":7467008,"data_stored":3990672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003377"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.271092+0000","last_change":"2024-03-21T13:26:17.243407+0000","last_active":"2024-03-21T13:26:52.271092+0000","last_peered":"2024-03-21T13:26:52.271092+0000","last_clean":"2024-03-21T13:26:52.271092+0000","last_became_active":"2024-03-21T13:26:17.243093+0000","last_became_peered":"2024-03-21T13:26:17.243093+0000","last_unstale":"2024-03-21T13:26:52.271092+0000","last_undegraded":"2024-03-21T13:26:52.271092+0000","last_fullsized":"2024-03-21T13:26:52.271092+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:28:36.314318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.639746+0000","last_change":"2024-03-21T13:26:15.233162+0000","last_active":"2024-03-21T13:26:52.639746+0000","last_peered":"2024-03-21T13:26:52.639746+0000","last_clean":"2024-03-21T13:26:52.639746+0000","last_became_active":"2024-03-21T13:26:15.232850+0000","last_became_peered":"2024-03-21T13:26:15.232850+0000","last_unstale":"2024-03-21T13:26:52.639746+0000","last_undegraded":"2024-03-21T13:26:52.639746+0000","last_fullsized":"2024-03-21T13:26:52.639746+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:32:57.507907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.640374+0000","last_change":"2024-03-21T13:26:11.195505+0000","last_active":"2024-03-21T13:26:52.640374+0000","last_peered":"2024-03-21T13:26:52.640374+0000","last_clean":"2024-03-21T13:26:52.640374+0000","last_became_active":"2024-03-21T13:26:11.195221+0000","last_became_peered":"2024-03-21T13:26:11.195221+0000","last_unstale":"2024-03-21T13:26:52.640374+0000","last_undegraded":"2024-03-21T13:26:52.640374+0000","last_fullsized":"2024-03-21T13:26:52.640374+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:57:02.841819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"67'9","reported_seq":45,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.270982+0000","last_change":"2024-03-21T13:26:13.221104+0000","last_active":"2024-03-21T13:26:52.270982+0000","last_peered":"2024-03-21T13:26:52.270982+0000","last_clean":"2024-03-21T13:26:52.270982+0000","last_became_active":"2024-03-21T13:26:13.220739+0000","last_became_peered":"2024-03-21T13:26:13.220739+0000","last_unstale":"2024-03-21T13:26:52.270982+0000","last_undegraded":"2024-03-21T13:26:52.270982+0000","last_fullsized":"2024-03-21T13:26:52.270982+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:26:07.984202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.269727+0000","last_change":"2024-03-21T13:26:17.307060+0000","last_active":"2024-03-21T13:26:52.269727+0000","last_peered":"2024-03-21T13:26:52.269727+0000","last_clean":"2024-03-21T13:26:52.269727+0000","last_became_active":"2024-03-21T13:26:17.306235+0000","last_became_peered":"2024-03-21T13:26:17.306235+0000","last_unstale":"2024-03-21T13:26:52.269727+0000","last_undegraded":"2024-03-21T13:26:52.269727+0000","last_fullsized":"2024-03-21T13:26:52.269727+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:15:23.859528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.270980+0000","last_change":"2024-03-21T13:26:15.228250+0000","last_active":"2024-03-21T13:26:52.270980+0000","last_peered":"2024-03-21T13:26:52.270980+0000","last_clean":"2024-03-21T13:26:52.270980+0000","last_became_active":"2024-03-21T13:26:15.227966+0000","last_became_peered":"2024-03-21T13:26:15.227966+0000","last_unstale":"2024-03-21T13:26:52.270980+0000","last_undegraded":"2024-03-21T13:26:52.270980+0000","last_fullsized":"2024-03-21T13:26:52.270980+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:43:04.868113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.269895+0000","last_change":"2024-03-21T13:26:11.198738+0000","last_active":"2024-03-21T13:26:52.269895+0000","last_peered":"2024-03-21T13:26:52.269895+0000","last_clean":"2024-03-21T13:26:52.269895+0000","last_became_active":"2024-03-21T13:26:11.198399+0000","last_became_peered":"2024-03-21T13:26:11.198399+0000","last_unstale":"2024-03-21T13:26:52.269895+0000","last_undegraded":"2024-03-21T13:26:52.269895+0000","last_fullsized":"2024-03-21T13:26:52.269895+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:50:03.374446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"67'15","reported_seq":54,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.269593+0000","last_change":"2024-03-21T13:26:13.220169+0000","last_active":"2024-03-21T13:26:52.269593+0000","last_peered":"2024-03-21T13:26:52.269593+0000","last_clean":"2024-03-21T13:26:52.269593+0000","last_became_active":"2024-03-21T13:26:13.219860+0000","last_became_peered":"2024-03-21T13:26:13.219860+0000","last_unstale":"2024-03-21T13:26:52.269593+0000","last_undegraded":"2024-03-21T13:26:52.269593+0000","last_fullsized":"2024-03-21T13:26:52.269593+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:16:00.508810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"67'1","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.640150+0000","last_change":"2024-03-21T13:26:17.245836+0000","last_active":"2024-03-21T13:26:52.640150+0000","last_peered":"2024-03-21T13:26:52.640150+0000","last_clean":"2024-03-21T13:26:52.640150+0000","last_became_active":"2024-03-21T13:26:17.245515+0000","last_became_peered":"2024-03-21T13:26:17.245515+0000","last_unstale":"2024-03-21T13:26:52.640150+0000","last_undegraded":"2024-03-21T13:26:52.640150+0000","last_fullsized":"2024-03-21T13:26:52.640150+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:06:23.562361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.374715+0000","last_change":"2024-03-21T13:26:15.232911+0000","last_active":"2024-03-21T13:26:52.374715+0000","last_peered":"2024-03-21T13:26:52.374715+0000","last_clean":"2024-03-21T13:26:52.374715+0000","last_became_active":"2024-03-21T13:26:15.232533+0000","last_became_peered":"2024-03-21T13:26:15.232533+0000","last_unstale":"2024-03-21T13:26:52.374715+0000","last_undegraded":"2024-03-21T13:26:52.374715+0000","last_fullsized":"2024-03-21T13:26:52.374715+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:41:14.000034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.374730+0000","last_change":"2024-03-21T13:26:11.200564+0000","last_active":"2024-03-21T13:26:52.374730+0000","last_peered":"2024-03-21T13:26:52.374730+0000","last_clean":"2024-03-21T13:26:52.374730+0000","last_became_active":"2024-03-21T13:26:11.200243+0000","last_became_peered":"2024-03-21T13:26:11.200243+0000","last_unstale":"2024-03-21T13:26:52.374730+0000","last_undegraded":"2024-03-21T13:26:52.374730+0000","last_fullsized":"2024-03-21T13:26:52.374730+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T00:42:33.426721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"67'9","reported_seq":45,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.271361+0000","last_change":"2024-03-21T13:26:13.225317+0000","last_active":"2024-03-21T13:26:52.271361+0000","last_peered":"2024-03-21T13:26:52.271361+0000","last_clean":"2024-03-21T13:26:52.271361+0000","last_became_active":"2024-03-21T13:26:13.225035+0000","last_became_peered":"2024-03-21T13:26:13.225035+0000","last_unstale":"2024-03-21T13:26:52.271361+0000","last_undegraded":"2024-03-21T13:26:52.271361+0000","last_fullsized":"2024-03-21T13:26:52.271361+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:07:04.065673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.374399+0000","last_change":"2024-03-21T13:26:17.249189+0000","last_active":"2024-03-21T13:26:52.374399+0000","last_peered":"2024-03-21T13:26:52.374399+0000","last_clean":"2024-03-21T13:26:52.374399+0000","last_became_active":"2024-03-21T13:26:17.248897+0000","last_became_peered":"2024-03-21T13:26:17.248897+0000","last_unstale":"2024-03-21T13:26:52.374399+0000","last_undegraded":"2024-03-21T13:26:52.374399+0000","last_fullsized":"2024-03-21T13:26:52.374399+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:08:14.907504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":24,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.386060+0000","last_change":"2024-03-21T13:26:15.240073+0000","last_active":"2024-03-21T13:26:52.386060+0000","last_peered":"2024-03-21T13:26:52.386060+0000","last_clean":"2024-03-21T13:26:52.386060+0000","last_became_active":"2024-03-21T13:26:15.239506+0000","last_became_peered":"2024-03-21T13:26:15.239506+0000","last_unstale":"2024-03-21T13:26:52.386060+0000","last_undegraded":"2024-03-21T13:26:52.386060+0000","last_fullsized":"2024-03-21T13:26:52.386060+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:56:55.889404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.374679+0000","last_change":"2024-03-21T13:26:11.199114+0000","last_active":"2024-03-21T13:26:52.374679+0000","last_peered":"2024-03-21T13:26:52.374679+0000","last_clean":"2024-03-21T13:26:52.374679+0000","last_became_active":"2024-03-21T13:26:11.198797+0000","last_became_peered":"2024-03-21T13:26:11.198797+0000","last_unstale":"2024-03-21T13:26:52.374679+0000","last_undegraded":"2024-03-21T13:26:52.374679+0000","last_fullsized":"2024-03-21T13:26:52.374679+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:30:07.486345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"67'5","reported_seq":39,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.272082+0000","last_change":"2024-03-21T13:26:13.224599+0000","last_active":"2024-03-21T13:26:52.272082+0000","last_peered":"2024-03-21T13:26:52.272082+0000","last_clean":"2024-03-21T13:26:52.272082+0000","last_became_active":"2024-03-21T13:26:13.224317+0000","last_became_peered":"2024-03-21T13:26:13.224317+0000","last_unstale":"2024-03-21T13:26:52.272082+0000","last_undegraded":"2024-03-21T13:26:52.272082+0000","last_fullsized":"2024-03-21T13:26:52.272082+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:07:34.516161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.272413+0000","last_change":"2024-03-21T13:26:17.305168+0000","last_active":"2024-03-21T13:26:52.272413+0000","last_peered":"2024-03-21T13:26:52.272413+0000","last_clean":"2024-03-21T13:26:52.272413+0000","last_became_active":"2024-03-21T13:26:17.304862+0000","last_became_peered":"2024-03-21T13:26:17.304862+0000","last_unstale":"2024-03-21T13:26:52.272413+0000","last_undegraded":"2024-03-21T13:26:52.272413+0000","last_fullsized":"2024-03-21T13:26:52.272413+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:56:34.232516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.640538+0000","last_change":"2024-03-21T13:26:15.227623+0000","last_active":"2024-03-21T13:26:52.640538+0000","last_peered":"2024-03-21T13:26:52.640538+0000","last_clean":"2024-03-21T13:26:52.640538+0000","last_became_active":"2024-03-21T13:26:15.227313+0000","last_became_peered":"2024-03-21T13:26:15.227313+0000","last_unstale":"2024-03-21T13:26:52.640538+0000","last_undegraded":"2024-03-21T13:26:52.640538+0000","last_fullsized":"2024-03-21T13:26:52.640538+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:50:33.810070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.641077+0000","last_change":"2024-03-21T13:26:11.201065+0000","last_active":"2024-03-21T13:26:52.641077+0000","last_peered":"2024-03-21T13:26:52.641077+0000","last_clean":"2024-03-21T13:26:52.641077+0000","last_became_active":"2024-03-21T13:26:11.200781+0000","last_became_peered":"2024-03-21T13:26:11.200781+0000","last_unstale":"2024-03-21T13:26:52.641077+0000","last_undegraded":"2024-03-21T13:26:52.641077+0000","last_fullsized":"2024-03-21T13:26:52.641077+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:05:36.701771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"67'15","reported_seq":54,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.382956+0000","last_change":"2024-03-21T13:26:13.224126+0000","last_active":"2024-03-21T13:26:52.382956+0000","last_peered":"2024-03-21T13:26:52.382956+0000","last_clean":"2024-03-21T13:26:52.382956+0000","last_became_active":"2024-03-21T13:26:13.223765+0000","last_became_peered":"2024-03-21T13:26:13.223765+0000","last_unstale":"2024-03-21T13:26:52.382956+0000","last_undegraded":"2024-03-21T13:26:52.382956+0000","last_fullsized":"2024-03-21T13:26:52.382956+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:16:41.903823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.271271+0000","last_change":"2024-03-21T13:26:17.305631+0000","last_active":"2024-03-21T13:26:52.271271+0000","last_peered":"2024-03-21T13:26:52.271271+0000","last_clean":"2024-03-21T13:26:52.271271+0000","last_became_active":"2024-03-21T13:26:17.305358+0000","last_became_peered":"2024-03-21T13:26:17.305358+0000","last_unstale":"2024-03-21T13:26:52.271271+0000","last_undegraded":"2024-03-21T13:26:52.271271+0000","last_fullsized":"2024-03-21T13:26:52.271271+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:20:57.295263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.271808+0000","last_change":"2024-03-21T13:26:15.230491+0000","last_active":"2024-03-21T13:26:52.271808+0000","last_peered":"2024-03-21T13:26:52.271808+0000","last_clean":"2024-03-21T13:26:52.271808+0000","last_became_active":"2024-03-21T13:26:15.229859+0000","last_became_peered":"2024-03-21T13:26:15.229859+0000","last_unstale":"2024-03-21T13:26:52.271808+0000","last_undegraded":"2024-03-21T13:26:52.271808+0000","last_fullsized":"2024-03-21T13:26:52.271808+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T00:26:02.981353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.271526+0000","last_change":"2024-03-21T13:26:11.194612+0000","last_active":"2024-03-21T13:26:52.271526+0000","last_peered":"2024-03-21T13:26:52.271526+0000","last_clean":"2024-03-21T13:26:52.271526+0000","last_became_active":"2024-03-21T13:26:11.194332+0000","last_became_peered":"2024-03-21T13:26:11.194332+0000","last_unstale":"2024-03-21T13:26:52.271526+0000","last_undegraded":"2024-03-21T13:26:52.271526+0000","last_fullsized":"2024-03-21T13:26:52.271526+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:28:26.556731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"67'12","reported_seq":70,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.270788+0000","last_change":"2024-03-21T13:26:32.433066+0000","last_active":"2024-03-21T13:26:52.270788+0000","last_peered":"2024-03-21T13:26:52.270788+0000","last_clean":"2024-03-21T13:26:52.270788+0000","last_became_active":"2024-03-21T13:26:32.417390+0000","last_became_peered":"2024-03-21T13:26:32.417390+0000","last_unstale":"2024-03-21T13:26:52.270788+0000","last_undegraded":"2024-03-21T13:26:52.270788+0000","last_fullsized":"2024-03-21T13:26:52.270788+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:51:55.210341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"67'1","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.386524+0000","last_change":"2024-03-21T13:26:17.252404+0000","last_active":"2024-03-21T13:26:52.386524+0000","last_peered":"2024-03-21T13:26:52.386524+0000","last_clean":"2024-03-21T13:26:52.386524+0000","last_became_active":"2024-03-21T13:26:17.252048+0000","last_became_peered":"2024-03-21T13:26:17.252048+0000","last_unstale":"2024-03-21T13:26:52.386524+0000","last_undegraded":"2024-03-21T13:26:52.386524+0000","last_fullsized":"2024-03-21T13:26:52.386524+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:18:43.223987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":370,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"67'8","reported_seq":52,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:27:21.250680+0000","last_change":"2024-03-21T13:26:15.233894+0000","last_active":"2024-03-21T13:27:21.250680+0000","last_peered":"2024-03-21T13:27:21.250680+0000","last_clean":"2024-03-21T13:27:21.250680+0000","last_became_active":"2024-03-21T13:26:15.233338+0000","last_became_peered":"2024-03-21T13:26:15.233338+0000","last_unstale":"2024-03-21T13:27:21.250680+0000","last_undegraded":"2024-03-21T13:27:21.250680+0000","last_fullsized":"2024-03-21T13:27:21.250680+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:46:08.680052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"60'1","reported_seq":34,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.639844+0000","last_change":"2024-03-21T13:26:11.196283+0000","last_active":"2024-03-21T13:26:52.639844+0000","last_peered":"2024-03-21T13:26:52.639844+0000","last_clean":"2024-03-21T13:26:52.639844+0000","last_became_active":"2024-03-21T13:26:11.195957+0000","last_became_peered":"2024-03-21T13:26:11.195957+0000","last_unstale":"2024-03-21T13:26:52.639844+0000","last_undegraded":"2024-03-21T13:26:52.639844+0000","last_fullsized":"2024-03-21T13:26:52.639844+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:22:55.172360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"67'10","reported_seq":44,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.640943+0000","last_change":"2024-03-21T13:26:13.219274+0000","last_active":"2024-03-21T13:26:52.640943+0000","last_peered":"2024-03-21T13:26:52.640943+0000","last_clean":"2024-03-21T13:26:52.640943+0000","last_became_active":"2024-03-21T13:26:13.218535+0000","last_became_peered":"2024-03-21T13:26:13.218535+0000","last_unstale":"2024-03-21T13:26:52.640943+0000","last_undegraded":"2024-03-21T13:26:52.640943+0000","last_fullsized":"2024-03-21T13:26:52.640943+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:20:31.477672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.641582+0000","last_change":"2024-03-21T13:26:17.244005+0000","last_active":"2024-03-21T13:26:52.641582+0000","last_peered":"2024-03-21T13:26:52.641582+0000","last_clean":"2024-03-21T13:26:52.641582+0000","last_became_active":"2024-03-21T13:26:17.243691+0000","last_became_peered":"2024-03-21T13:26:17.243691+0000","last_unstale":"2024-03-21T13:26:52.641582+0000","last_undegraded":"2024-03-21T13:26:52.641582+0000","last_fullsized":"2024-03-21T13:26:52.641582+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T00:49:51.167480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.641132+0000","last_change":"2024-03-21T13:26:15.227555+0000","last_active":"2024-03-21T13:26:52.641132+0000","last_peered":"2024-03-21T13:26:52.641132+0000","last_clean":"2024-03-21T13:26:52.641132+0000","last_became_active":"2024-03-21T13:26:15.227222+0000","last_became_peered":"2024-03-21T13:26:15.227222+0000","last_unstale":"2024-03-21T13:26:52.641132+0000","last_undegraded":"2024-03-21T13:26:52.641132+0000","last_fullsized":"2024-03-21T13:26:52.641132+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:29:30.251861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.269051+0000","last_change":"2024-03-21T13:26:11.196773+0000","last_active":"2024-03-21T13:26:52.269051+0000","last_peered":"2024-03-21T13:26:52.269051+0000","last_clean":"2024-03-21T13:26:52.269051+0000","last_became_active":"2024-03-21T13:26:11.196214+0000","last_became_peered":"2024-03-21T13:26:11.196214+0000","last_unstale":"2024-03-21T13:26:52.269051+0000","last_undegraded":"2024-03-21T13:26:52.269051+0000","last_fullsized":"2024-03-21T13:26:52.269051+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T00:43:01.860805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"67'11","reported_seq":48,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.547554+0000","last_change":"2024-03-21T13:26:13.223859+0000","last_active":"2024-03-21T13:26:52.547554+0000","last_peered":"2024-03-21T13:26:52.547554+0000","last_clean":"2024-03-21T13:26:52.547554+0000","last_became_active":"2024-03-21T13:26:13.223564+0000","last_became_peered":"2024-03-21T13:26:13.223564+0000","last_unstale":"2024-03-21T13:26:52.547554+0000","last_undegraded":"2024-03-21T13:26:52.547554+0000","last_fullsized":"2024-03-21T13:26:52.547554+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:35:22.228078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":20,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.386032+0000","last_change":"2024-03-21T13:26:17.252364+0000","last_active":"2024-03-21T13:26:52.386032+0000","last_peered":"2024-03-21T13:26:52.386032+0000","last_clean":"2024-03-21T13:26:52.386032+0000","last_became_active":"2024-03-21T13:26:17.252073+0000","last_became_peered":"2024-03-21T13:26:17.252073+0000","last_unstale":"2024-03-21T13:26:52.386032+0000","last_undegraded":"2024-03-21T13:26:52.386032+0000","last_fullsized":"2024-03-21T13:26:52.386032+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T01:24:36.811892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.546922+0000","last_change":"2024-03-21T13:26:15.232697+0000","last_active":"2024-03-21T13:26:52.546922+0000","last_peered":"2024-03-21T13:26:52.546922+0000","last_clean":"2024-03-21T13:26:52.546922+0000","last_became_active":"2024-03-21T13:26:15.232208+0000","last_became_peered":"2024-03-21T13:26:15.232208+0000","last_unstale":"2024-03-21T13:26:52.546922+0000","last_undegraded":"2024-03-21T13:26:52.546922+0000","last_fullsized":"2024-03-21T13:26:52.546922+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:18:56.080086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.640282+0000","last_change":"2024-03-21T13:26:11.200672+0000","last_active":"2024-03-21T13:26:52.640282+0000","last_peered":"2024-03-21T13:26:52.640282+0000","last_clean":"2024-03-21T13:26:52.640282+0000","last_became_active":"2024-03-21T13:26:11.200402+0000","last_became_peered":"2024-03-21T13:26:11.200402+0000","last_unstale":"2024-03-21T13:26:52.640282+0000","last_undegraded":"2024-03-21T13:26:52.640282+0000","last_fullsized":"2024-03-21T13:26:52.640282+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:19:56.865749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"67'4","reported_seq":35,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.269084+0000","last_change":"2024-03-21T13:26:13.223883+0000","last_active":"2024-03-21T13:26:52.269084+0000","last_peered":"2024-03-21T13:26:52.269084+0000","last_clean":"2024-03-21T13:26:52.269084+0000","last_became_active":"2024-03-21T13:26:13.223548+0000","last_became_peered":"2024-03-21T13:26:13.223548+0000","last_unstale":"2024-03-21T13:26:52.269084+0000","last_undegraded":"2024-03-21T13:26:52.269084+0000","last_fullsized":"2024-03-21T13:26:52.269084+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:05:30.889217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.269509+0000","last_change":"2024-03-21T13:26:17.307158+0000","last_active":"2024-03-21T13:26:52.269509+0000","last_peered":"2024-03-21T13:26:52.269509+0000","last_clean":"2024-03-21T13:26:52.269509+0000","last_became_active":"2024-03-21T13:26:17.306160+0000","last_became_peered":"2024-03-21T13:26:17.306160+0000","last_unstale":"2024-03-21T13:26:52.269509+0000","last_undegraded":"2024-03-21T13:26:52.269509+0000","last_fullsized":"2024-03-21T13:26:52.269509+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T00:17:20.173596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":24,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.386303+0000","last_change":"2024-03-21T13:26:15.233113+0000","last_active":"2024-03-21T13:26:52.386303+0000","last_peered":"2024-03-21T13:26:52.386303+0000","last_clean":"2024-03-21T13:26:52.386303+0000","last_became_active":"2024-03-21T13:26:15.232761+0000","last_became_peered":"2024-03-21T13:26:15.232761+0000","last_unstale":"2024-03-21T13:26:52.386303+0000","last_undegraded":"2024-03-21T13:26:52.386303+0000","last_fullsized":"2024-03-21T13:26:52.386303+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:40:41.443170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.374978+0000","last_change":"2024-03-21T13:26:11.197392+0000","last_active":"2024-03-21T13:26:52.374978+0000","last_peered":"2024-03-21T13:26:52.374978+0000","last_clean":"2024-03-21T13:26:52.374978+0000","last_became_active":"2024-03-21T13:26:11.197083+0000","last_became_peered":"2024-03-21T13:26:11.197083+0000","last_unstale":"2024-03-21T13:26:52.374978+0000","last_undegraded":"2024-03-21T13:26:52.374978+0000","last_fullsized":"2024-03-21T13:26:52.374978+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:15:23.542845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"67'11","reported_seq":48,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.269550+0000","last_change":"2024-03-21T13:26:13.221030+0000","last_active":"2024-03-21T13:26:52.269550+0000","last_peered":"2024-03-21T13:26:52.269550+0000","last_clean":"2024-03-21T13:26:52.269550+0000","last_became_active":"2024-03-21T13:26:13.220710+0000","last_became_peered":"2024-03-21T13:26:13.220710+0000","last_unstale":"2024-03-21T13:26:52.269550+0000","last_undegraded":"2024-03-21T13:26:52.269550+0000","last_fullsized":"2024-03-21T13:26:52.269550+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:40:46.842543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.639195+0000","last_change":"2024-03-21T13:26:17.245884+0000","last_active":"2024-03-21T13:26:52.639195+0000","last_peered":"2024-03-21T13:26:52.639195+0000","last_clean":"2024-03-21T13:26:52.639195+0000","last_became_active":"2024-03-21T13:26:17.245564+0000","last_became_peered":"2024-03-21T13:26:17.245564+0000","last_unstale":"2024-03-21T13:26:52.639195+0000","last_undegraded":"2024-03-21T13:26:52.639195+0000","last_fullsized":"2024-03-21T13:26:52.639195+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:26:57.034878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.269828+0000","last_change":"2024-03-21T13:26:15.229227+0000","last_active":"2024-03-21T13:26:52.269828+0000","last_peered":"2024-03-21T13:26:52.269828+0000","last_clean":"2024-03-21T13:26:52.269828+0000","last_became_active":"2024-03-21T13:26:15.228860+0000","last_became_peered":"2024-03-21T13:26:15.228860+0000","last_unstale":"2024-03-21T13:26:52.269828+0000","last_undegraded":"2024-03-21T13:26:52.269828+0000","last_fullsized":"2024-03-21T13:26:52.269828+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:44:39.938958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":32,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.386521+0000","last_change":"2024-03-21T13:26:11.202322+0000","last_active":"2024-03-21T13:26:52.386521+0000","last_peered":"2024-03-21T13:26:52.386521+0000","last_clean":"2024-03-21T13:26:52.386521+0000","last_became_active":"2024-03-21T13:26:11.202018+0000","last_became_peered":"2024-03-21T13:26:11.202018+0000","last_unstale":"2024-03-21T13:26:52.386521+0000","last_undegraded":"2024-03-21T13:26:52.386521+0000","last_fullsized":"2024-03-21T13:26:52.386521+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:47:07.061477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"67'9","reported_seq":45,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.641698+0000","last_change":"2024-03-21T13:26:13.223870+0000","last_active":"2024-03-21T13:26:52.641698+0000","last_peered":"2024-03-21T13:26:52.641698+0000","last_clean":"2024-03-21T13:26:52.641698+0000","last_became_active":"2024-03-21T13:26:13.223536+0000","last_became_peered":"2024-03-21T13:26:13.223536+0000","last_unstale":"2024-03-21T13:26:52.641698+0000","last_undegraded":"2024-03-21T13:26:52.641698+0000","last_fullsized":"2024-03-21T13:26:52.641698+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:00:43.711112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.269200+0000","last_change":"2024-03-21T13:26:17.247820+0000","last_active":"2024-03-21T13:26:52.269200+0000","last_peered":"2024-03-21T13:26:52.269200+0000","last_clean":"2024-03-21T13:26:52.269200+0000","last_became_active":"2024-03-21T13:26:17.247405+0000","last_became_peered":"2024-03-21T13:26:17.247405+0000","last_unstale":"2024-03-21T13:26:52.269200+0000","last_undegraded":"2024-03-21T13:26:52.269200+0000","last_fullsized":"2024-03-21T13:26:52.269200+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:43:44.382313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.640084+0000","last_change":"2024-03-21T13:26:15.234664+0000","last_active":"2024-03-21T13:26:52.640084+0000","last_peered":"2024-03-21T13:26:52.640084+0000","last_clean":"2024-03-21T13:26:52.640084+0000","last_became_active":"2024-03-21T13:26:15.234007+0000","last_became_peered":"2024-03-21T13:26:15.234007+0000","last_unstale":"2024-03-21T13:26:52.640084+0000","last_undegraded":"2024-03-21T13:26:52.640084+0000","last_fullsized":"2024-03-21T13:26:52.640084+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:54:30.608977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.272102+0000","last_change":"2024-03-21T13:26:11.198059+0000","last_active":"2024-03-21T13:26:52.272102+0000","last_peered":"2024-03-21T13:26:52.272102+0000","last_clean":"2024-03-21T13:26:52.272102+0000","last_became_active":"2024-03-21T13:26:11.197786+0000","last_became_peered":"2024-03-21T13:26:11.197786+0000","last_unstale":"2024-03-21T13:26:52.272102+0000","last_undegraded":"2024-03-21T13:26:52.272102+0000","last_fullsized":"2024-03-21T13:26:52.272102+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T00:27:48.584970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"67'11","reported_seq":48,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.271901+0000","last_change":"2024-03-21T13:26:13.223122+0000","last_active":"2024-03-21T13:26:52.271901+0000","last_peered":"2024-03-21T13:26:52.271901+0000","last_clean":"2024-03-21T13:26:52.271901+0000","last_became_active":"2024-03-21T13:26:13.222859+0000","last_became_peered":"2024-03-21T13:26:13.222859+0000","last_unstale":"2024-03-21T13:26:52.271901+0000","last_undegraded":"2024-03-21T13:26:52.271901+0000","last_fullsized":"2024-03-21T13:26:52.271901+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:27:49.056642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.640703+0000","last_change":"2024-03-21T13:26:17.249015+0000","last_active":"2024-03-21T13:26:52.640703+0000","last_peered":"2024-03-21T13:26:52.640703+0000","last_clean":"2024-03-21T13:26:52.640703+0000","last_became_active":"2024-03-21T13:26:17.248654+0000","last_became_peered":"2024-03-21T13:26:17.248654+0000","last_unstale":"2024-03-21T13:26:52.640703+0000","last_undegraded":"2024-03-21T13:26:52.640703+0000","last_fullsized":"2024-03-21T13:26:52.640703+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:09:06.584717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"67'8","reported_seq":49,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:27:21.250631+0000","last_change":"2024-03-21T13:26:15.233021+0000","last_active":"2024-03-21T13:27:21.250631+0000","last_peered":"2024-03-21T13:27:21.250631+0000","last_clean":"2024-03-21T13:27:21.250631+0000","last_became_active":"2024-03-21T13:26:15.232194+0000","last_became_peered":"2024-03-21T13:26:15.232194+0000","last_unstale":"2024-03-21T13:27:21.250631+0000","last_undegraded":"2024-03-21T13:27:21.250631+0000","last_fullsized":"2024-03-21T13:27:21.250631+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T00:44:08.125960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":32,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.386105+0000","last_change":"2024-03-21T13:26:11.196875+0000","last_active":"2024-03-21T13:26:52.386105+0000","last_peered":"2024-03-21T13:26:52.386105+0000","last_clean":"2024-03-21T13:26:52.386105+0000","last_became_active":"2024-03-21T13:26:11.196574+0000","last_became_peered":"2024-03-21T13:26:11.196574+0000","last_unstale":"2024-03-21T13:26:52.386105+0000","last_undegraded":"2024-03-21T13:26:52.386105+0000","last_fullsized":"2024-03-21T13:26:52.386105+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T01:09:34.867011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"67'10","reported_seq":44,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.269800+0000","last_change":"2024-03-21T13:26:13.228578+0000","last_active":"2024-03-21T13:26:52.269800+0000","last_peered":"2024-03-21T13:26:52.269800+0000","last_clean":"2024-03-21T13:26:52.269800+0000","last_became_active":"2024-03-21T13:26:13.227999+0000","last_became_peered":"2024-03-21T13:26:13.227999+0000","last_unstale":"2024-03-21T13:26:52.269800+0000","last_undegraded":"2024-03-21T13:26:52.269800+0000","last_fullsized":"2024-03-21T13:26:52.269800+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:09:24.782441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.272590+0000","last_change":"2024-03-21T13:26:17.248927+0000","last_active":"2024-03-21T13:26:52.272590+0000","last_peered":"2024-03-21T13:26:52.272590+0000","last_clean":"2024-03-21T13:26:52.272590+0000","last_became_active":"2024-03-21T13:26:17.248348+0000","last_became_peered":"2024-03-21T13:26:17.248348+0000","last_unstale":"2024-03-21T13:26:52.272590+0000","last_undegraded":"2024-03-21T13:26:52.272590+0000","last_fullsized":"2024-03-21T13:26:52.272590+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:35:45.907943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"67'8","reported_seq":49,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:27:21.250587+0000","last_change":"2024-03-21T13:26:15.237341+0000","last_active":"2024-03-21T13:27:21.250587+0000","last_peered":"2024-03-21T13:27:21.250587+0000","last_clean":"2024-03-21T13:27:21.250587+0000","last_became_active":"2024-03-21T13:26:15.237025+0000","last_became_peered":"2024-03-21T13:26:15.237025+0000","last_unstale":"2024-03-21T13:27:21.250587+0000","last_undegraded":"2024-03-21T13:27:21.250587+0000","last_fullsized":"2024-03-21T13:27:21.250587+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:09:19.571483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.640598+0000","last_change":"2024-03-21T13:26:11.196835+0000","last_active":"2024-03-21T13:26:52.640598+0000","last_peered":"2024-03-21T13:26:52.640598+0000","last_clean":"2024-03-21T13:26:52.640598+0000","last_became_active":"2024-03-21T13:26:11.196462+0000","last_became_peered":"2024-03-21T13:26:11.196462+0000","last_unstale":"2024-03-21T13:26:52.640598+0000","last_undegraded":"2024-03-21T13:26:52.640598+0000","last_fullsized":"2024-03-21T13:26:52.640598+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:08:34.618745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"67'9","reported_seq":45,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.375063+0000","last_change":"2024-03-21T13:26:13.220137+0000","last_active":"2024-03-21T13:26:52.375063+0000","last_peered":"2024-03-21T13:26:52.375063+0000","last_clean":"2024-03-21T13:26:52.375063+0000","last_became_active":"2024-03-21T13:26:13.219824+0000","last_became_peered":"2024-03-21T13:26:13.219824+0000","last_unstale":"2024-03-21T13:26:52.375063+0000","last_undegraded":"2024-03-21T13:26:52.375063+0000","last_fullsized":"2024-03-21T13:26:52.375063+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T00:46:53.170532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.383131+0000","last_change":"2024-03-21T13:26:17.252431+0000","last_active":"2024-03-21T13:26:52.383131+0000","last_peered":"2024-03-21T13:26:52.383131+0000","last_clean":"2024-03-21T13:26:52.383131+0000","last_became_active":"2024-03-21T13:26:17.252059+0000","last_became_peered":"2024-03-21T13:26:17.252059+0000","last_unstale":"2024-03-21T13:26:52.383131+0000","last_undegraded":"2024-03-21T13:26:52.383131+0000","last_fullsized":"2024-03-21T13:26:52.383131+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:23:21.459141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.271185+0000","last_change":"2024-03-21T13:26:15.229589+0000","last_active":"2024-03-21T13:26:52.271185+0000","last_peered":"2024-03-21T13:26:52.271185+0000","last_clean":"2024-03-21T13:26:52.271185+0000","last_became_active":"2024-03-21T13:26:15.229128+0000","last_became_peered":"2024-03-21T13:26:15.229128+0000","last_unstale":"2024-03-21T13:26:52.271185+0000","last_undegraded":"2024-03-21T13:26:52.271185+0000","last_fullsized":"2024-03-21T13:26:52.271185+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:39:29.636968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":32,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.386377+0000","last_change":"2024-03-21T13:26:11.202357+0000","last_active":"2024-03-21T13:26:52.386377+0000","last_peered":"2024-03-21T13:26:52.386377+0000","last_clean":"2024-03-21T13:26:52.386377+0000","last_became_active":"2024-03-21T13:26:11.201668+0000","last_became_peered":"2024-03-21T13:26:11.201668+0000","last_unstale":"2024-03-21T13:26:52.386377+0000","last_undegraded":"2024-03-21T13:26:52.386377+0000","last_fullsized":"2024-03-21T13:26:52.386377+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:59:59.977918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"67'9","reported_seq":45,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.640996+0000","last_change":"2024-03-21T13:26:13.218912+0000","last_active":"2024-03-21T13:26:52.640996+0000","last_peered":"2024-03-21T13:26:52.640996+0000","last_clean":"2024-03-21T13:26:52.640996+0000","last_became_active":"2024-03-21T13:26:13.218090+0000","last_became_peered":"2024-03-21T13:26:13.218090+0000","last_unstale":"2024-03-21T13:26:52.640996+0000","last_undegraded":"2024-03-21T13:26:52.640996+0000","last_fullsized":"2024-03-21T13:26:52.640996+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:51:49.321539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":20,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.386476+0000","last_change":"2024-03-21T13:26:17.306715+0000","last_active":"2024-03-21T13:26:52.386476+0000","last_peered":"2024-03-21T13:26:52.386476+0000","last_clean":"2024-03-21T13:26:52.386476+0000","last_became_active":"2024-03-21T13:26:17.306408+0000","last_became_peered":"2024-03-21T13:26:17.306408+0000","last_unstale":"2024-03-21T13:26:52.386476+0000","last_undegraded":"2024-03-21T13:26:52.386476+0000","last_fullsized":"2024-03-21T13:26:52.386476+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:00:05.045057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.375008+0000","last_change":"2024-03-21T13:26:15.233948+0000","last_active":"2024-03-21T13:26:52.375008+0000","last_peered":"2024-03-21T13:26:52.375008+0000","last_clean":"2024-03-21T13:26:52.375008+0000","last_became_active":"2024-03-21T13:26:15.233657+0000","last_became_peered":"2024-03-21T13:26:15.233657+0000","last_unstale":"2024-03-21T13:26:52.375008+0000","last_undegraded":"2024-03-21T13:26:52.375008+0000","last_fullsized":"2024-03-21T13:26:52.375008+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:14:27.120444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.546875+0000","last_change":"2024-03-21T13:26:11.195449+0000","last_active":"2024-03-21T13:26:52.546875+0000","last_peered":"2024-03-21T13:26:52.546875+0000","last_clean":"2024-03-21T13:26:52.546875+0000","last_became_active":"2024-03-21T13:26:11.195164+0000","last_became_peered":"2024-03-21T13:26:11.195164+0000","last_unstale":"2024-03-21T13:26:52.546875+0000","last_undegraded":"2024-03-21T13:26:52.546875+0000","last_fullsized":"2024-03-21T13:26:52.546875+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:58:29.196507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"67'6","reported_seq":38,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.271359+0000","last_change":"2024-03-21T13:26:13.228326+0000","last_active":"2024-03-21T13:26:52.271359+0000","last_peered":"2024-03-21T13:26:52.271359+0000","last_clean":"2024-03-21T13:26:52.271359+0000","last_became_active":"2024-03-21T13:26:13.228006+0000","last_became_peered":"2024-03-21T13:26:13.228006+0000","last_unstale":"2024-03-21T13:26:52.271359+0000","last_undegraded":"2024-03-21T13:26:52.271359+0000","last_fullsized":"2024-03-21T13:26:52.271359+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:08:26.515863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.374690+0000","last_change":"2024-03-21T13:26:17.305291+0000","last_active":"2024-03-21T13:26:52.374690+0000","last_peered":"2024-03-21T13:26:52.374690+0000","last_clean":"2024-03-21T13:26:52.374690+0000","last_became_active":"2024-03-21T13:26:17.304979+0000","last_became_peered":"2024-03-21T13:26:17.304979+0000","last_unstale":"2024-03-21T13:26:52.374690+0000","last_undegraded":"2024-03-21T13:26:52.374690+0000","last_fullsized":"2024-03-21T13:26:52.374690+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:32:09.256312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"60'2","reported_seq":44,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.386865+0000","last_change":"2024-03-21T13:26:11.203158+0000","last_active":"2024-03-21T13:26:52.386865+0000","last_peered":"2024-03-21T13:26:52.386865+0000","last_clean":"2024-03-21T13:26:52.386865+0000","last_became_active":"2024-03-21T13:26:11.201706+0000","last_became_peered":"2024-03-21T13:26:11.201706+0000","last_unstale":"2024-03-21T13:26:52.386865+0000","last_undegraded":"2024-03-21T13:26:52.386865+0000","last_fullsized":"2024-03-21T13:26:52.386865+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:29:51.958157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"67'15","reported_seq":54,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.374431+0000","last_change":"2024-03-21T13:26:13.216022+0000","last_active":"2024-03-21T13:26:52.374431+0000","last_peered":"2024-03-21T13:26:52.374431+0000","last_clean":"2024-03-21T13:26:52.374431+0000","last_became_active":"2024-03-21T13:26:13.215749+0000","last_became_peered":"2024-03-21T13:26:13.215749+0000","last_unstale":"2024-03-21T13:26:52.374431+0000","last_undegraded":"2024-03-21T13:26:52.374431+0000","last_fullsized":"2024-03-21T13:26:52.374431+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:30:43.191153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"67'8","reported_seq":51,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:27:21.250710+0000","last_change":"2024-03-21T13:26:15.237504+0000","last_active":"2024-03-21T13:27:21.250710+0000","last_peered":"2024-03-21T13:27:21.250710+0000","last_clean":"2024-03-21T13:27:21.250710+0000","last_became_active":"2024-03-21T13:26:15.237182+0000","last_became_peered":"2024-03-21T13:26:15.237182+0000","last_unstale":"2024-03-21T13:27:21.250710+0000","last_undegraded":"2024-03-21T13:27:21.250710+0000","last_fullsized":"2024-03-21T13:27:21.250710+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T00:09:41.375653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.269616+0000","last_change":"2024-03-21T13:26:17.247720+0000","last_active":"2024-03-21T13:26:52.269616+0000","last_peered":"2024-03-21T13:26:52.269616+0000","last_clean":"2024-03-21T13:26:52.269616+0000","last_became_active":"2024-03-21T13:26:17.247360+0000","last_became_peered":"2024-03-21T13:26:17.247360+0000","last_unstale":"2024-03-21T13:26:52.269616+0000","last_undegraded":"2024-03-21T13:26:52.269616+0000","last_fullsized":"2024-03-21T13:26:52.269616+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:26:39.004847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":32,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.386880+0000","last_change":"2024-03-21T13:26:11.201159+0000","last_active":"2024-03-21T13:26:52.386880+0000","last_peered":"2024-03-21T13:26:52.386880+0000","last_clean":"2024-03-21T13:26:52.386880+0000","last_became_active":"2024-03-21T13:26:11.200889+0000","last_became_peered":"2024-03-21T13:26:11.200889+0000","last_unstale":"2024-03-21T13:26:52.386880+0000","last_undegraded":"2024-03-21T13:26:52.386880+0000","last_fullsized":"2024-03-21T13:26:52.386880+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:13:54.023383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"67'12","reported_seq":52,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.271287+0000","last_change":"2024-03-21T13:26:13.225296+0000","last_active":"2024-03-21T13:26:52.271287+0000","last_peered":"2024-03-21T13:26:52.271287+0000","last_clean":"2024-03-21T13:26:52.271287+0000","last_became_active":"2024-03-21T13:26:13.225038+0000","last_became_peered":"2024-03-21T13:26:13.225038+0000","last_unstale":"2024-03-21T13:26:52.271287+0000","last_undegraded":"2024-03-21T13:26:52.271287+0000","last_fullsized":"2024-03-21T13:26:52.271287+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:19:32.745497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.382321+0000","last_change":"2024-03-21T13:26:15.225181+0000","last_active":"2024-03-21T13:26:52.382321+0000","last_peered":"2024-03-21T13:26:52.382321+0000","last_clean":"2024-03-21T13:26:52.382321+0000","last_became_active":"2024-03-21T13:26:15.224882+0000","last_became_peered":"2024-03-21T13:26:15.224882+0000","last_unstale":"2024-03-21T13:26:52.382321+0000","last_undegraded":"2024-03-21T13:26:52.382321+0000","last_fullsized":"2024-03-21T13:26:52.382321+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:57:44.392944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":20,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.385885+0000","last_change":"2024-03-21T13:26:17.252287+0000","last_active":"2024-03-21T13:26:52.385885+0000","last_peered":"2024-03-21T13:26:52.385885+0000","last_clean":"2024-03-21T13:26:52.385885+0000","last_became_active":"2024-03-21T13:26:17.251966+0000","last_became_peered":"2024-03-21T13:26:17.251966+0000","last_unstale":"2024-03-21T13:26:52.385885+0000","last_undegraded":"2024-03-21T13:26:52.385885+0000","last_fullsized":"2024-03-21T13:26:52.385885+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T00:09:54.589933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.382798+0000","last_change":"2024-03-21T13:26:15.232352+0000","last_active":"2024-03-21T13:26:52.382798+0000","last_peered":"2024-03-21T13:26:52.382798+0000","last_clean":"2024-03-21T13:26:52.382798+0000","last_became_active":"2024-03-21T13:26:15.232036+0000","last_became_peered":"2024-03-21T13:26:15.232036+0000","last_unstale":"2024-03-21T13:26:52.382798+0000","last_undegraded":"2024-03-21T13:26:52.382798+0000","last_fullsized":"2024-03-21T13:26:52.382798+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:37:51.494333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":32,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.386740+0000","last_change":"2024-03-21T13:26:11.197526+0000","last_active":"2024-03-21T13:26:52.386740+0000","last_peered":"2024-03-21T13:26:52.386740+0000","last_clean":"2024-03-21T13:26:52.386740+0000","last_became_active":"2024-03-21T13:26:11.196727+0000","last_became_peered":"2024-03-21T13:26:11.196727+0000","last_unstale":"2024-03-21T13:26:52.386740+0000","last_undegraded":"2024-03-21T13:26:52.386740+0000","last_fullsized":"2024-03-21T13:26:52.386740+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:03:19.084737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"67'19","reported_seq":60,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.547520+0000","last_change":"2024-03-21T13:26:13.221628+0000","last_active":"2024-03-21T13:26:52.547520+0000","last_peered":"2024-03-21T13:26:52.547520+0000","last_clean":"2024-03-21T13:26:52.547520+0000","last_became_active":"2024-03-21T13:26:13.221351+0000","last_became_peered":"2024-03-21T13:26:13.221351+0000","last_unstale":"2024-03-21T13:26:52.547520+0000","last_undegraded":"2024-03-21T13:26:52.547520+0000","last_fullsized":"2024-03-21T13:26:52.547520+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:45:00.519834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.639298+0000","last_change":"2024-03-21T13:26:17.245249+0000","last_active":"2024-03-21T13:26:52.639298+0000","last_peered":"2024-03-21T13:26:52.639298+0000","last_clean":"2024-03-21T13:26:52.639298+0000","last_became_active":"2024-03-21T13:26:17.244930+0000","last_became_peered":"2024-03-21T13:26:17.244930+0000","last_unstale":"2024-03-21T13:26:52.639298+0000","last_undegraded":"2024-03-21T13:26:52.639298+0000","last_fullsized":"2024-03-21T13:26:52.639298+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:55:39.894721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.382540+0000","last_change":"2024-03-21T13:26:15.224697+0000","last_active":"2024-03-21T13:26:52.382540+0000","last_peered":"2024-03-21T13:26:52.382540+0000","last_clean":"2024-03-21T13:26:52.382540+0000","last_became_active":"2024-03-21T13:26:15.224322+0000","last_became_peered":"2024-03-21T13:26:15.224322+0000","last_unstale":"2024-03-21T13:26:52.382540+0000","last_undegraded":"2024-03-21T13:26:52.382540+0000","last_fullsized":"2024-03-21T13:26:52.382540+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T00:20:38.425097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.374725+0000","last_change":"2024-03-21T13:26:11.195530+0000","last_active":"2024-03-21T13:26:52.374725+0000","last_peered":"2024-03-21T13:26:52.374725+0000","last_clean":"2024-03-21T13:26:52.374725+0000","last_became_active":"2024-03-21T13:26:11.195248+0000","last_became_peered":"2024-03-21T13:26:11.195248+0000","last_unstale":"2024-03-21T13:26:52.374725+0000","last_undegraded":"2024-03-21T13:26:52.374725+0000","last_fullsized":"2024-03-21T13:26:52.374725+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:07:13.257027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"67'9","reported_seq":45,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.641214+0000","last_change":"2024-03-21T13:26:13.221547+0000","last_active":"2024-03-21T13:26:52.641214+0000","last_peered":"2024-03-21T13:26:52.641214+0000","last_clean":"2024-03-21T13:26:52.641214+0000","last_became_active":"2024-03-21T13:26:13.220184+0000","last_became_peered":"2024-03-21T13:26:13.220184+0000","last_unstale":"2024-03-21T13:26:52.641214+0000","last_undegraded":"2024-03-21T13:26:52.641214+0000","last_fullsized":"2024-03-21T13:26:52.641214+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:19:30.788571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.269682+0000","last_change":"2024-03-21T13:26:17.247824+0000","last_active":"2024-03-21T13:26:52.269682+0000","last_peered":"2024-03-21T13:26:52.269682+0000","last_clean":"2024-03-21T13:26:52.269682+0000","last_became_active":"2024-03-21T13:26:17.247506+0000","last_became_peered":"2024-03-21T13:26:17.247506+0000","last_unstale":"2024-03-21T13:26:52.269682+0000","last_undegraded":"2024-03-21T13:26:52.269682+0000","last_fullsized":"2024-03-21T13:26:52.269682+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T00:56:55.828142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"64'2","reported_seq":42,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.374516+0000","last_change":"2024-03-21T13:26:15.231433+0000","last_active":"2024-03-21T13:26:52.374516+0000","last_peered":"2024-03-21T13:26:52.374516+0000","last_clean":"2024-03-21T13:26:52.374516+0000","last_became_active":"2024-03-21T13:26:11.197652+0000","last_became_peered":"2024-03-21T13:26:11.197652+0000","last_unstale":"2024-03-21T13:26:52.374516+0000","last_undegraded":"2024-03-21T13:26:52.374516+0000","last_fullsized":"2024-03-21T13:26:52.374516+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T01:01:02.629415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004041551,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.271684+0000","last_change":"2024-03-21T13:26:11.199491+0000","last_active":"2024-03-21T13:26:52.271684+0000","last_peered":"2024-03-21T13:26:52.271684+0000","last_clean":"2024-03-21T13:26:52.271684+0000","last_became_active":"2024-03-21T13:26:11.199109+0000","last_became_peered":"2024-03-21T13:26:11.199109+0000","last_unstale":"2024-03-21T13:26:52.271684+0000","last_undegraded":"2024-03-21T13:26:52.271684+0000","last_fullsized":"2024-03-21T13:26:52.271684+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:50:27.710077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"67'28","reported_seq":93,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:27:21.250540+0000","last_change":"2024-03-21T13:26:13.224310+0000","last_active":"2024-03-21T13:27:21.250540+0000","last_peered":"2024-03-21T13:27:21.250540+0000","last_clean":"2024-03-21T13:27:21.250540+0000","last_became_active":"2024-03-21T13:26:13.223957+0000","last_became_peered":"2024-03-21T13:26:13.223957+0000","last_unstale":"2024-03-21T13:27:21.250540+0000","last_undegraded":"2024-03-21T13:27:21.250540+0000","last_fullsized":"2024-03-21T13:27:21.250540+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:58:13.081276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.639933+0000","last_change":"2024-03-21T13:26:15.225377+0000","last_active":"2024-03-21T13:26:52.639933+0000","last_peered":"2024-03-21T13:26:52.639933+0000","last_clean":"2024-03-21T13:26:52.639933+0000","last_became_active":"2024-03-21T13:26:15.225049+0000","last_became_peered":"2024-03-21T13:26:15.225049+0000","last_unstale":"2024-03-21T13:26:52.639933+0000","last_undegraded":"2024-03-21T13:26:52.639933+0000","last_fullsized":"2024-03-21T13:26:52.639933+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:05:34.989909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":20,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.386859+0000","last_change":"2024-03-21T13:26:17.306745+0000","last_active":"2024-03-21T13:26:52.386859+0000","last_peered":"2024-03-21T13:26:52.386859+0000","last_clean":"2024-03-21T13:26:52.386859+0000","last_became_active":"2024-03-21T13:26:17.306424+0000","last_became_peered":"2024-03-21T13:26:17.306424+0000","last_unstale":"2024-03-21T13:26:52.386859+0000","last_undegraded":"2024-03-21T13:26:52.386859+0000","last_fullsized":"2024-03-21T13:26:52.386859+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:56:49.516361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.640071+0000","last_change":"2024-03-21T13:26:11.194777+0000","last_active":"2024-03-21T13:26:52.640071+0000","last_peered":"2024-03-21T13:26:52.640071+0000","last_clean":"2024-03-21T13:26:52.640071+0000","last_became_active":"2024-03-21T13:26:11.194085+0000","last_became_peered":"2024-03-21T13:26:11.194085+0000","last_unstale":"2024-03-21T13:26:52.640071+0000","last_undegraded":"2024-03-21T13:26:52.640071+0000","last_fullsized":"2024-03-21T13:26:52.640071+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:20:59.362538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":40,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.382459+0000","last_change":"2024-03-21T13:26:15.215584+0000","last_active":"2024-03-21T13:26:52.382459+0000","last_peered":"2024-03-21T13:26:52.382459+0000","last_clean":"2024-03-21T13:26:52.382459+0000","last_became_active":"2024-03-21T13:26:11.195550+0000","last_became_peered":"2024-03-21T13:26:11.195550+0000","last_unstale":"2024-03-21T13:26:52.382459+0000","last_undegraded":"2024-03-21T13:26:52.382459+0000","last_fullsized":"2024-03-21T13:26:52.382459+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:11:07.000090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055458799999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"67'13","reported_seq":56,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.641370+0000","last_change":"2024-03-21T13:26:13.223129+0000","last_active":"2024-03-21T13:26:52.641370+0000","last_peered":"2024-03-21T13:26:52.641370+0000","last_clean":"2024-03-21T13:26:52.641370+0000","last_became_active":"2024-03-21T13:26:13.222689+0000","last_became_peered":"2024-03-21T13:26:13.222689+0000","last_unstale":"2024-03-21T13:26:52.641370+0000","last_undegraded":"2024-03-21T13:26:52.641370+0000","last_fullsized":"2024-03-21T13:26:52.641370+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:48:45.238762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.383136+0000","last_change":"2024-03-21T13:26:15.233122+0000","last_active":"2024-03-21T13:26:52.383136+0000","last_peered":"2024-03-21T13:26:52.383136+0000","last_clean":"2024-03-21T13:26:52.383136+0000","last_became_active":"2024-03-21T13:26:15.232807+0000","last_became_peered":"2024-03-21T13:26:15.232807+0000","last_unstale":"2024-03-21T13:26:52.383136+0000","last_undegraded":"2024-03-21T13:26:52.383136+0000","last_fullsized":"2024-03-21T13:26:52.383136+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:41:15.086830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.374944+0000","last_change":"2024-03-21T13:26:17.249433+0000","last_active":"2024-03-21T13:26:52.374944+0000","last_peered":"2024-03-21T13:26:52.374944+0000","last_clean":"2024-03-21T13:26:52.374944+0000","last_became_active":"2024-03-21T13:26:17.249153+0000","last_became_peered":"2024-03-21T13:26:17.249153+0000","last_unstale":"2024-03-21T13:26:52.374944+0000","last_undegraded":"2024-03-21T13:26:52.374944+0000","last_fullsized":"2024-03-21T13:26:52.374944+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:31:38.351777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"71'128","reported_seq":275,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:54.371886+0000","last_change":"2024-03-21T13:26:01.178260+0000","last_active":"2024-03-21T13:26:54.371886+0000","last_peered":"2024-03-21T13:26:54.371886+0000","last_clean":"2024-03-21T13:26:54.371886+0000","last_became_active":"2024-03-21T13:26:01.177752+0000","last_became_peered":"2024-03-21T13:26:01.177752+0000","last_unstale":"2024-03-21T13:26:54.371886+0000","last_undegraded":"2024-03-21T13:26:54.371886+0000","last_fullsized":"2024-03-21T13:26:54.371886+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:23:52.146116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:23:52.146116+0000","last_clean_scrub_stamp":"2024-03-21T13:23:52.146116+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T01:06:35.726853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.269533+0000","last_change":"2024-03-21T13:26:11.197631+0000","last_active":"2024-03-21T13:26:52.269533+0000","last_peered":"2024-03-21T13:26:52.269533+0000","last_clean":"2024-03-21T13:26:52.269533+0000","last_became_active":"2024-03-21T13:26:11.197291+0000","last_became_peered":"2024-03-21T13:26:11.197291+0000","last_unstale":"2024-03-21T13:26:52.269533+0000","last_undegraded":"2024-03-21T13:26:52.269533+0000","last_fullsized":"2024-03-21T13:26:52.269533+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:38:47.648305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"67'16","reported_seq":67,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:27:21.250969+0000","last_change":"2024-03-21T13:26:13.220005+0000","last_active":"2024-03-21T13:27:21.250969+0000","last_peered":"2024-03-21T13:27:21.250969+0000","last_clean":"2024-03-21T13:27:21.250969+0000","last_became_active":"2024-03-21T13:26:13.219283+0000","last_became_peered":"2024-03-21T13:26:13.219283+0000","last_unstale":"2024-03-21T13:27:21.250969+0000","last_undegraded":"2024-03-21T13:27:21.250969+0000","last_fullsized":"2024-03-21T13:27:21.250969+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:44:55.006479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":24,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.386709+0000","last_change":"2024-03-21T13:26:15.241005+0000","last_active":"2024-03-21T13:26:52.386709+0000","last_peered":"2024-03-21T13:26:52.386709+0000","last_clean":"2024-03-21T13:26:52.386709+0000","last_became_active":"2024-03-21T13:26:15.240661+0000","last_became_peered":"2024-03-21T13:26:15.240661+0000","last_unstale":"2024-03-21T13:26:52.386709+0000","last_undegraded":"2024-03-21T13:26:52.386709+0000","last_fullsized":"2024-03-21T13:26:52.386709+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:42:05.198556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.640429+0000","last_change":"2024-03-21T13:26:17.250469+0000","last_active":"2024-03-21T13:26:52.640429+0000","last_peered":"2024-03-21T13:26:52.640429+0000","last_clean":"2024-03-21T13:26:52.640429+0000","last_became_active":"2024-03-21T13:26:17.250144+0000","last_became_peered":"2024-03-21T13:26:17.250144+0000","last_unstale":"2024-03-21T13:26:52.640429+0000","last_undegraded":"2024-03-21T13:26:52.640429+0000","last_fullsized":"2024-03-21T13:26:52.640429+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:28:11.133824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.639497+0000","last_change":"2024-03-21T13:26:11.197464+0000","last_active":"2024-03-21T13:26:52.639497+0000","last_peered":"2024-03-21T13:26:52.639497+0000","last_clean":"2024-03-21T13:26:52.639497+0000","last_became_active":"2024-03-21T13:26:11.197172+0000","last_became_peered":"2024-03-21T13:26:11.197172+0000","last_unstale":"2024-03-21T13:26:52.639497+0000","last_undegraded":"2024-03-21T13:26:52.639497+0000","last_fullsized":"2024-03-21T13:26:52.639497+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:58:15.194164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"68'5","reported_seq":112,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:27:26.371275+0000","last_change":"2024-03-21T13:26:15.229957+0000","last_active":"2024-03-21T13:27:26.371275+0000","last_peered":"2024-03-21T13:27:26.371275+0000","last_clean":"2024-03-21T13:27:26.371275+0000","last_became_active":"2024-03-21T13:26:11.197128+0000","last_became_peered":"2024-03-21T13:26:11.197128+0000","last_unstale":"2024-03-21T13:27:26.371275+0000","last_undegraded":"2024-03-21T13:27:26.371275+0000","last_fullsized":"2024-03-21T13:27:26.371275+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:50:26.083584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044008600000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":70,"num_read_kb":65,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"67'12","reported_seq":67,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.640802+0000","last_change":"2024-03-21T13:26:32.425101+0000","last_active":"2024-03-21T13:26:52.640802+0000","last_peered":"2024-03-21T13:26:52.640802+0000","last_clean":"2024-03-21T13:26:52.640802+0000","last_became_active":"2024-03-21T13:26:32.415693+0000","last_became_peered":"2024-03-21T13:26:32.415693+0000","last_unstale":"2024-03-21T13:26:52.640802+0000","last_undegraded":"2024-03-21T13:26:52.640802+0000","last_fullsized":"2024-03-21T13:26:52.640802+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:23:28.199377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.375213+0000","last_change":"2024-03-21T13:26:15.238216+0000","last_active":"2024-03-21T13:26:52.375213+0000","last_peered":"2024-03-21T13:26:52.375213+0000","last_clean":"2024-03-21T13:26:52.375213+0000","last_became_active":"2024-03-21T13:26:15.237907+0000","last_became_peered":"2024-03-21T13:26:15.237907+0000","last_unstale":"2024-03-21T13:26:52.375213+0000","last_undegraded":"2024-03-21T13:26:52.375213+0000","last_fullsized":"2024-03-21T13:26:52.375213+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:05:23.428509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.639637+0000","last_change":"2024-03-21T13:26:17.307080+0000","last_active":"2024-03-21T13:26:52.639637+0000","last_peered":"2024-03-21T13:26:52.639637+0000","last_clean":"2024-03-21T13:26:52.639637+0000","last_became_active":"2024-03-21T13:26:17.306454+0000","last_became_peered":"2024-03-21T13:26:17.306454+0000","last_unstale":"2024-03-21T13:26:52.639637+0000","last_undegraded":"2024-03-21T13:26:52.639637+0000","last_fullsized":"2024-03-21T13:26:52.639637+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:29:41.493818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"60'1","reported_seq":39,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.640749+0000","last_change":"2024-03-21T13:26:11.199094+0000","last_active":"2024-03-21T13:26:52.640749+0000","last_peered":"2024-03-21T13:26:52.640749+0000","last_clean":"2024-03-21T13:26:52.640749+0000","last_became_active":"2024-03-21T13:26:11.198739+0000","last_became_peered":"2024-03-21T13:26:11.198739+0000","last_unstale":"2024-03-21T13:26:52.640749+0000","last_undegraded":"2024-03-21T13:26:52.640749+0000","last_fullsized":"2024-03-21T13:26:52.640749+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:03:15.816625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":911,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"67'19","reported_seq":65,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.640934+0000","last_change":"2024-03-21T13:26:13.215770+0000","last_active":"2024-03-21T13:26:52.640934+0000","last_peered":"2024-03-21T13:26:52.640934+0000","last_clean":"2024-03-21T13:26:52.640934+0000","last_became_active":"2024-03-21T13:26:13.215343+0000","last_became_peered":"2024-03-21T13:26:13.215343+0000","last_unstale":"2024-03-21T13:26:52.640934+0000","last_undegraded":"2024-03-21T13:26:52.640934+0000","last_fullsized":"2024-03-21T13:26:52.640934+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:57:05.065089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.547563+0000","last_change":"2024-03-21T13:26:15.234129+0000","last_active":"2024-03-21T13:26:52.547563+0000","last_peered":"2024-03-21T13:26:52.547563+0000","last_clean":"2024-03-21T13:26:52.547563+0000","last_became_active":"2024-03-21T13:26:15.233803+0000","last_became_peered":"2024-03-21T13:26:15.233803+0000","last_unstale":"2024-03-21T13:26:52.547563+0000","last_undegraded":"2024-03-21T13:26:52.547563+0000","last_fullsized":"2024-03-21T13:26:52.547563+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:15:35.028442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.639204+0000","last_change":"2024-03-21T13:26:17.249942+0000","last_active":"2024-03-21T13:26:52.639204+0000","last_peered":"2024-03-21T13:26:52.639204+0000","last_clean":"2024-03-21T13:26:52.639204+0000","last_became_active":"2024-03-21T13:26:17.249616+0000","last_became_peered":"2024-03-21T13:26:17.249616+0000","last_unstale":"2024-03-21T13:26:52.639204+0000","last_undegraded":"2024-03-21T13:26:52.639204+0000","last_fullsized":"2024-03-21T13:26:52.639204+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:26:42.140259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.374872+0000","last_change":"2024-03-21T13:26:11.195031+0000","last_active":"2024-03-21T13:26:52.374872+0000","last_peered":"2024-03-21T13:26:52.374872+0000","last_clean":"2024-03-21T13:26:52.374872+0000","last_became_active":"2024-03-21T13:26:11.194727+0000","last_became_peered":"2024-03-21T13:26:11.194727+0000","last_unstale":"2024-03-21T13:26:52.374872+0000","last_undegraded":"2024-03-21T13:26:52.374872+0000","last_fullsized":"2024-03-21T13:26:52.374872+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:36:25.687188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"67'10","reported_seq":44,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.641672+0000","last_change":"2024-03-21T13:26:13.222254+0000","last_active":"2024-03-21T13:26:52.641672+0000","last_peered":"2024-03-21T13:26:52.641672+0000","last_clean":"2024-03-21T13:26:52.641672+0000","last_became_active":"2024-03-21T13:26:13.220420+0000","last_became_peered":"2024-03-21T13:26:13.220420+0000","last_unstale":"2024-03-21T13:26:52.641672+0000","last_undegraded":"2024-03-21T13:26:52.641672+0000","last_fullsized":"2024-03-21T13:26:52.641672+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T00:31:51.133481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"67'8","reported_seq":49,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:27:21.250611+0000","last_change":"2024-03-21T13:26:15.232582+0000","last_active":"2024-03-21T13:27:21.250611+0000","last_peered":"2024-03-21T13:27:21.250611+0000","last_clean":"2024-03-21T13:27:21.250611+0000","last_became_active":"2024-03-21T13:26:15.232270+0000","last_became_peered":"2024-03-21T13:26:15.232270+0000","last_unstale":"2024-03-21T13:27:21.250611+0000","last_undegraded":"2024-03-21T13:27:21.250611+0000","last_fullsized":"2024-03-21T13:27:21.250611+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:57:50.128265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":20,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.386510+0000","last_change":"2024-03-21T13:26:17.308604+0000","last_active":"2024-03-21T13:26:52.386510+0000","last_peered":"2024-03-21T13:26:52.386510+0000","last_clean":"2024-03-21T13:26:52.386510+0000","last_became_active":"2024-03-21T13:26:17.308310+0000","last_became_peered":"2024-03-21T13:26:17.308310+0000","last_unstale":"2024-03-21T13:26:52.386510+0000","last_undegraded":"2024-03-21T13:26:52.386510+0000","last_fullsized":"2024-03-21T13:26:52.386510+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:33:56.518710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"60'2","reported_seq":40,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.640815+0000","last_change":"2024-03-21T13:26:11.197777+0000","last_active":"2024-03-21T13:26:52.640815+0000","last_peered":"2024-03-21T13:26:52.640815+0000","last_clean":"2024-03-21T13:26:52.640815+0000","last_became_active":"2024-03-21T13:26:11.197280+0000","last_became_peered":"2024-03-21T13:26:11.197280+0000","last_unstale":"2024-03-21T13:26:52.640815+0000","last_undegraded":"2024-03-21T13:26:52.640815+0000","last_fullsized":"2024-03-21T13:26:52.640815+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:37:03.885564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":482,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"67'14","reported_seq":50,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.271186+0000","last_change":"2024-03-21T13:26:13.218843+0000","last_active":"2024-03-21T13:26:52.271186+0000","last_peered":"2024-03-21T13:26:52.271186+0000","last_clean":"2024-03-21T13:26:52.271186+0000","last_became_active":"2024-03-21T13:26:13.218561+0000","last_became_peered":"2024-03-21T13:26:13.218561+0000","last_unstale":"2024-03-21T13:26:52.271186+0000","last_undegraded":"2024-03-21T13:26:52.271186+0000","last_fullsized":"2024-03-21T13:26:52.271186+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:59:16.467897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"67'8","reported_seq":49,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:27:21.250561+0000","last_change":"2024-03-21T13:26:15.235401+0000","last_active":"2024-03-21T13:27:21.250561+0000","last_peered":"2024-03-21T13:27:21.250561+0000","last_clean":"2024-03-21T13:27:21.250561+0000","last_became_active":"2024-03-21T13:26:15.235025+0000","last_became_peered":"2024-03-21T13:26:15.235025+0000","last_unstale":"2024-03-21T13:27:21.250561+0000","last_undegraded":"2024-03-21T13:27:21.250561+0000","last_fullsized":"2024-03-21T13:27:21.250561+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T00:30:53.823248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.271209+0000","last_change":"2024-03-21T13:26:17.247947+0000","last_active":"2024-03-21T13:26:52.271209+0000","last_peered":"2024-03-21T13:26:52.271209+0000","last_clean":"2024-03-21T13:26:52.271209+0000","last_became_active":"2024-03-21T13:26:17.247641+0000","last_became_peered":"2024-03-21T13:26:17.247641+0000","last_unstale":"2024-03-21T13:26:52.271209+0000","last_undegraded":"2024-03-21T13:26:52.271209+0000","last_fullsized":"2024-03-21T13:26:52.271209+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T00:00:51.801628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.271685+0000","last_change":"2024-03-21T13:26:11.196455+0000","last_active":"2024-03-21T13:26:52.271685+0000","last_peered":"2024-03-21T13:26:52.271685+0000","last_clean":"2024-03-21T13:26:52.271685+0000","last_became_active":"2024-03-21T13:26:11.195000+0000","last_became_peered":"2024-03-21T13:26:11.195000+0000","last_unstale":"2024-03-21T13:26:52.271685+0000","last_undegraded":"2024-03-21T13:26:52.271685+0000","last_fullsized":"2024-03-21T13:26:52.271685+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:46:59.830691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"67'18","reported_seq":61,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.269232+0000","last_change":"2024-03-21T13:26:13.219901+0000","last_active":"2024-03-21T13:26:52.269232+0000","last_peered":"2024-03-21T13:26:52.269232+0000","last_clean":"2024-03-21T13:26:52.269232+0000","last_became_active":"2024-03-21T13:26:13.219568+0000","last_became_peered":"2024-03-21T13:26:13.219568+0000","last_unstale":"2024-03-21T13:26:52.269232+0000","last_undegraded":"2024-03-21T13:26:52.269232+0000","last_fullsized":"2024-03-21T13:26:52.269232+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:59:30.215346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.271032+0000","last_change":"2024-03-21T13:26:15.229888+0000","last_active":"2024-03-21T13:26:52.271032+0000","last_peered":"2024-03-21T13:26:52.271032+0000","last_clean":"2024-03-21T13:26:52.271032+0000","last_became_active":"2024-03-21T13:26:15.229607+0000","last_became_peered":"2024-03-21T13:26:15.229607+0000","last_unstale":"2024-03-21T13:26:52.271032+0000","last_undegraded":"2024-03-21T13:26:52.271032+0000","last_fullsized":"2024-03-21T13:26:52.271032+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:30:34.420895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.374640+0000","last_change":"2024-03-21T13:26:17.245904+0000","last_active":"2024-03-21T13:26:52.374640+0000","last_peered":"2024-03-21T13:26:52.374640+0000","last_clean":"2024-03-21T13:26:52.374640+0000","last_became_active":"2024-03-21T13:26:17.245638+0000","last_became_peered":"2024-03-21T13:26:17.245638+0000","last_unstale":"2024-03-21T13:26:52.374640+0000","last_undegraded":"2024-03-21T13:26:52.374640+0000","last_fullsized":"2024-03-21T13:26:52.374640+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:53:47.375388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"67'8","reported_seq":49,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:27:21.250676+0000","last_change":"2024-03-21T13:26:15.235193+0000","last_active":"2024-03-21T13:27:21.250676+0000","last_peered":"2024-03-21T13:27:21.250676+0000","last_clean":"2024-03-21T13:27:21.250676+0000","last_became_active":"2024-03-21T13:26:15.234885+0000","last_became_peered":"2024-03-21T13:26:15.234885+0000","last_unstale":"2024-03-21T13:27:21.250676+0000","last_undegraded":"2024-03-21T13:27:21.250676+0000","last_fullsized":"2024-03-21T13:27:21.250676+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:04:47.692918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.269201+0000","last_change":"2024-03-21T13:26:11.193648+0000","last_active":"2024-03-21T13:26:52.269201+0000","last_peered":"2024-03-21T13:26:52.269201+0000","last_clean":"2024-03-21T13:26:52.269201+0000","last_became_active":"2024-03-21T13:26:11.193211+0000","last_became_peered":"2024-03-21T13:26:11.193211+0000","last_unstale":"2024-03-21T13:26:52.269201+0000","last_undegraded":"2024-03-21T13:26:52.269201+0000","last_fullsized":"2024-03-21T13:26:52.269201+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:28:43.496752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"67'15","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.383461+0000","last_change":"2024-03-21T13:26:32.428003+0000","last_active":"2024-03-21T13:26:52.383461+0000","last_peered":"2024-03-21T13:26:52.383461+0000","last_clean":"2024-03-21T13:26:52.383461+0000","last_became_active":"2024-03-21T13:26:32.418711+0000","last_became_peered":"2024-03-21T13:26:32.418711+0000","last_unstale":"2024-03-21T13:26:52.383461+0000","last_undegraded":"2024-03-21T13:26:52.383461+0000","last_fullsized":"2024-03-21T13:26:52.383461+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:51:55.750843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.269708+0000","last_change":"2024-03-21T13:26:17.305635+0000","last_active":"2024-03-21T13:26:52.269708+0000","last_peered":"2024-03-21T13:26:52.269708+0000","last_clean":"2024-03-21T13:26:52.269708+0000","last_became_active":"2024-03-21T13:26:17.305293+0000","last_became_peered":"2024-03-21T13:26:17.305293+0000","last_unstale":"2024-03-21T13:26:52.269708+0000","last_undegraded":"2024-03-21T13:26:52.269708+0000","last_fullsized":"2024-03-21T13:26:52.269708+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:42:58.999639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.375209+0000","last_change":"2024-03-21T13:26:15.238323+0000","last_active":"2024-03-21T13:26:52.375209+0000","last_peered":"2024-03-21T13:26:52.375209+0000","last_clean":"2024-03-21T13:26:52.375209+0000","last_became_active":"2024-03-21T13:26:15.238000+0000","last_became_peered":"2024-03-21T13:26:15.238000+0000","last_unstale":"2024-03-21T13:26:52.375209+0000","last_undegraded":"2024-03-21T13:26:52.375209+0000","last_fullsized":"2024-03-21T13:26:52.375209+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:42:57.216972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.271057+0000","last_change":"2024-03-21T13:26:11.196402+0000","last_active":"2024-03-21T13:26:52.271057+0000","last_peered":"2024-03-21T13:26:52.271057+0000","last_clean":"2024-03-21T13:26:52.271057+0000","last_became_active":"2024-03-21T13:26:11.195729+0000","last_became_peered":"2024-03-21T13:26:11.195729+0000","last_unstale":"2024-03-21T13:26:52.271057+0000","last_undegraded":"2024-03-21T13:26:52.271057+0000","last_fullsized":"2024-03-21T13:26:52.271057+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:00:40.926613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"67'11","reported_seq":48,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.272512+0000","last_change":"2024-03-21T13:26:13.222434+0000","last_active":"2024-03-21T13:26:52.272512+0000","last_peered":"2024-03-21T13:26:52.272512+0000","last_clean":"2024-03-21T13:26:52.272512+0000","last_became_active":"2024-03-21T13:26:13.221644+0000","last_became_peered":"2024-03-21T13:26:13.221644+0000","last_unstale":"2024-03-21T13:26:52.272512+0000","last_undegraded":"2024-03-21T13:26:52.272512+0000","last_fullsized":"2024-03-21T13:26:52.272512+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T00:18:55.479249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.383604+0000","last_change":"2024-03-21T13:26:17.251871+0000","last_active":"2024-03-21T13:26:52.383604+0000","last_peered":"2024-03-21T13:26:52.383604+0000","last_clean":"2024-03-21T13:26:52.383604+0000","last_became_active":"2024-03-21T13:26:17.251545+0000","last_became_peered":"2024-03-21T13:26:17.251545+0000","last_unstale":"2024-03-21T13:26:52.383604+0000","last_undegraded":"2024-03-21T13:26:52.383604+0000","last_fullsized":"2024-03-21T13:26:52.383604+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:57:07.626204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.641081+0000","last_change":"2024-03-21T13:26:15.226863+0000","last_active":"2024-03-21T13:26:52.641081+0000","last_peered":"2024-03-21T13:26:52.641081+0000","last_clean":"2024-03-21T13:26:52.641081+0000","last_became_active":"2024-03-21T13:26:15.226490+0000","last_became_peered":"2024-03-21T13:26:15.226490+0000","last_unstale":"2024-03-21T13:26:52.641081+0000","last_undegraded":"2024-03-21T13:26:52.641081+0000","last_fullsized":"2024-03-21T13:26:52.641081+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:07:37.450970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.547032+0000","last_change":"2024-03-21T13:26:11.196591+0000","last_active":"2024-03-21T13:26:52.547032+0000","last_peered":"2024-03-21T13:26:52.547032+0000","last_clean":"2024-03-21T13:26:52.547032+0000","last_became_active":"2024-03-21T13:26:11.196304+0000","last_became_peered":"2024-03-21T13:26:11.196304+0000","last_unstale":"2024-03-21T13:26:52.547032+0000","last_undegraded":"2024-03-21T13:26:52.547032+0000","last_fullsized":"2024-03-21T13:26:52.547032+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:55:25.839773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"67'17","reported_seq":57,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.272192+0000","last_change":"2024-03-21T13:26:13.222129+0000","last_active":"2024-03-21T13:26:52.272192+0000","last_peered":"2024-03-21T13:26:52.272192+0000","last_clean":"2024-03-21T13:26:52.272192+0000","last_became_active":"2024-03-21T13:26:13.221554+0000","last_became_peered":"2024-03-21T13:26:13.221554+0000","last_unstale":"2024-03-21T13:26:52.272192+0000","last_undegraded":"2024-03-21T13:26:52.272192+0000","last_fullsized":"2024-03-21T13:26:52.272192+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:30:27.492301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.272635+0000","last_change":"2024-03-21T13:26:17.248809+0000","last_active":"2024-03-21T13:26:52.272635+0000","last_peered":"2024-03-21T13:26:52.272635+0000","last_clean":"2024-03-21T13:26:52.272635+0000","last_became_active":"2024-03-21T13:26:17.248504+0000","last_became_peered":"2024-03-21T13:26:17.248504+0000","last_unstale":"2024-03-21T13:26:52.272635+0000","last_undegraded":"2024-03-21T13:26:52.272635+0000","last_fullsized":"2024-03-21T13:26:52.272635+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T01:17:19.221467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"67'8","reported_seq":49,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:27:21.250519+0000","last_change":"2024-03-21T13:26:15.232591+0000","last_active":"2024-03-21T13:27:21.250519+0000","last_peered":"2024-03-21T13:27:21.250519+0000","last_clean":"2024-03-21T13:27:21.250519+0000","last_became_active":"2024-03-21T13:26:15.232239+0000","last_became_peered":"2024-03-21T13:26:15.232239+0000","last_unstale":"2024-03-21T13:27:21.250519+0000","last_undegraded":"2024-03-21T13:27:21.250519+0000","last_fullsized":"2024-03-21T13:27:21.250519+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:59:18.606505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.269921+0000","last_change":"2024-03-21T13:26:11.195917+0000","last_active":"2024-03-21T13:26:52.269921+0000","last_peered":"2024-03-21T13:26:52.269921+0000","last_clean":"2024-03-21T13:26:52.269921+0000","last_became_active":"2024-03-21T13:26:11.195581+0000","last_became_peered":"2024-03-21T13:26:11.195581+0000","last_unstale":"2024-03-21T13:26:52.269921+0000","last_undegraded":"2024-03-21T13:26:52.269921+0000","last_fullsized":"2024-03-21T13:26:52.269921+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:17:56.216644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"67'10","reported_seq":44,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.271906+0000","last_change":"2024-03-21T13:26:13.224564+0000","last_active":"2024-03-21T13:26:52.271906+0000","last_peered":"2024-03-21T13:26:52.271906+0000","last_clean":"2024-03-21T13:26:52.271906+0000","last_became_active":"2024-03-21T13:26:13.224245+0000","last_became_peered":"2024-03-21T13:26:13.224245+0000","last_unstale":"2024-03-21T13:26:52.271906+0000","last_undegraded":"2024-03-21T13:26:52.271906+0000","last_fullsized":"2024-03-21T13:26:52.271906+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:46:05.005340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":383,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1531,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":65,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":9,"acting":9,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234004,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32448,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712704,"statfs":{"total":95995035648,"available":95961808896,"internally_reserved":0,"allocated":1302528,"data_stored":867673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495641,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32428,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712724,"statfs":{"total":95995035648,"available":95961829376,"internally_reserved":0,"allocated":1277952,"data_stored":866608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27752,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717400,"statfs":{"total":95995035648,"available":95966617600,"internally_reserved":0,"allocated":749568,"data_stored":279055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738425,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32460,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712692,"statfs":{"total":95995035648,"available":95961796608,"internally_reserved":0,"allocated":1310720,"data_stored":867851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411374,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27700,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717452,"statfs":{"total":95995035648,"available":95966670848,"internally_reserved":0,"allocated":696320,"data_stored":277596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182441,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31884,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713268,"statfs":{"total":95995035648,"available":95962386432,"internally_reserved":0,"allocated":720896,"data_stored":277131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018916,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31880,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713272,"statfs":{"total":95995035648,"available":95962390528,"internally_reserved":0,"allocated":716800,"data_stored":277281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757278,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31856,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713296,"statfs":{"total":95995035648,"available":95962415104,"internally_reserved":0,"allocated":692224,"data_stored":277477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-21T13:27:31.555 DEBUG:teuthology.orchestra.run.smithi097:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph pg dump --format=json 2024-03-21T13:27:31.810 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T13:27:32.229 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:27:31 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:27:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:27:32.229 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:32 smithi097 ceph-mon[39416]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:32.229 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:32 smithi097 ceph-mon[39416]: Updating smithi097:/var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T13:27:32.229 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:32 smithi097 ceph-mon[39416]: Updating smithi144:/var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T13:27:32.229 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:32 smithi097 ceph-mon[39416]: Updating smithi097:/etc/ceph/ceph.client.admin.keyring 2024-03-21T13:27:32.229 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:32 smithi097 ceph-mon[39416]: Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-03-21T13:27:32.230 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:32 smithi097 ceph-mon[39416]: Updating smithi097:/var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T13:27:32.230 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:32 smithi097 ceph-mon[39416]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T13:27:32.230 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:32 smithi097 ceph-mon[39416]: Updating smithi144:/var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T13:27:32.230 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:32 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:32.230 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:32 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:32.230 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:32 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:32.230 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:32 smithi097 ceph-mon[39416]: Deploying daemon alertmanager.a on smithi097 2024-03-21T13:27:32.230 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:32 smithi097 ceph-mon[30040]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:32.230 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:32 smithi097 ceph-mon[30040]: Updating smithi097:/var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T13:27:32.230 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:32 smithi097 ceph-mon[30040]: Updating smithi144:/var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T13:27:32.231 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:32 smithi097 ceph-mon[30040]: Updating smithi097:/etc/ceph/ceph.client.admin.keyring 2024-03-21T13:27:32.231 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:32 smithi097 ceph-mon[30040]: Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-03-21T13:27:32.231 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:32 smithi097 ceph-mon[30040]: Updating smithi097:/var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T13:27:32.231 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:32 smithi097 ceph-mon[30040]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T13:27:32.231 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:32 smithi097 ceph-mon[30040]: Updating smithi144:/var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T13:27:32.231 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:32 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:32.231 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:32 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:32.231 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:32 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:32.231 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:32 smithi097 ceph-mon[30040]: Deploying daemon alertmanager.a on smithi097 2024-03-21T13:27:32.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:32 smithi144 ceph-mon[33633]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:32.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:32 smithi144 ceph-mon[33633]: Updating smithi097:/var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T13:27:32.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:32 smithi144 ceph-mon[33633]: Updating smithi144:/var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T13:27:32.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:32 smithi144 ceph-mon[33633]: Updating smithi097:/etc/ceph/ceph.client.admin.keyring 2024-03-21T13:27:32.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:32 smithi144 ceph-mon[33633]: Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-03-21T13:27:32.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:32 smithi144 ceph-mon[33633]: Updating smithi097:/var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T13:27:32.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:32 smithi144 ceph-mon[33633]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T13:27:32.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:32 smithi144 ceph-mon[33633]: Updating smithi144:/var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T13:27:32.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:32 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:32.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:32 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:32.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:32 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:32.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:32 smithi144 ceph-mon[33633]: Deploying daemon alertmanager.a on smithi097 2024-03-21T13:27:33.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:27:33 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:27:34.116 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:34 smithi097 ceph-mon[39416]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:34.117 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:34 smithi097 ceph-mon[30040]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:34.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.115+0000 7f56b1bdc700 1 -- 172.21.15.97:0/2532652767 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f56ac1023d0 msgr2=0x7f56ac1027b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:34.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.115+0000 7f56b1bdc700 1 --2- 172.21.15.97:0/2532652767 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f56ac1023d0 0x7f56ac1027b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f569c0099c0 tx=0x7f569c009cd0 comp rx=0 tx=0).stop 2024-03-21T13:27:34.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.115+0000 7f56b1bdc700 1 -- 172.21.15.97:0/2532652767 shutdown_connections 2024-03-21T13:27:34.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.115+0000 7f56b1bdc700 1 --2- 172.21.15.97:0/2532652767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56ac1074b0 0x7f56ac10f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:34.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.115+0000 7f56b1bdc700 1 --2- 172.21.15.97:0/2532652767 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f56ac1023d0 0x7f56ac1027b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:34.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.115+0000 7f56b1bdc700 1 --2- 172.21.15.97:0/2532652767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56ac102d80 0x7f56ac106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:34.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.115+0000 7f56b1bdc700 1 -- 172.21.15.97:0/2532652767 >> 172.21.15.97:0/2532652767 conn(0x7f56ac0fdc40 msgr2=0x7f56ac100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:27:34.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.115+0000 7f56b1bdc700 1 -- 172.21.15.97:0/2532652767 shutdown_connections 2024-03-21T13:27:34.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.115+0000 7f56b1bdc700 1 -- 172.21.15.97:0/2532652767 wait complete. 2024-03-21T13:27:34.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.116+0000 7f56b1bdc700 1 Processor -- start 2024-03-21T13:27:34.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.116+0000 7f56b1bdc700 1 -- start start 2024-03-21T13:27:34.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.116+0000 7f56b1bdc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56ac1023d0 0x7f56ac1980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:34.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.116+0000 7f56b1bdc700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f56ac102d80 0x7f56ac198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:34.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.116+0000 7f56b1bdc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56ac1074b0 0x7f56ac19c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:34.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.116+0000 7f56b1bdc700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ac111c10 con 0x7f56ac1074b0 2024-03-21T13:27:34.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.116+0000 7f56b1bdc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ac111a90 con 0x7f56ac1023d0 2024-03-21T13:27:34.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.116+0000 7f56b1bdc700 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56ac111d90 con 0x7f56ac102d80 2024-03-21T13:27:34.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.117+0000 7f56abfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56ac1074b0 0x7f56ac19c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:34.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.117+0000 7f56ab7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56ac1023d0 0x7f56ac1980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:34.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.117+0000 7f56aaffd700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f56ac102d80 0x7f56ac198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:34.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.117+0000 7f56abfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56ac1074b0 0x7f56ac19c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45678/0 (socket says 172.21.15.97:45678) 2024-03-21T13:27:34.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.117+0000 7f56abfff700 1 -- 172.21.15.97:0/1563239598 learned_addr learned my addr 172.21.15.97:0/1563239598 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:27:34.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.117+0000 7f56aaffd700 1 -- 172.21.15.97:0/1563239598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56ac1023d0 msgr2=0x7f56ac1980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:34.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.117+0000 7f56aaffd700 1 --2- 172.21.15.97:0/1563239598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56ac1023d0 0x7f56ac1980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:34.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.117+0000 7f56aaffd700 1 -- 172.21.15.97:0/1563239598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56ac1074b0 msgr2=0x7f56ac19c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:34.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.117+0000 7f56aaffd700 1 --2- 172.21.15.97:0/1563239598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56ac1074b0 0x7f56ac19c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:34.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.117+0000 7f56aaffd700 1 -- 172.21.15.97:0/1563239598 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56ac19d0e0 con 0x7f56ac102d80 2024-03-21T13:27:34.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.117+0000 7f56abfff700 1 --2- 172.21.15.97:0/1563239598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56ac1074b0 0x7f56ac19c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T13:27:34.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.117+0000 7f56aaffd700 1 --2- 172.21.15.97:0/1563239598 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f56ac102d80 0x7f56ac198610 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f56ac069be0 tx=0x7f56a000b730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:27:34.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.117+0000 7f56a8ff9700 1 -- 172.21.15.97:0/1563239598 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a0015020 con 0x7f56ac102d80 2024-03-21T13:27:34.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.117+0000 7f56b1bdc700 1 -- 172.21.15.97:0/1563239598 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56ac19d3d0 con 0x7f56ac102d80 2024-03-21T13:27:34.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.117+0000 7f56a8ff9700 1 -- 172.21.15.97:0/1563239598 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f56a000be30 con 0x7f56ac102d80 2024-03-21T13:27:34.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.117+0000 7f56a8ff9700 1 -- 172.21.15.97:0/1563239598 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a0010bc0 con 0x7f56ac102d80 2024-03-21T13:27:34.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.117+0000 7f56b1bdc700 1 -- 172.21.15.97:0/1563239598 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56ac0743c0 con 0x7f56ac102d80 2024-03-21T13:27:34.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.118+0000 7f56b1bdc700 1 -- 172.21.15.97:0/1563239598 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56ac068e50 con 0x7f56ac102d80 2024-03-21T13:27:34.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.119+0000 7f56a8ff9700 1 -- 172.21.15.97:0/1563239598 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 22) v1 ==== 92152+0+0 (secure 0 0 0) 0x7f56a0018020 con 0x7f56ac102d80 2024-03-21T13:27:34.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.119+0000 7f56a8ff9700 1 --2- 172.21.15.97:0/1563239598 >> v2:172.21.15.97:6800/12494470 conn(0x7f569406e300 0x7f56940707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:34.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.119+0000 7f56ab7fe700 1 --2- 172.21.15.97:0/1563239598 >> v2:172.21.15.97:6800/12494470 conn(0x7f569406e300 0x7f56940707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:34.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.119+0000 7f56a8ff9700 1 -- 172.21.15.97:0/1563239598 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f56a008da10 con 0x7f56ac102d80 2024-03-21T13:27:34.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.120+0000 7f56ab7fe700 1 --2- 172.21.15.97:0/1563239598 >> v2:172.21.15.97:6800/12494470 conn(0x7f569406e300 0x7f56940707c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f569c000c00 tx=0x7f569c005fb0 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:27:34.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.124+0000 7f56a8ff9700 1 -- 172.21.15.97:0/1563239598 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f56a001f020 con 0x7f56ac102d80 2024-03-21T13:27:34.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.319+0000 7f56b1bdc700 1 -- 172.21.15.97:0/1563239598 --> v2:172.21.15.97:6800/12494470 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f56ac0fe830 con 0x7f569406e300 2024-03-21T13:27:34.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.328+0000 7f56a8ff9700 1 -- 172.21.15.97:0/1563239598 <== mgr.24482 v2:172.21.15.97:6800/12494470 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+346469 (secure 0 0 0) 0x7f56ac0fe830 con 0x7f569406e300 2024-03-21T13:27:34.333 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:27:34.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.332+0000 7f56b1bdc700 1 -- 172.21.15.97:0/1563239598 >> v2:172.21.15.97:6800/12494470 conn(0x7f569406e300 msgr2=0x7f56940707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:34.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.332+0000 7f56b1bdc700 1 --2- 172.21.15.97:0/1563239598 >> v2:172.21.15.97:6800/12494470 conn(0x7f569406e300 0x7f56940707c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f569c000c00 tx=0x7f569c005fb0 comp rx=0 tx=0).stop 2024-03-21T13:27:34.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.332+0000 7f56b1bdc700 1 -- 172.21.15.97:0/1563239598 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f56ac102d80 msgr2=0x7f56ac198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:34.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.332+0000 7f56b1bdc700 1 --2- 172.21.15.97:0/1563239598 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f56ac102d80 0x7f56ac198610 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f56ac069be0 tx=0x7f56a000b730 comp rx=0 tx=0).stop 2024-03-21T13:27:34.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.332+0000 7f56b1bdc700 1 -- 172.21.15.97:0/1563239598 shutdown_connections 2024-03-21T13:27:34.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.332+0000 7f56b1bdc700 1 --2- 172.21.15.97:0/1563239598 >> v2:172.21.15.97:6800/12494470 conn(0x7f569406e300 0x7f56940707c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:34.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.332+0000 7f56b1bdc700 1 --2- 172.21.15.97:0/1563239598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56ac1074b0 0x7f56ac19c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:34.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.332+0000 7f56b1bdc700 1 --2- 172.21.15.97:0/1563239598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56ac1023d0 0x7f56ac1980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:34.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.332+0000 7f56b1bdc700 1 --2- 172.21.15.97:0/1563239598 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f56ac102d80 0x7f56ac198610 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:34.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.332+0000 7f56b1bdc700 1 -- 172.21.15.97:0/1563239598 >> 172.21.15.97:0/1563239598 conn(0x7f56ac0fdc40 msgr2=0x7f56ac0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:27:34.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.332+0000 7f56b1bdc700 1 -- 172.21.15.97:0/1563239598 shutdown_connections 2024-03-21T13:27:34.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:34.332+0000 7f56b1bdc700 1 -- 172.21.15.97:0/1563239598 wait complete. 2024-03-21T13:27:34.336 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-03-21T13:27:34.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:34 smithi144 ceph-mon[33633]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:35.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:35 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:27:35.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:35 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:27:35.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:35 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:27:35.862 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-03-21T13:27:34.287722+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596392,"num_objects":198,"num_object_clones":0,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":198,"num_whiteouts":0,"num_read":1009,"num_read_kb":855,"num_write":615,"num_write_kb":2461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":1771434,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":600,"ondisk_log_size":600,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":248408,"kb_used_data":7292,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":749712808,"statfs":{"total":767960285184,"available":767705915392,"internally_reserved":0,"allocated":7467008,"data_stored":3990672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003089"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.271092+0000","last_change":"2024-03-21T13:26:17.243407+0000","last_active":"2024-03-21T13:26:52.271092+0000","last_peered":"2024-03-21T13:26:52.271092+0000","last_clean":"2024-03-21T13:26:52.271092+0000","last_became_active":"2024-03-21T13:26:17.243093+0000","last_became_peered":"2024-03-21T13:26:17.243093+0000","last_unstale":"2024-03-21T13:26:52.271092+0000","last_undegraded":"2024-03-21T13:26:52.271092+0000","last_fullsized":"2024-03-21T13:26:52.271092+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:28:36.314318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.639746+0000","last_change":"2024-03-21T13:26:15.233162+0000","last_active":"2024-03-21T13:26:52.639746+0000","last_peered":"2024-03-21T13:26:52.639746+0000","last_clean":"2024-03-21T13:26:52.639746+0000","last_became_active":"2024-03-21T13:26:15.232850+0000","last_became_peered":"2024-03-21T13:26:15.232850+0000","last_unstale":"2024-03-21T13:26:52.639746+0000","last_undegraded":"2024-03-21T13:26:52.639746+0000","last_fullsized":"2024-03-21T13:26:52.639746+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:32:57.507907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.640374+0000","last_change":"2024-03-21T13:26:11.195505+0000","last_active":"2024-03-21T13:26:52.640374+0000","last_peered":"2024-03-21T13:26:52.640374+0000","last_clean":"2024-03-21T13:26:52.640374+0000","last_became_active":"2024-03-21T13:26:11.195221+0000","last_became_peered":"2024-03-21T13:26:11.195221+0000","last_unstale":"2024-03-21T13:26:52.640374+0000","last_undegraded":"2024-03-21T13:26:52.640374+0000","last_fullsized":"2024-03-21T13:26:52.640374+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:57:02.841819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"67'9","reported_seq":45,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.270982+0000","last_change":"2024-03-21T13:26:13.221104+0000","last_active":"2024-03-21T13:26:52.270982+0000","last_peered":"2024-03-21T13:26:52.270982+0000","last_clean":"2024-03-21T13:26:52.270982+0000","last_became_active":"2024-03-21T13:26:13.220739+0000","last_became_peered":"2024-03-21T13:26:13.220739+0000","last_unstale":"2024-03-21T13:26:52.270982+0000","last_undegraded":"2024-03-21T13:26:52.270982+0000","last_fullsized":"2024-03-21T13:26:52.270982+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:26:07.984202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.269727+0000","last_change":"2024-03-21T13:26:17.307060+0000","last_active":"2024-03-21T13:26:52.269727+0000","last_peered":"2024-03-21T13:26:52.269727+0000","last_clean":"2024-03-21T13:26:52.269727+0000","last_became_active":"2024-03-21T13:26:17.306235+0000","last_became_peered":"2024-03-21T13:26:17.306235+0000","last_unstale":"2024-03-21T13:26:52.269727+0000","last_undegraded":"2024-03-21T13:26:52.269727+0000","last_fullsized":"2024-03-21T13:26:52.269727+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:15:23.859528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.270980+0000","last_change":"2024-03-21T13:26:15.228250+0000","last_active":"2024-03-21T13:26:52.270980+0000","last_peered":"2024-03-21T13:26:52.270980+0000","last_clean":"2024-03-21T13:26:52.270980+0000","last_became_active":"2024-03-21T13:26:15.227966+0000","last_became_peered":"2024-03-21T13:26:15.227966+0000","last_unstale":"2024-03-21T13:26:52.270980+0000","last_undegraded":"2024-03-21T13:26:52.270980+0000","last_fullsized":"2024-03-21T13:26:52.270980+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:43:04.868113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.269895+0000","last_change":"2024-03-21T13:26:11.198738+0000","last_active":"2024-03-21T13:26:52.269895+0000","last_peered":"2024-03-21T13:26:52.269895+0000","last_clean":"2024-03-21T13:26:52.269895+0000","last_became_active":"2024-03-21T13:26:11.198399+0000","last_became_peered":"2024-03-21T13:26:11.198399+0000","last_unstale":"2024-03-21T13:26:52.269895+0000","last_undegraded":"2024-03-21T13:26:52.269895+0000","last_fullsized":"2024-03-21T13:26:52.269895+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:50:03.374446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"67'15","reported_seq":54,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.269593+0000","last_change":"2024-03-21T13:26:13.220169+0000","last_active":"2024-03-21T13:26:52.269593+0000","last_peered":"2024-03-21T13:26:52.269593+0000","last_clean":"2024-03-21T13:26:52.269593+0000","last_became_active":"2024-03-21T13:26:13.219860+0000","last_became_peered":"2024-03-21T13:26:13.219860+0000","last_unstale":"2024-03-21T13:26:52.269593+0000","last_undegraded":"2024-03-21T13:26:52.269593+0000","last_fullsized":"2024-03-21T13:26:52.269593+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:16:00.508810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"67'1","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.640150+0000","last_change":"2024-03-21T13:26:17.245836+0000","last_active":"2024-03-21T13:26:52.640150+0000","last_peered":"2024-03-21T13:26:52.640150+0000","last_clean":"2024-03-21T13:26:52.640150+0000","last_became_active":"2024-03-21T13:26:17.245515+0000","last_became_peered":"2024-03-21T13:26:17.245515+0000","last_unstale":"2024-03-21T13:26:52.640150+0000","last_undegraded":"2024-03-21T13:26:52.640150+0000","last_fullsized":"2024-03-21T13:26:52.640150+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:06:23.562361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.374715+0000","last_change":"2024-03-21T13:26:15.232911+0000","last_active":"2024-03-21T13:26:52.374715+0000","last_peered":"2024-03-21T13:26:52.374715+0000","last_clean":"2024-03-21T13:26:52.374715+0000","last_became_active":"2024-03-21T13:26:15.232533+0000","last_became_peered":"2024-03-21T13:26:15.232533+0000","last_unstale":"2024-03-21T13:26:52.374715+0000","last_undegraded":"2024-03-21T13:26:52.374715+0000","last_fullsized":"2024-03-21T13:26:52.374715+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:41:14.000034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.374730+0000","last_change":"2024-03-21T13:26:11.200564+0000","last_active":"2024-03-21T13:26:52.374730+0000","last_peered":"2024-03-21T13:26:52.374730+0000","last_clean":"2024-03-21T13:26:52.374730+0000","last_became_active":"2024-03-21T13:26:11.200243+0000","last_became_peered":"2024-03-21T13:26:11.200243+0000","last_unstale":"2024-03-21T13:26:52.374730+0000","last_undegraded":"2024-03-21T13:26:52.374730+0000","last_fullsized":"2024-03-21T13:26:52.374730+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T00:42:33.426721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"67'9","reported_seq":45,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.271361+0000","last_change":"2024-03-21T13:26:13.225317+0000","last_active":"2024-03-21T13:26:52.271361+0000","last_peered":"2024-03-21T13:26:52.271361+0000","last_clean":"2024-03-21T13:26:52.271361+0000","last_became_active":"2024-03-21T13:26:13.225035+0000","last_became_peered":"2024-03-21T13:26:13.225035+0000","last_unstale":"2024-03-21T13:26:52.271361+0000","last_undegraded":"2024-03-21T13:26:52.271361+0000","last_fullsized":"2024-03-21T13:26:52.271361+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:07:04.065673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.374399+0000","last_change":"2024-03-21T13:26:17.249189+0000","last_active":"2024-03-21T13:26:52.374399+0000","last_peered":"2024-03-21T13:26:52.374399+0000","last_clean":"2024-03-21T13:26:52.374399+0000","last_became_active":"2024-03-21T13:26:17.248897+0000","last_became_peered":"2024-03-21T13:26:17.248897+0000","last_unstale":"2024-03-21T13:26:52.374399+0000","last_undegraded":"2024-03-21T13:26:52.374399+0000","last_fullsized":"2024-03-21T13:26:52.374399+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:08:14.907504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":24,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.386060+0000","last_change":"2024-03-21T13:26:15.240073+0000","last_active":"2024-03-21T13:26:52.386060+0000","last_peered":"2024-03-21T13:26:52.386060+0000","last_clean":"2024-03-21T13:26:52.386060+0000","last_became_active":"2024-03-21T13:26:15.239506+0000","last_became_peered":"2024-03-21T13:26:15.239506+0000","last_unstale":"2024-03-21T13:26:52.386060+0000","last_undegraded":"2024-03-21T13:26:52.386060+0000","last_fullsized":"2024-03-21T13:26:52.386060+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:56:55.889404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.374679+0000","last_change":"2024-03-21T13:26:11.199114+0000","last_active":"2024-03-21T13:26:52.374679+0000","last_peered":"2024-03-21T13:26:52.374679+0000","last_clean":"2024-03-21T13:26:52.374679+0000","last_became_active":"2024-03-21T13:26:11.198797+0000","last_became_peered":"2024-03-21T13:26:11.198797+0000","last_unstale":"2024-03-21T13:26:52.374679+0000","last_undegraded":"2024-03-21T13:26:52.374679+0000","last_fullsized":"2024-03-21T13:26:52.374679+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:30:07.486345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"67'5","reported_seq":39,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.272082+0000","last_change":"2024-03-21T13:26:13.224599+0000","last_active":"2024-03-21T13:26:52.272082+0000","last_peered":"2024-03-21T13:26:52.272082+0000","last_clean":"2024-03-21T13:26:52.272082+0000","last_became_active":"2024-03-21T13:26:13.224317+0000","last_became_peered":"2024-03-21T13:26:13.224317+0000","last_unstale":"2024-03-21T13:26:52.272082+0000","last_undegraded":"2024-03-21T13:26:52.272082+0000","last_fullsized":"2024-03-21T13:26:52.272082+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:07:34.516161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.272413+0000","last_change":"2024-03-21T13:26:17.305168+0000","last_active":"2024-03-21T13:26:52.272413+0000","last_peered":"2024-03-21T13:26:52.272413+0000","last_clean":"2024-03-21T13:26:52.272413+0000","last_became_active":"2024-03-21T13:26:17.304862+0000","last_became_peered":"2024-03-21T13:26:17.304862+0000","last_unstale":"2024-03-21T13:26:52.272413+0000","last_undegraded":"2024-03-21T13:26:52.272413+0000","last_fullsized":"2024-03-21T13:26:52.272413+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:56:34.232516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.640538+0000","last_change":"2024-03-21T13:26:15.227623+0000","last_active":"2024-03-21T13:26:52.640538+0000","last_peered":"2024-03-21T13:26:52.640538+0000","last_clean":"2024-03-21T13:26:52.640538+0000","last_became_active":"2024-03-21T13:26:15.227313+0000","last_became_peered":"2024-03-21T13:26:15.227313+0000","last_unstale":"2024-03-21T13:26:52.640538+0000","last_undegraded":"2024-03-21T13:26:52.640538+0000","last_fullsized":"2024-03-21T13:26:52.640538+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:50:33.810070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.641077+0000","last_change":"2024-03-21T13:26:11.201065+0000","last_active":"2024-03-21T13:26:52.641077+0000","last_peered":"2024-03-21T13:26:52.641077+0000","last_clean":"2024-03-21T13:26:52.641077+0000","last_became_active":"2024-03-21T13:26:11.200781+0000","last_became_peered":"2024-03-21T13:26:11.200781+0000","last_unstale":"2024-03-21T13:26:52.641077+0000","last_undegraded":"2024-03-21T13:26:52.641077+0000","last_fullsized":"2024-03-21T13:26:52.641077+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:05:36.701771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"67'15","reported_seq":54,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.382956+0000","last_change":"2024-03-21T13:26:13.224126+0000","last_active":"2024-03-21T13:26:52.382956+0000","last_peered":"2024-03-21T13:26:52.382956+0000","last_clean":"2024-03-21T13:26:52.382956+0000","last_became_active":"2024-03-21T13:26:13.223765+0000","last_became_peered":"2024-03-21T13:26:13.223765+0000","last_unstale":"2024-03-21T13:26:52.382956+0000","last_undegraded":"2024-03-21T13:26:52.382956+0000","last_fullsized":"2024-03-21T13:26:52.382956+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:16:41.903823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.271271+0000","last_change":"2024-03-21T13:26:17.305631+0000","last_active":"2024-03-21T13:26:52.271271+0000","last_peered":"2024-03-21T13:26:52.271271+0000","last_clean":"2024-03-21T13:26:52.271271+0000","last_became_active":"2024-03-21T13:26:17.305358+0000","last_became_peered":"2024-03-21T13:26:17.305358+0000","last_unstale":"2024-03-21T13:26:52.271271+0000","last_undegraded":"2024-03-21T13:26:52.271271+0000","last_fullsized":"2024-03-21T13:26:52.271271+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:20:57.295263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.271808+0000","last_change":"2024-03-21T13:26:15.230491+0000","last_active":"2024-03-21T13:26:52.271808+0000","last_peered":"2024-03-21T13:26:52.271808+0000","last_clean":"2024-03-21T13:26:52.271808+0000","last_became_active":"2024-03-21T13:26:15.229859+0000","last_became_peered":"2024-03-21T13:26:15.229859+0000","last_unstale":"2024-03-21T13:26:52.271808+0000","last_undegraded":"2024-03-21T13:26:52.271808+0000","last_fullsized":"2024-03-21T13:26:52.271808+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T00:26:02.981353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.271526+0000","last_change":"2024-03-21T13:26:11.194612+0000","last_active":"2024-03-21T13:26:52.271526+0000","last_peered":"2024-03-21T13:26:52.271526+0000","last_clean":"2024-03-21T13:26:52.271526+0000","last_became_active":"2024-03-21T13:26:11.194332+0000","last_became_peered":"2024-03-21T13:26:11.194332+0000","last_unstale":"2024-03-21T13:26:52.271526+0000","last_undegraded":"2024-03-21T13:26:52.271526+0000","last_fullsized":"2024-03-21T13:26:52.271526+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:28:26.556731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"67'12","reported_seq":70,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.270788+0000","last_change":"2024-03-21T13:26:32.433066+0000","last_active":"2024-03-21T13:26:52.270788+0000","last_peered":"2024-03-21T13:26:52.270788+0000","last_clean":"2024-03-21T13:26:52.270788+0000","last_became_active":"2024-03-21T13:26:32.417390+0000","last_became_peered":"2024-03-21T13:26:32.417390+0000","last_unstale":"2024-03-21T13:26:52.270788+0000","last_undegraded":"2024-03-21T13:26:52.270788+0000","last_fullsized":"2024-03-21T13:26:52.270788+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:51:55.210341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"67'1","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.386524+0000","last_change":"2024-03-21T13:26:17.252404+0000","last_active":"2024-03-21T13:26:52.386524+0000","last_peered":"2024-03-21T13:26:52.386524+0000","last_clean":"2024-03-21T13:26:52.386524+0000","last_became_active":"2024-03-21T13:26:17.252048+0000","last_became_peered":"2024-03-21T13:26:17.252048+0000","last_unstale":"2024-03-21T13:26:52.386524+0000","last_undegraded":"2024-03-21T13:26:52.386524+0000","last_fullsized":"2024-03-21T13:26:52.386524+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:18:43.223987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":370,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"67'8","reported_seq":53,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:27:26.251698+0000","last_change":"2024-03-21T13:26:15.233894+0000","last_active":"2024-03-21T13:27:26.251698+0000","last_peered":"2024-03-21T13:27:26.251698+0000","last_clean":"2024-03-21T13:27:26.251698+0000","last_became_active":"2024-03-21T13:26:15.233338+0000","last_became_peered":"2024-03-21T13:26:15.233338+0000","last_unstale":"2024-03-21T13:27:26.251698+0000","last_undegraded":"2024-03-21T13:27:26.251698+0000","last_fullsized":"2024-03-21T13:27:26.251698+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:46:08.680052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"60'1","reported_seq":34,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.639844+0000","last_change":"2024-03-21T13:26:11.196283+0000","last_active":"2024-03-21T13:26:52.639844+0000","last_peered":"2024-03-21T13:26:52.639844+0000","last_clean":"2024-03-21T13:26:52.639844+0000","last_became_active":"2024-03-21T13:26:11.195957+0000","last_became_peered":"2024-03-21T13:26:11.195957+0000","last_unstale":"2024-03-21T13:26:52.639844+0000","last_undegraded":"2024-03-21T13:26:52.639844+0000","last_fullsized":"2024-03-21T13:26:52.639844+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:22:55.172360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"67'10","reported_seq":44,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.640943+0000","last_change":"2024-03-21T13:26:13.219274+0000","last_active":"2024-03-21T13:26:52.640943+0000","last_peered":"2024-03-21T13:26:52.640943+0000","last_clean":"2024-03-21T13:26:52.640943+0000","last_became_active":"2024-03-21T13:26:13.218535+0000","last_became_peered":"2024-03-21T13:26:13.218535+0000","last_unstale":"2024-03-21T13:26:52.640943+0000","last_undegraded":"2024-03-21T13:26:52.640943+0000","last_fullsized":"2024-03-21T13:26:52.640943+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:20:31.477672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.641582+0000","last_change":"2024-03-21T13:26:17.244005+0000","last_active":"2024-03-21T13:26:52.641582+0000","last_peered":"2024-03-21T13:26:52.641582+0000","last_clean":"2024-03-21T13:26:52.641582+0000","last_became_active":"2024-03-21T13:26:17.243691+0000","last_became_peered":"2024-03-21T13:26:17.243691+0000","last_unstale":"2024-03-21T13:26:52.641582+0000","last_undegraded":"2024-03-21T13:26:52.641582+0000","last_fullsized":"2024-03-21T13:26:52.641582+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T00:49:51.167480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.641132+0000","last_change":"2024-03-21T13:26:15.227555+0000","last_active":"2024-03-21T13:26:52.641132+0000","last_peered":"2024-03-21T13:26:52.641132+0000","last_clean":"2024-03-21T13:26:52.641132+0000","last_became_active":"2024-03-21T13:26:15.227222+0000","last_became_peered":"2024-03-21T13:26:15.227222+0000","last_unstale":"2024-03-21T13:26:52.641132+0000","last_undegraded":"2024-03-21T13:26:52.641132+0000","last_fullsized":"2024-03-21T13:26:52.641132+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:29:30.251861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.269051+0000","last_change":"2024-03-21T13:26:11.196773+0000","last_active":"2024-03-21T13:26:52.269051+0000","last_peered":"2024-03-21T13:26:52.269051+0000","last_clean":"2024-03-21T13:26:52.269051+0000","last_became_active":"2024-03-21T13:26:11.196214+0000","last_became_peered":"2024-03-21T13:26:11.196214+0000","last_unstale":"2024-03-21T13:26:52.269051+0000","last_undegraded":"2024-03-21T13:26:52.269051+0000","last_fullsized":"2024-03-21T13:26:52.269051+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T00:43:01.860805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"67'11","reported_seq":48,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.547554+0000","last_change":"2024-03-21T13:26:13.223859+0000","last_active":"2024-03-21T13:26:52.547554+0000","last_peered":"2024-03-21T13:26:52.547554+0000","last_clean":"2024-03-21T13:26:52.547554+0000","last_became_active":"2024-03-21T13:26:13.223564+0000","last_became_peered":"2024-03-21T13:26:13.223564+0000","last_unstale":"2024-03-21T13:26:52.547554+0000","last_undegraded":"2024-03-21T13:26:52.547554+0000","last_fullsized":"2024-03-21T13:26:52.547554+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:35:22.228078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":20,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.386032+0000","last_change":"2024-03-21T13:26:17.252364+0000","last_active":"2024-03-21T13:26:52.386032+0000","last_peered":"2024-03-21T13:26:52.386032+0000","last_clean":"2024-03-21T13:26:52.386032+0000","last_became_active":"2024-03-21T13:26:17.252073+0000","last_became_peered":"2024-03-21T13:26:17.252073+0000","last_unstale":"2024-03-21T13:26:52.386032+0000","last_undegraded":"2024-03-21T13:26:52.386032+0000","last_fullsized":"2024-03-21T13:26:52.386032+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T01:24:36.811892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.546922+0000","last_change":"2024-03-21T13:26:15.232697+0000","last_active":"2024-03-21T13:26:52.546922+0000","last_peered":"2024-03-21T13:26:52.546922+0000","last_clean":"2024-03-21T13:26:52.546922+0000","last_became_active":"2024-03-21T13:26:15.232208+0000","last_became_peered":"2024-03-21T13:26:15.232208+0000","last_unstale":"2024-03-21T13:26:52.546922+0000","last_undegraded":"2024-03-21T13:26:52.546922+0000","last_fullsized":"2024-03-21T13:26:52.546922+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:18:56.080086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.640282+0000","last_change":"2024-03-21T13:26:11.200672+0000","last_active":"2024-03-21T13:26:52.640282+0000","last_peered":"2024-03-21T13:26:52.640282+0000","last_clean":"2024-03-21T13:26:52.640282+0000","last_became_active":"2024-03-21T13:26:11.200402+0000","last_became_peered":"2024-03-21T13:26:11.200402+0000","last_unstale":"2024-03-21T13:26:52.640282+0000","last_undegraded":"2024-03-21T13:26:52.640282+0000","last_fullsized":"2024-03-21T13:26:52.640282+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:19:56.865749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"67'4","reported_seq":35,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.269084+0000","last_change":"2024-03-21T13:26:13.223883+0000","last_active":"2024-03-21T13:26:52.269084+0000","last_peered":"2024-03-21T13:26:52.269084+0000","last_clean":"2024-03-21T13:26:52.269084+0000","last_became_active":"2024-03-21T13:26:13.223548+0000","last_became_peered":"2024-03-21T13:26:13.223548+0000","last_unstale":"2024-03-21T13:26:52.269084+0000","last_undegraded":"2024-03-21T13:26:52.269084+0000","last_fullsized":"2024-03-21T13:26:52.269084+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:05:30.889217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.269509+0000","last_change":"2024-03-21T13:26:17.307158+0000","last_active":"2024-03-21T13:26:52.269509+0000","last_peered":"2024-03-21T13:26:52.269509+0000","last_clean":"2024-03-21T13:26:52.269509+0000","last_became_active":"2024-03-21T13:26:17.306160+0000","last_became_peered":"2024-03-21T13:26:17.306160+0000","last_unstale":"2024-03-21T13:26:52.269509+0000","last_undegraded":"2024-03-21T13:26:52.269509+0000","last_fullsized":"2024-03-21T13:26:52.269509+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T00:17:20.173596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":24,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.386303+0000","last_change":"2024-03-21T13:26:15.233113+0000","last_active":"2024-03-21T13:26:52.386303+0000","last_peered":"2024-03-21T13:26:52.386303+0000","last_clean":"2024-03-21T13:26:52.386303+0000","last_became_active":"2024-03-21T13:26:15.232761+0000","last_became_peered":"2024-03-21T13:26:15.232761+0000","last_unstale":"2024-03-21T13:26:52.386303+0000","last_undegraded":"2024-03-21T13:26:52.386303+0000","last_fullsized":"2024-03-21T13:26:52.386303+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:40:41.443170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.374978+0000","last_change":"2024-03-21T13:26:11.197392+0000","last_active":"2024-03-21T13:26:52.374978+0000","last_peered":"2024-03-21T13:26:52.374978+0000","last_clean":"2024-03-21T13:26:52.374978+0000","last_became_active":"2024-03-21T13:26:11.197083+0000","last_became_peered":"2024-03-21T13:26:11.197083+0000","last_unstale":"2024-03-21T13:26:52.374978+0000","last_undegraded":"2024-03-21T13:26:52.374978+0000","last_fullsized":"2024-03-21T13:26:52.374978+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:15:23.542845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"67'11","reported_seq":48,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.269550+0000","last_change":"2024-03-21T13:26:13.221030+0000","last_active":"2024-03-21T13:26:52.269550+0000","last_peered":"2024-03-21T13:26:52.269550+0000","last_clean":"2024-03-21T13:26:52.269550+0000","last_became_active":"2024-03-21T13:26:13.220710+0000","last_became_peered":"2024-03-21T13:26:13.220710+0000","last_unstale":"2024-03-21T13:26:52.269550+0000","last_undegraded":"2024-03-21T13:26:52.269550+0000","last_fullsized":"2024-03-21T13:26:52.269550+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:40:46.842543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.639195+0000","last_change":"2024-03-21T13:26:17.245884+0000","last_active":"2024-03-21T13:26:52.639195+0000","last_peered":"2024-03-21T13:26:52.639195+0000","last_clean":"2024-03-21T13:26:52.639195+0000","last_became_active":"2024-03-21T13:26:17.245564+0000","last_became_peered":"2024-03-21T13:26:17.245564+0000","last_unstale":"2024-03-21T13:26:52.639195+0000","last_undegraded":"2024-03-21T13:26:52.639195+0000","last_fullsized":"2024-03-21T13:26:52.639195+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:26:57.034878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.269828+0000","last_change":"2024-03-21T13:26:15.229227+0000","last_active":"2024-03-21T13:26:52.269828+0000","last_peered":"2024-03-21T13:26:52.269828+0000","last_clean":"2024-03-21T13:26:52.269828+0000","last_became_active":"2024-03-21T13:26:15.228860+0000","last_became_peered":"2024-03-21T13:26:15.228860+0000","last_unstale":"2024-03-21T13:26:52.269828+0000","last_undegraded":"2024-03-21T13:26:52.269828+0000","last_fullsized":"2024-03-21T13:26:52.269828+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:44:39.938958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":32,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.386521+0000","last_change":"2024-03-21T13:26:11.202322+0000","last_active":"2024-03-21T13:26:52.386521+0000","last_peered":"2024-03-21T13:26:52.386521+0000","last_clean":"2024-03-21T13:26:52.386521+0000","last_became_active":"2024-03-21T13:26:11.202018+0000","last_became_peered":"2024-03-21T13:26:11.202018+0000","last_unstale":"2024-03-21T13:26:52.386521+0000","last_undegraded":"2024-03-21T13:26:52.386521+0000","last_fullsized":"2024-03-21T13:26:52.386521+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:47:07.061477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"67'9","reported_seq":45,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.641698+0000","last_change":"2024-03-21T13:26:13.223870+0000","last_active":"2024-03-21T13:26:52.641698+0000","last_peered":"2024-03-21T13:26:52.641698+0000","last_clean":"2024-03-21T13:26:52.641698+0000","last_became_active":"2024-03-21T13:26:13.223536+0000","last_became_peered":"2024-03-21T13:26:13.223536+0000","last_unstale":"2024-03-21T13:26:52.641698+0000","last_undegraded":"2024-03-21T13:26:52.641698+0000","last_fullsized":"2024-03-21T13:26:52.641698+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:00:43.711112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.269200+0000","last_change":"2024-03-21T13:26:17.247820+0000","last_active":"2024-03-21T13:26:52.269200+0000","last_peered":"2024-03-21T13:26:52.269200+0000","last_clean":"2024-03-21T13:26:52.269200+0000","last_became_active":"2024-03-21T13:26:17.247405+0000","last_became_peered":"2024-03-21T13:26:17.247405+0000","last_unstale":"2024-03-21T13:26:52.269200+0000","last_undegraded":"2024-03-21T13:26:52.269200+0000","last_fullsized":"2024-03-21T13:26:52.269200+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:43:44.382313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.640084+0000","last_change":"2024-03-21T13:26:15.234664+0000","last_active":"2024-03-21T13:26:52.640084+0000","last_peered":"2024-03-21T13:26:52.640084+0000","last_clean":"2024-03-21T13:26:52.640084+0000","last_became_active":"2024-03-21T13:26:15.234007+0000","last_became_peered":"2024-03-21T13:26:15.234007+0000","last_unstale":"2024-03-21T13:26:52.640084+0000","last_undegraded":"2024-03-21T13:26:52.640084+0000","last_fullsized":"2024-03-21T13:26:52.640084+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:54:30.608977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.272102+0000","last_change":"2024-03-21T13:26:11.198059+0000","last_active":"2024-03-21T13:26:52.272102+0000","last_peered":"2024-03-21T13:26:52.272102+0000","last_clean":"2024-03-21T13:26:52.272102+0000","last_became_active":"2024-03-21T13:26:11.197786+0000","last_became_peered":"2024-03-21T13:26:11.197786+0000","last_unstale":"2024-03-21T13:26:52.272102+0000","last_undegraded":"2024-03-21T13:26:52.272102+0000","last_fullsized":"2024-03-21T13:26:52.272102+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T00:27:48.584970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"67'11","reported_seq":48,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.271901+0000","last_change":"2024-03-21T13:26:13.223122+0000","last_active":"2024-03-21T13:26:52.271901+0000","last_peered":"2024-03-21T13:26:52.271901+0000","last_clean":"2024-03-21T13:26:52.271901+0000","last_became_active":"2024-03-21T13:26:13.222859+0000","last_became_peered":"2024-03-21T13:26:13.222859+0000","last_unstale":"2024-03-21T13:26:52.271901+0000","last_undegraded":"2024-03-21T13:26:52.271901+0000","last_fullsized":"2024-03-21T13:26:52.271901+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:27:49.056642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.640703+0000","last_change":"2024-03-21T13:26:17.249015+0000","last_active":"2024-03-21T13:26:52.640703+0000","last_peered":"2024-03-21T13:26:52.640703+0000","last_clean":"2024-03-21T13:26:52.640703+0000","last_became_active":"2024-03-21T13:26:17.248654+0000","last_became_peered":"2024-03-21T13:26:17.248654+0000","last_unstale":"2024-03-21T13:26:52.640703+0000","last_undegraded":"2024-03-21T13:26:52.640703+0000","last_fullsized":"2024-03-21T13:26:52.640703+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:09:06.584717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"67'8","reported_seq":50,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:27:26.251698+0000","last_change":"2024-03-21T13:26:15.233021+0000","last_active":"2024-03-21T13:27:26.251698+0000","last_peered":"2024-03-21T13:27:26.251698+0000","last_clean":"2024-03-21T13:27:26.251698+0000","last_became_active":"2024-03-21T13:26:15.232194+0000","last_became_peered":"2024-03-21T13:26:15.232194+0000","last_unstale":"2024-03-21T13:27:26.251698+0000","last_undegraded":"2024-03-21T13:27:26.251698+0000","last_fullsized":"2024-03-21T13:27:26.251698+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T00:44:08.125960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":32,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.386105+0000","last_change":"2024-03-21T13:26:11.196875+0000","last_active":"2024-03-21T13:26:52.386105+0000","last_peered":"2024-03-21T13:26:52.386105+0000","last_clean":"2024-03-21T13:26:52.386105+0000","last_became_active":"2024-03-21T13:26:11.196574+0000","last_became_peered":"2024-03-21T13:26:11.196574+0000","last_unstale":"2024-03-21T13:26:52.386105+0000","last_undegraded":"2024-03-21T13:26:52.386105+0000","last_fullsized":"2024-03-21T13:26:52.386105+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T01:09:34.867011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"67'10","reported_seq":44,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.269800+0000","last_change":"2024-03-21T13:26:13.228578+0000","last_active":"2024-03-21T13:26:52.269800+0000","last_peered":"2024-03-21T13:26:52.269800+0000","last_clean":"2024-03-21T13:26:52.269800+0000","last_became_active":"2024-03-21T13:26:13.227999+0000","last_became_peered":"2024-03-21T13:26:13.227999+0000","last_unstale":"2024-03-21T13:26:52.269800+0000","last_undegraded":"2024-03-21T13:26:52.269800+0000","last_fullsized":"2024-03-21T13:26:52.269800+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:09:24.782441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.272590+0000","last_change":"2024-03-21T13:26:17.248927+0000","last_active":"2024-03-21T13:26:52.272590+0000","last_peered":"2024-03-21T13:26:52.272590+0000","last_clean":"2024-03-21T13:26:52.272590+0000","last_became_active":"2024-03-21T13:26:17.248348+0000","last_became_peered":"2024-03-21T13:26:17.248348+0000","last_unstale":"2024-03-21T13:26:52.272590+0000","last_undegraded":"2024-03-21T13:26:52.272590+0000","last_fullsized":"2024-03-21T13:26:52.272590+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:35:45.907943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"67'8","reported_seq":50,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:27:26.251589+0000","last_change":"2024-03-21T13:26:15.237341+0000","last_active":"2024-03-21T13:27:26.251589+0000","last_peered":"2024-03-21T13:27:26.251589+0000","last_clean":"2024-03-21T13:27:26.251589+0000","last_became_active":"2024-03-21T13:26:15.237025+0000","last_became_peered":"2024-03-21T13:26:15.237025+0000","last_unstale":"2024-03-21T13:27:26.251589+0000","last_undegraded":"2024-03-21T13:27:26.251589+0000","last_fullsized":"2024-03-21T13:27:26.251589+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:09:19.571483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.640598+0000","last_change":"2024-03-21T13:26:11.196835+0000","last_active":"2024-03-21T13:26:52.640598+0000","last_peered":"2024-03-21T13:26:52.640598+0000","last_clean":"2024-03-21T13:26:52.640598+0000","last_became_active":"2024-03-21T13:26:11.196462+0000","last_became_peered":"2024-03-21T13:26:11.196462+0000","last_unstale":"2024-03-21T13:26:52.640598+0000","last_undegraded":"2024-03-21T13:26:52.640598+0000","last_fullsized":"2024-03-21T13:26:52.640598+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:08:34.618745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"67'9","reported_seq":45,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.375063+0000","last_change":"2024-03-21T13:26:13.220137+0000","last_active":"2024-03-21T13:26:52.375063+0000","last_peered":"2024-03-21T13:26:52.375063+0000","last_clean":"2024-03-21T13:26:52.375063+0000","last_became_active":"2024-03-21T13:26:13.219824+0000","last_became_peered":"2024-03-21T13:26:13.219824+0000","last_unstale":"2024-03-21T13:26:52.375063+0000","last_undegraded":"2024-03-21T13:26:52.375063+0000","last_fullsized":"2024-03-21T13:26:52.375063+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T00:46:53.170532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.383131+0000","last_change":"2024-03-21T13:26:17.252431+0000","last_active":"2024-03-21T13:26:52.383131+0000","last_peered":"2024-03-21T13:26:52.383131+0000","last_clean":"2024-03-21T13:26:52.383131+0000","last_became_active":"2024-03-21T13:26:17.252059+0000","last_became_peered":"2024-03-21T13:26:17.252059+0000","last_unstale":"2024-03-21T13:26:52.383131+0000","last_undegraded":"2024-03-21T13:26:52.383131+0000","last_fullsized":"2024-03-21T13:26:52.383131+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:23:21.459141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.271185+0000","last_change":"2024-03-21T13:26:15.229589+0000","last_active":"2024-03-21T13:26:52.271185+0000","last_peered":"2024-03-21T13:26:52.271185+0000","last_clean":"2024-03-21T13:26:52.271185+0000","last_became_active":"2024-03-21T13:26:15.229128+0000","last_became_peered":"2024-03-21T13:26:15.229128+0000","last_unstale":"2024-03-21T13:26:52.271185+0000","last_undegraded":"2024-03-21T13:26:52.271185+0000","last_fullsized":"2024-03-21T13:26:52.271185+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:39:29.636968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":32,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.386377+0000","last_change":"2024-03-21T13:26:11.202357+0000","last_active":"2024-03-21T13:26:52.386377+0000","last_peered":"2024-03-21T13:26:52.386377+0000","last_clean":"2024-03-21T13:26:52.386377+0000","last_became_active":"2024-03-21T13:26:11.201668+0000","last_became_peered":"2024-03-21T13:26:11.201668+0000","last_unstale":"2024-03-21T13:26:52.386377+0000","last_undegraded":"2024-03-21T13:26:52.386377+0000","last_fullsized":"2024-03-21T13:26:52.386377+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:59:59.977918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"67'9","reported_seq":45,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.640996+0000","last_change":"2024-03-21T13:26:13.218912+0000","last_active":"2024-03-21T13:26:52.640996+0000","last_peered":"2024-03-21T13:26:52.640996+0000","last_clean":"2024-03-21T13:26:52.640996+0000","last_became_active":"2024-03-21T13:26:13.218090+0000","last_became_peered":"2024-03-21T13:26:13.218090+0000","last_unstale":"2024-03-21T13:26:52.640996+0000","last_undegraded":"2024-03-21T13:26:52.640996+0000","last_fullsized":"2024-03-21T13:26:52.640996+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:51:49.321539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":20,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.386476+0000","last_change":"2024-03-21T13:26:17.306715+0000","last_active":"2024-03-21T13:26:52.386476+0000","last_peered":"2024-03-21T13:26:52.386476+0000","last_clean":"2024-03-21T13:26:52.386476+0000","last_became_active":"2024-03-21T13:26:17.306408+0000","last_became_peered":"2024-03-21T13:26:17.306408+0000","last_unstale":"2024-03-21T13:26:52.386476+0000","last_undegraded":"2024-03-21T13:26:52.386476+0000","last_fullsized":"2024-03-21T13:26:52.386476+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:00:05.045057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.375008+0000","last_change":"2024-03-21T13:26:15.233948+0000","last_active":"2024-03-21T13:26:52.375008+0000","last_peered":"2024-03-21T13:26:52.375008+0000","last_clean":"2024-03-21T13:26:52.375008+0000","last_became_active":"2024-03-21T13:26:15.233657+0000","last_became_peered":"2024-03-21T13:26:15.233657+0000","last_unstale":"2024-03-21T13:26:52.375008+0000","last_undegraded":"2024-03-21T13:26:52.375008+0000","last_fullsized":"2024-03-21T13:26:52.375008+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:14:27.120444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.546875+0000","last_change":"2024-03-21T13:26:11.195449+0000","last_active":"2024-03-21T13:26:52.546875+0000","last_peered":"2024-03-21T13:26:52.546875+0000","last_clean":"2024-03-21T13:26:52.546875+0000","last_became_active":"2024-03-21T13:26:11.195164+0000","last_became_peered":"2024-03-21T13:26:11.195164+0000","last_unstale":"2024-03-21T13:26:52.546875+0000","last_undegraded":"2024-03-21T13:26:52.546875+0000","last_fullsized":"2024-03-21T13:26:52.546875+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:58:29.196507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"67'6","reported_seq":38,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.271359+0000","last_change":"2024-03-21T13:26:13.228326+0000","last_active":"2024-03-21T13:26:52.271359+0000","last_peered":"2024-03-21T13:26:52.271359+0000","last_clean":"2024-03-21T13:26:52.271359+0000","last_became_active":"2024-03-21T13:26:13.228006+0000","last_became_peered":"2024-03-21T13:26:13.228006+0000","last_unstale":"2024-03-21T13:26:52.271359+0000","last_undegraded":"2024-03-21T13:26:52.271359+0000","last_fullsized":"2024-03-21T13:26:52.271359+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:08:26.515863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.374690+0000","last_change":"2024-03-21T13:26:17.305291+0000","last_active":"2024-03-21T13:26:52.374690+0000","last_peered":"2024-03-21T13:26:52.374690+0000","last_clean":"2024-03-21T13:26:52.374690+0000","last_became_active":"2024-03-21T13:26:17.304979+0000","last_became_peered":"2024-03-21T13:26:17.304979+0000","last_unstale":"2024-03-21T13:26:52.374690+0000","last_undegraded":"2024-03-21T13:26:52.374690+0000","last_fullsized":"2024-03-21T13:26:52.374690+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:32:09.256312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"60'2","reported_seq":44,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.386865+0000","last_change":"2024-03-21T13:26:11.203158+0000","last_active":"2024-03-21T13:26:52.386865+0000","last_peered":"2024-03-21T13:26:52.386865+0000","last_clean":"2024-03-21T13:26:52.386865+0000","last_became_active":"2024-03-21T13:26:11.201706+0000","last_became_peered":"2024-03-21T13:26:11.201706+0000","last_unstale":"2024-03-21T13:26:52.386865+0000","last_undegraded":"2024-03-21T13:26:52.386865+0000","last_fullsized":"2024-03-21T13:26:52.386865+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:29:51.958157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"67'15","reported_seq":54,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.374431+0000","last_change":"2024-03-21T13:26:13.216022+0000","last_active":"2024-03-21T13:26:52.374431+0000","last_peered":"2024-03-21T13:26:52.374431+0000","last_clean":"2024-03-21T13:26:52.374431+0000","last_became_active":"2024-03-21T13:26:13.215749+0000","last_became_peered":"2024-03-21T13:26:13.215749+0000","last_unstale":"2024-03-21T13:26:52.374431+0000","last_undegraded":"2024-03-21T13:26:52.374431+0000","last_fullsized":"2024-03-21T13:26:52.374431+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:30:43.191153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"67'8","reported_seq":52,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:27:26.251707+0000","last_change":"2024-03-21T13:26:15.237504+0000","last_active":"2024-03-21T13:27:26.251707+0000","last_peered":"2024-03-21T13:27:26.251707+0000","last_clean":"2024-03-21T13:27:26.251707+0000","last_became_active":"2024-03-21T13:26:15.237182+0000","last_became_peered":"2024-03-21T13:26:15.237182+0000","last_unstale":"2024-03-21T13:27:26.251707+0000","last_undegraded":"2024-03-21T13:27:26.251707+0000","last_fullsized":"2024-03-21T13:27:26.251707+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T00:09:41.375653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.269616+0000","last_change":"2024-03-21T13:26:17.247720+0000","last_active":"2024-03-21T13:26:52.269616+0000","last_peered":"2024-03-21T13:26:52.269616+0000","last_clean":"2024-03-21T13:26:52.269616+0000","last_became_active":"2024-03-21T13:26:17.247360+0000","last_became_peered":"2024-03-21T13:26:17.247360+0000","last_unstale":"2024-03-21T13:26:52.269616+0000","last_undegraded":"2024-03-21T13:26:52.269616+0000","last_fullsized":"2024-03-21T13:26:52.269616+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:26:39.004847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":32,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.386880+0000","last_change":"2024-03-21T13:26:11.201159+0000","last_active":"2024-03-21T13:26:52.386880+0000","last_peered":"2024-03-21T13:26:52.386880+0000","last_clean":"2024-03-21T13:26:52.386880+0000","last_became_active":"2024-03-21T13:26:11.200889+0000","last_became_peered":"2024-03-21T13:26:11.200889+0000","last_unstale":"2024-03-21T13:26:52.386880+0000","last_undegraded":"2024-03-21T13:26:52.386880+0000","last_fullsized":"2024-03-21T13:26:52.386880+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:13:54.023383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"67'12","reported_seq":52,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.271287+0000","last_change":"2024-03-21T13:26:13.225296+0000","last_active":"2024-03-21T13:26:52.271287+0000","last_peered":"2024-03-21T13:26:52.271287+0000","last_clean":"2024-03-21T13:26:52.271287+0000","last_became_active":"2024-03-21T13:26:13.225038+0000","last_became_peered":"2024-03-21T13:26:13.225038+0000","last_unstale":"2024-03-21T13:26:52.271287+0000","last_undegraded":"2024-03-21T13:26:52.271287+0000","last_fullsized":"2024-03-21T13:26:52.271287+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:19:32.745497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.382321+0000","last_change":"2024-03-21T13:26:15.225181+0000","last_active":"2024-03-21T13:26:52.382321+0000","last_peered":"2024-03-21T13:26:52.382321+0000","last_clean":"2024-03-21T13:26:52.382321+0000","last_became_active":"2024-03-21T13:26:15.224882+0000","last_became_peered":"2024-03-21T13:26:15.224882+0000","last_unstale":"2024-03-21T13:26:52.382321+0000","last_undegraded":"2024-03-21T13:26:52.382321+0000","last_fullsized":"2024-03-21T13:26:52.382321+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:57:44.392944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":20,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.385885+0000","last_change":"2024-03-21T13:26:17.252287+0000","last_active":"2024-03-21T13:26:52.385885+0000","last_peered":"2024-03-21T13:26:52.385885+0000","last_clean":"2024-03-21T13:26:52.385885+0000","last_became_active":"2024-03-21T13:26:17.251966+0000","last_became_peered":"2024-03-21T13:26:17.251966+0000","last_unstale":"2024-03-21T13:26:52.385885+0000","last_undegraded":"2024-03-21T13:26:52.385885+0000","last_fullsized":"2024-03-21T13:26:52.385885+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T00:09:54.589933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.382798+0000","last_change":"2024-03-21T13:26:15.232352+0000","last_active":"2024-03-21T13:26:52.382798+0000","last_peered":"2024-03-21T13:26:52.382798+0000","last_clean":"2024-03-21T13:26:52.382798+0000","last_became_active":"2024-03-21T13:26:15.232036+0000","last_became_peered":"2024-03-21T13:26:15.232036+0000","last_unstale":"2024-03-21T13:26:52.382798+0000","last_undegraded":"2024-03-21T13:26:52.382798+0000","last_fullsized":"2024-03-21T13:26:52.382798+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:37:51.494333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":32,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.386740+0000","last_change":"2024-03-21T13:26:11.197526+0000","last_active":"2024-03-21T13:26:52.386740+0000","last_peered":"2024-03-21T13:26:52.386740+0000","last_clean":"2024-03-21T13:26:52.386740+0000","last_became_active":"2024-03-21T13:26:11.196727+0000","last_became_peered":"2024-03-21T13:26:11.196727+0000","last_unstale":"2024-03-21T13:26:52.386740+0000","last_undegraded":"2024-03-21T13:26:52.386740+0000","last_fullsized":"2024-03-21T13:26:52.386740+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:03:19.084737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"67'19","reported_seq":60,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.547520+0000","last_change":"2024-03-21T13:26:13.221628+0000","last_active":"2024-03-21T13:26:52.547520+0000","last_peered":"2024-03-21T13:26:52.547520+0000","last_clean":"2024-03-21T13:26:52.547520+0000","last_became_active":"2024-03-21T13:26:13.221351+0000","last_became_peered":"2024-03-21T13:26:13.221351+0000","last_unstale":"2024-03-21T13:26:52.547520+0000","last_undegraded":"2024-03-21T13:26:52.547520+0000","last_fullsized":"2024-03-21T13:26:52.547520+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:45:00.519834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.639298+0000","last_change":"2024-03-21T13:26:17.245249+0000","last_active":"2024-03-21T13:26:52.639298+0000","last_peered":"2024-03-21T13:26:52.639298+0000","last_clean":"2024-03-21T13:26:52.639298+0000","last_became_active":"2024-03-21T13:26:17.244930+0000","last_became_peered":"2024-03-21T13:26:17.244930+0000","last_unstale":"2024-03-21T13:26:52.639298+0000","last_undegraded":"2024-03-21T13:26:52.639298+0000","last_fullsized":"2024-03-21T13:26:52.639298+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:55:39.894721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.382540+0000","last_change":"2024-03-21T13:26:15.224697+0000","last_active":"2024-03-21T13:26:52.382540+0000","last_peered":"2024-03-21T13:26:52.382540+0000","last_clean":"2024-03-21T13:26:52.382540+0000","last_became_active":"2024-03-21T13:26:15.224322+0000","last_became_peered":"2024-03-21T13:26:15.224322+0000","last_unstale":"2024-03-21T13:26:52.382540+0000","last_undegraded":"2024-03-21T13:26:52.382540+0000","last_fullsized":"2024-03-21T13:26:52.382540+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T00:20:38.425097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.374725+0000","last_change":"2024-03-21T13:26:11.195530+0000","last_active":"2024-03-21T13:26:52.374725+0000","last_peered":"2024-03-21T13:26:52.374725+0000","last_clean":"2024-03-21T13:26:52.374725+0000","last_became_active":"2024-03-21T13:26:11.195248+0000","last_became_peered":"2024-03-21T13:26:11.195248+0000","last_unstale":"2024-03-21T13:26:52.374725+0000","last_undegraded":"2024-03-21T13:26:52.374725+0000","last_fullsized":"2024-03-21T13:26:52.374725+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:07:13.257027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"67'9","reported_seq":45,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.641214+0000","last_change":"2024-03-21T13:26:13.221547+0000","last_active":"2024-03-21T13:26:52.641214+0000","last_peered":"2024-03-21T13:26:52.641214+0000","last_clean":"2024-03-21T13:26:52.641214+0000","last_became_active":"2024-03-21T13:26:13.220184+0000","last_became_peered":"2024-03-21T13:26:13.220184+0000","last_unstale":"2024-03-21T13:26:52.641214+0000","last_undegraded":"2024-03-21T13:26:52.641214+0000","last_fullsized":"2024-03-21T13:26:52.641214+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:19:30.788571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.269682+0000","last_change":"2024-03-21T13:26:17.247824+0000","last_active":"2024-03-21T13:26:52.269682+0000","last_peered":"2024-03-21T13:26:52.269682+0000","last_clean":"2024-03-21T13:26:52.269682+0000","last_became_active":"2024-03-21T13:26:17.247506+0000","last_became_peered":"2024-03-21T13:26:17.247506+0000","last_unstale":"2024-03-21T13:26:52.269682+0000","last_undegraded":"2024-03-21T13:26:52.269682+0000","last_fullsized":"2024-03-21T13:26:52.269682+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T00:56:55.828142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"64'2","reported_seq":42,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.374516+0000","last_change":"2024-03-21T13:26:15.231433+0000","last_active":"2024-03-21T13:26:52.374516+0000","last_peered":"2024-03-21T13:26:52.374516+0000","last_clean":"2024-03-21T13:26:52.374516+0000","last_became_active":"2024-03-21T13:26:11.197652+0000","last_became_peered":"2024-03-21T13:26:11.197652+0000","last_unstale":"2024-03-21T13:26:52.374516+0000","last_undegraded":"2024-03-21T13:26:52.374516+0000","last_fullsized":"2024-03-21T13:26:52.374516+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T01:01:02.629415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004041551,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.271684+0000","last_change":"2024-03-21T13:26:11.199491+0000","last_active":"2024-03-21T13:26:52.271684+0000","last_peered":"2024-03-21T13:26:52.271684+0000","last_clean":"2024-03-21T13:26:52.271684+0000","last_became_active":"2024-03-21T13:26:11.199109+0000","last_became_peered":"2024-03-21T13:26:11.199109+0000","last_unstale":"2024-03-21T13:26:52.271684+0000","last_undegraded":"2024-03-21T13:26:52.271684+0000","last_fullsized":"2024-03-21T13:26:52.271684+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:50:27.710077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"67'28","reported_seq":94,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:27:26.251661+0000","last_change":"2024-03-21T13:26:13.224310+0000","last_active":"2024-03-21T13:27:26.251661+0000","last_peered":"2024-03-21T13:27:26.251661+0000","last_clean":"2024-03-21T13:27:26.251661+0000","last_became_active":"2024-03-21T13:26:13.223957+0000","last_became_peered":"2024-03-21T13:26:13.223957+0000","last_unstale":"2024-03-21T13:27:26.251661+0000","last_undegraded":"2024-03-21T13:27:26.251661+0000","last_fullsized":"2024-03-21T13:27:26.251661+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:58:13.081276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.639933+0000","last_change":"2024-03-21T13:26:15.225377+0000","last_active":"2024-03-21T13:26:52.639933+0000","last_peered":"2024-03-21T13:26:52.639933+0000","last_clean":"2024-03-21T13:26:52.639933+0000","last_became_active":"2024-03-21T13:26:15.225049+0000","last_became_peered":"2024-03-21T13:26:15.225049+0000","last_unstale":"2024-03-21T13:26:52.639933+0000","last_undegraded":"2024-03-21T13:26:52.639933+0000","last_fullsized":"2024-03-21T13:26:52.639933+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:05:34.989909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":20,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.386859+0000","last_change":"2024-03-21T13:26:17.306745+0000","last_active":"2024-03-21T13:26:52.386859+0000","last_peered":"2024-03-21T13:26:52.386859+0000","last_clean":"2024-03-21T13:26:52.386859+0000","last_became_active":"2024-03-21T13:26:17.306424+0000","last_became_peered":"2024-03-21T13:26:17.306424+0000","last_unstale":"2024-03-21T13:26:52.386859+0000","last_undegraded":"2024-03-21T13:26:52.386859+0000","last_fullsized":"2024-03-21T13:26:52.386859+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:56:49.516361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.640071+0000","last_change":"2024-03-21T13:26:11.194777+0000","last_active":"2024-03-21T13:26:52.640071+0000","last_peered":"2024-03-21T13:26:52.640071+0000","last_clean":"2024-03-21T13:26:52.640071+0000","last_became_active":"2024-03-21T13:26:11.194085+0000","last_became_peered":"2024-03-21T13:26:11.194085+0000","last_unstale":"2024-03-21T13:26:52.640071+0000","last_undegraded":"2024-03-21T13:26:52.640071+0000","last_fullsized":"2024-03-21T13:26:52.640071+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:20:59.362538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":40,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.382459+0000","last_change":"2024-03-21T13:26:15.215584+0000","last_active":"2024-03-21T13:26:52.382459+0000","last_peered":"2024-03-21T13:26:52.382459+0000","last_clean":"2024-03-21T13:26:52.382459+0000","last_became_active":"2024-03-21T13:26:11.195550+0000","last_became_peered":"2024-03-21T13:26:11.195550+0000","last_unstale":"2024-03-21T13:26:52.382459+0000","last_undegraded":"2024-03-21T13:26:52.382459+0000","last_fullsized":"2024-03-21T13:26:52.382459+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:11:07.000090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055458799999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"67'13","reported_seq":56,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.641370+0000","last_change":"2024-03-21T13:26:13.223129+0000","last_active":"2024-03-21T13:26:52.641370+0000","last_peered":"2024-03-21T13:26:52.641370+0000","last_clean":"2024-03-21T13:26:52.641370+0000","last_became_active":"2024-03-21T13:26:13.222689+0000","last_became_peered":"2024-03-21T13:26:13.222689+0000","last_unstale":"2024-03-21T13:26:52.641370+0000","last_undegraded":"2024-03-21T13:26:52.641370+0000","last_fullsized":"2024-03-21T13:26:52.641370+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:48:45.238762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.383136+0000","last_change":"2024-03-21T13:26:15.233122+0000","last_active":"2024-03-21T13:26:52.383136+0000","last_peered":"2024-03-21T13:26:52.383136+0000","last_clean":"2024-03-21T13:26:52.383136+0000","last_became_active":"2024-03-21T13:26:15.232807+0000","last_became_peered":"2024-03-21T13:26:15.232807+0000","last_unstale":"2024-03-21T13:26:52.383136+0000","last_undegraded":"2024-03-21T13:26:52.383136+0000","last_fullsized":"2024-03-21T13:26:52.383136+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:41:15.086830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.374944+0000","last_change":"2024-03-21T13:26:17.249433+0000","last_active":"2024-03-21T13:26:52.374944+0000","last_peered":"2024-03-21T13:26:52.374944+0000","last_clean":"2024-03-21T13:26:52.374944+0000","last_became_active":"2024-03-21T13:26:17.249153+0000","last_became_peered":"2024-03-21T13:26:17.249153+0000","last_unstale":"2024-03-21T13:26:52.374944+0000","last_undegraded":"2024-03-21T13:26:52.374944+0000","last_fullsized":"2024-03-21T13:26:52.374944+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:31:38.351777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"71'128","reported_seq":275,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:54.371886+0000","last_change":"2024-03-21T13:26:01.178260+0000","last_active":"2024-03-21T13:26:54.371886+0000","last_peered":"2024-03-21T13:26:54.371886+0000","last_clean":"2024-03-21T13:26:54.371886+0000","last_became_active":"2024-03-21T13:26:01.177752+0000","last_became_peered":"2024-03-21T13:26:01.177752+0000","last_unstale":"2024-03-21T13:26:54.371886+0000","last_undegraded":"2024-03-21T13:26:54.371886+0000","last_fullsized":"2024-03-21T13:26:54.371886+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:23:52.146116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:23:52.146116+0000","last_clean_scrub_stamp":"2024-03-21T13:23:52.146116+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T01:06:35.726853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.269533+0000","last_change":"2024-03-21T13:26:11.197631+0000","last_active":"2024-03-21T13:26:52.269533+0000","last_peered":"2024-03-21T13:26:52.269533+0000","last_clean":"2024-03-21T13:26:52.269533+0000","last_became_active":"2024-03-21T13:26:11.197291+0000","last_became_peered":"2024-03-21T13:26:11.197291+0000","last_unstale":"2024-03-21T13:26:52.269533+0000","last_undegraded":"2024-03-21T13:26:52.269533+0000","last_fullsized":"2024-03-21T13:26:52.269533+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:38:47.648305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"67'16","reported_seq":68,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:27:26.251992+0000","last_change":"2024-03-21T13:26:13.220005+0000","last_active":"2024-03-21T13:27:26.251992+0000","last_peered":"2024-03-21T13:27:26.251992+0000","last_clean":"2024-03-21T13:27:26.251992+0000","last_became_active":"2024-03-21T13:26:13.219283+0000","last_became_peered":"2024-03-21T13:26:13.219283+0000","last_unstale":"2024-03-21T13:27:26.251992+0000","last_undegraded":"2024-03-21T13:27:26.251992+0000","last_fullsized":"2024-03-21T13:27:26.251992+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:44:55.006479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":24,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.386709+0000","last_change":"2024-03-21T13:26:15.241005+0000","last_active":"2024-03-21T13:26:52.386709+0000","last_peered":"2024-03-21T13:26:52.386709+0000","last_clean":"2024-03-21T13:26:52.386709+0000","last_became_active":"2024-03-21T13:26:15.240661+0000","last_became_peered":"2024-03-21T13:26:15.240661+0000","last_unstale":"2024-03-21T13:26:52.386709+0000","last_undegraded":"2024-03-21T13:26:52.386709+0000","last_fullsized":"2024-03-21T13:26:52.386709+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:42:05.198556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.640429+0000","last_change":"2024-03-21T13:26:17.250469+0000","last_active":"2024-03-21T13:26:52.640429+0000","last_peered":"2024-03-21T13:26:52.640429+0000","last_clean":"2024-03-21T13:26:52.640429+0000","last_became_active":"2024-03-21T13:26:17.250144+0000","last_became_peered":"2024-03-21T13:26:17.250144+0000","last_unstale":"2024-03-21T13:26:52.640429+0000","last_undegraded":"2024-03-21T13:26:52.640429+0000","last_fullsized":"2024-03-21T13:26:52.640429+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:28:11.133824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.639497+0000","last_change":"2024-03-21T13:26:11.197464+0000","last_active":"2024-03-21T13:26:52.639497+0000","last_peered":"2024-03-21T13:26:52.639497+0000","last_clean":"2024-03-21T13:26:52.639497+0000","last_became_active":"2024-03-21T13:26:11.197172+0000","last_became_peered":"2024-03-21T13:26:11.197172+0000","last_unstale":"2024-03-21T13:26:52.639497+0000","last_undegraded":"2024-03-21T13:26:52.639497+0000","last_fullsized":"2024-03-21T13:26:52.639497+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:58:15.194164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"68'5","reported_seq":117,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:27:31.378320+0000","last_change":"2024-03-21T13:26:15.229957+0000","last_active":"2024-03-21T13:27:31.378320+0000","last_peered":"2024-03-21T13:27:31.378320+0000","last_clean":"2024-03-21T13:27:31.378320+0000","last_became_active":"2024-03-21T13:26:11.197128+0000","last_became_peered":"2024-03-21T13:26:11.197128+0000","last_unstale":"2024-03-21T13:27:31.378320+0000","last_undegraded":"2024-03-21T13:27:31.378320+0000","last_fullsized":"2024-03-21T13:27:31.378320+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:50:26.083584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044008600000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":75,"num_read_kb":70,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"67'12","reported_seq":67,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.640802+0000","last_change":"2024-03-21T13:26:32.425101+0000","last_active":"2024-03-21T13:26:52.640802+0000","last_peered":"2024-03-21T13:26:52.640802+0000","last_clean":"2024-03-21T13:26:52.640802+0000","last_became_active":"2024-03-21T13:26:32.415693+0000","last_became_peered":"2024-03-21T13:26:32.415693+0000","last_unstale":"2024-03-21T13:26:52.640802+0000","last_undegraded":"2024-03-21T13:26:52.640802+0000","last_fullsized":"2024-03-21T13:26:52.640802+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:23:28.199377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.375213+0000","last_change":"2024-03-21T13:26:15.238216+0000","last_active":"2024-03-21T13:26:52.375213+0000","last_peered":"2024-03-21T13:26:52.375213+0000","last_clean":"2024-03-21T13:26:52.375213+0000","last_became_active":"2024-03-21T13:26:15.237907+0000","last_became_peered":"2024-03-21T13:26:15.237907+0000","last_unstale":"2024-03-21T13:26:52.375213+0000","last_undegraded":"2024-03-21T13:26:52.375213+0000","last_fullsized":"2024-03-21T13:26:52.375213+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:05:23.428509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.639637+0000","last_change":"2024-03-21T13:26:17.307080+0000","last_active":"2024-03-21T13:26:52.639637+0000","last_peered":"2024-03-21T13:26:52.639637+0000","last_clean":"2024-03-21T13:26:52.639637+0000","last_became_active":"2024-03-21T13:26:17.306454+0000","last_became_peered":"2024-03-21T13:26:17.306454+0000","last_unstale":"2024-03-21T13:26:52.639637+0000","last_undegraded":"2024-03-21T13:26:52.639637+0000","last_fullsized":"2024-03-21T13:26:52.639637+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:29:41.493818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"60'1","reported_seq":39,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.640749+0000","last_change":"2024-03-21T13:26:11.199094+0000","last_active":"2024-03-21T13:26:52.640749+0000","last_peered":"2024-03-21T13:26:52.640749+0000","last_clean":"2024-03-21T13:26:52.640749+0000","last_became_active":"2024-03-21T13:26:11.198739+0000","last_became_peered":"2024-03-21T13:26:11.198739+0000","last_unstale":"2024-03-21T13:26:52.640749+0000","last_undegraded":"2024-03-21T13:26:52.640749+0000","last_fullsized":"2024-03-21T13:26:52.640749+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:03:15.816625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":911,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"67'19","reported_seq":65,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.640934+0000","last_change":"2024-03-21T13:26:13.215770+0000","last_active":"2024-03-21T13:26:52.640934+0000","last_peered":"2024-03-21T13:26:52.640934+0000","last_clean":"2024-03-21T13:26:52.640934+0000","last_became_active":"2024-03-21T13:26:13.215343+0000","last_became_peered":"2024-03-21T13:26:13.215343+0000","last_unstale":"2024-03-21T13:26:52.640934+0000","last_undegraded":"2024-03-21T13:26:52.640934+0000","last_fullsized":"2024-03-21T13:26:52.640934+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:57:05.065089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.547563+0000","last_change":"2024-03-21T13:26:15.234129+0000","last_active":"2024-03-21T13:26:52.547563+0000","last_peered":"2024-03-21T13:26:52.547563+0000","last_clean":"2024-03-21T13:26:52.547563+0000","last_became_active":"2024-03-21T13:26:15.233803+0000","last_became_peered":"2024-03-21T13:26:15.233803+0000","last_unstale":"2024-03-21T13:26:52.547563+0000","last_undegraded":"2024-03-21T13:26:52.547563+0000","last_fullsized":"2024-03-21T13:26:52.547563+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:15:35.028442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.639204+0000","last_change":"2024-03-21T13:26:17.249942+0000","last_active":"2024-03-21T13:26:52.639204+0000","last_peered":"2024-03-21T13:26:52.639204+0000","last_clean":"2024-03-21T13:26:52.639204+0000","last_became_active":"2024-03-21T13:26:17.249616+0000","last_became_peered":"2024-03-21T13:26:17.249616+0000","last_unstale":"2024-03-21T13:26:52.639204+0000","last_undegraded":"2024-03-21T13:26:52.639204+0000","last_fullsized":"2024-03-21T13:26:52.639204+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:26:42.140259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.374872+0000","last_change":"2024-03-21T13:26:11.195031+0000","last_active":"2024-03-21T13:26:52.374872+0000","last_peered":"2024-03-21T13:26:52.374872+0000","last_clean":"2024-03-21T13:26:52.374872+0000","last_became_active":"2024-03-21T13:26:11.194727+0000","last_became_peered":"2024-03-21T13:26:11.194727+0000","last_unstale":"2024-03-21T13:26:52.374872+0000","last_undegraded":"2024-03-21T13:26:52.374872+0000","last_fullsized":"2024-03-21T13:26:52.374872+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:36:25.687188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"67'10","reported_seq":44,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.641672+0000","last_change":"2024-03-21T13:26:13.222254+0000","last_active":"2024-03-21T13:26:52.641672+0000","last_peered":"2024-03-21T13:26:52.641672+0000","last_clean":"2024-03-21T13:26:52.641672+0000","last_became_active":"2024-03-21T13:26:13.220420+0000","last_became_peered":"2024-03-21T13:26:13.220420+0000","last_unstale":"2024-03-21T13:26:52.641672+0000","last_undegraded":"2024-03-21T13:26:52.641672+0000","last_fullsized":"2024-03-21T13:26:52.641672+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T00:31:51.133481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"67'8","reported_seq":50,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:27:26.251456+0000","last_change":"2024-03-21T13:26:15.232582+0000","last_active":"2024-03-21T13:27:26.251456+0000","last_peered":"2024-03-21T13:27:26.251456+0000","last_clean":"2024-03-21T13:27:26.251456+0000","last_became_active":"2024-03-21T13:26:15.232270+0000","last_became_peered":"2024-03-21T13:26:15.232270+0000","last_unstale":"2024-03-21T13:27:26.251456+0000","last_undegraded":"2024-03-21T13:27:26.251456+0000","last_fullsized":"2024-03-21T13:27:26.251456+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:57:50.128265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":20,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.386510+0000","last_change":"2024-03-21T13:26:17.308604+0000","last_active":"2024-03-21T13:26:52.386510+0000","last_peered":"2024-03-21T13:26:52.386510+0000","last_clean":"2024-03-21T13:26:52.386510+0000","last_became_active":"2024-03-21T13:26:17.308310+0000","last_became_peered":"2024-03-21T13:26:17.308310+0000","last_unstale":"2024-03-21T13:26:52.386510+0000","last_undegraded":"2024-03-21T13:26:52.386510+0000","last_fullsized":"2024-03-21T13:26:52.386510+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:33:56.518710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"60'2","reported_seq":40,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.640815+0000","last_change":"2024-03-21T13:26:11.197777+0000","last_active":"2024-03-21T13:26:52.640815+0000","last_peered":"2024-03-21T13:26:52.640815+0000","last_clean":"2024-03-21T13:26:52.640815+0000","last_became_active":"2024-03-21T13:26:11.197280+0000","last_became_peered":"2024-03-21T13:26:11.197280+0000","last_unstale":"2024-03-21T13:26:52.640815+0000","last_undegraded":"2024-03-21T13:26:52.640815+0000","last_fullsized":"2024-03-21T13:26:52.640815+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:37:03.885564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":482,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"67'14","reported_seq":50,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.271186+0000","last_change":"2024-03-21T13:26:13.218843+0000","last_active":"2024-03-21T13:26:52.271186+0000","last_peered":"2024-03-21T13:26:52.271186+0000","last_clean":"2024-03-21T13:26:52.271186+0000","last_became_active":"2024-03-21T13:26:13.218561+0000","last_became_peered":"2024-03-21T13:26:13.218561+0000","last_unstale":"2024-03-21T13:26:52.271186+0000","last_undegraded":"2024-03-21T13:26:52.271186+0000","last_fullsized":"2024-03-21T13:26:52.271186+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:59:16.467897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"67'8","reported_seq":50,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:27:26.251470+0000","last_change":"2024-03-21T13:26:15.235401+0000","last_active":"2024-03-21T13:27:26.251470+0000","last_peered":"2024-03-21T13:27:26.251470+0000","last_clean":"2024-03-21T13:27:26.251470+0000","last_became_active":"2024-03-21T13:26:15.235025+0000","last_became_peered":"2024-03-21T13:26:15.235025+0000","last_unstale":"2024-03-21T13:27:26.251470+0000","last_undegraded":"2024-03-21T13:27:26.251470+0000","last_fullsized":"2024-03-21T13:27:26.251470+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T00:30:53.823248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.271209+0000","last_change":"2024-03-21T13:26:17.247947+0000","last_active":"2024-03-21T13:26:52.271209+0000","last_peered":"2024-03-21T13:26:52.271209+0000","last_clean":"2024-03-21T13:26:52.271209+0000","last_became_active":"2024-03-21T13:26:17.247641+0000","last_became_peered":"2024-03-21T13:26:17.247641+0000","last_unstale":"2024-03-21T13:26:52.271209+0000","last_undegraded":"2024-03-21T13:26:52.271209+0000","last_fullsized":"2024-03-21T13:26:52.271209+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T00:00:51.801628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.271685+0000","last_change":"2024-03-21T13:26:11.196455+0000","last_active":"2024-03-21T13:26:52.271685+0000","last_peered":"2024-03-21T13:26:52.271685+0000","last_clean":"2024-03-21T13:26:52.271685+0000","last_became_active":"2024-03-21T13:26:11.195000+0000","last_became_peered":"2024-03-21T13:26:11.195000+0000","last_unstale":"2024-03-21T13:26:52.271685+0000","last_undegraded":"2024-03-21T13:26:52.271685+0000","last_fullsized":"2024-03-21T13:26:52.271685+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:46:59.830691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"67'18","reported_seq":61,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.269232+0000","last_change":"2024-03-21T13:26:13.219901+0000","last_active":"2024-03-21T13:26:52.269232+0000","last_peered":"2024-03-21T13:26:52.269232+0000","last_clean":"2024-03-21T13:26:52.269232+0000","last_became_active":"2024-03-21T13:26:13.219568+0000","last_became_peered":"2024-03-21T13:26:13.219568+0000","last_unstale":"2024-03-21T13:26:52.269232+0000","last_undegraded":"2024-03-21T13:26:52.269232+0000","last_fullsized":"2024-03-21T13:26:52.269232+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:59:30.215346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.271032+0000","last_change":"2024-03-21T13:26:15.229888+0000","last_active":"2024-03-21T13:26:52.271032+0000","last_peered":"2024-03-21T13:26:52.271032+0000","last_clean":"2024-03-21T13:26:52.271032+0000","last_became_active":"2024-03-21T13:26:15.229607+0000","last_became_peered":"2024-03-21T13:26:15.229607+0000","last_unstale":"2024-03-21T13:26:52.271032+0000","last_undegraded":"2024-03-21T13:26:52.271032+0000","last_fullsized":"2024-03-21T13:26:52.271032+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:30:34.420895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.374640+0000","last_change":"2024-03-21T13:26:17.245904+0000","last_active":"2024-03-21T13:26:52.374640+0000","last_peered":"2024-03-21T13:26:52.374640+0000","last_clean":"2024-03-21T13:26:52.374640+0000","last_became_active":"2024-03-21T13:26:17.245638+0000","last_became_peered":"2024-03-21T13:26:17.245638+0000","last_unstale":"2024-03-21T13:26:52.374640+0000","last_undegraded":"2024-03-21T13:26:52.374640+0000","last_fullsized":"2024-03-21T13:26:52.374640+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:53:47.375388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"67'8","reported_seq":50,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:27:26.251739+0000","last_change":"2024-03-21T13:26:15.235193+0000","last_active":"2024-03-21T13:27:26.251739+0000","last_peered":"2024-03-21T13:27:26.251739+0000","last_clean":"2024-03-21T13:27:26.251739+0000","last_became_active":"2024-03-21T13:26:15.234885+0000","last_became_peered":"2024-03-21T13:26:15.234885+0000","last_unstale":"2024-03-21T13:27:26.251739+0000","last_undegraded":"2024-03-21T13:27:26.251739+0000","last_fullsized":"2024-03-21T13:27:26.251739+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:04:47.692918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.269201+0000","last_change":"2024-03-21T13:26:11.193648+0000","last_active":"2024-03-21T13:26:52.269201+0000","last_peered":"2024-03-21T13:26:52.269201+0000","last_clean":"2024-03-21T13:26:52.269201+0000","last_became_active":"2024-03-21T13:26:11.193211+0000","last_became_peered":"2024-03-21T13:26:11.193211+0000","last_unstale":"2024-03-21T13:26:52.269201+0000","last_undegraded":"2024-03-21T13:26:52.269201+0000","last_fullsized":"2024-03-21T13:26:52.269201+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:28:43.496752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"67'15","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.383461+0000","last_change":"2024-03-21T13:26:32.428003+0000","last_active":"2024-03-21T13:26:52.383461+0000","last_peered":"2024-03-21T13:26:52.383461+0000","last_clean":"2024-03-21T13:26:52.383461+0000","last_became_active":"2024-03-21T13:26:32.418711+0000","last_became_peered":"2024-03-21T13:26:32.418711+0000","last_unstale":"2024-03-21T13:26:52.383461+0000","last_undegraded":"2024-03-21T13:26:52.383461+0000","last_fullsized":"2024-03-21T13:26:52.383461+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:51:55.750843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.269708+0000","last_change":"2024-03-21T13:26:17.305635+0000","last_active":"2024-03-21T13:26:52.269708+0000","last_peered":"2024-03-21T13:26:52.269708+0000","last_clean":"2024-03-21T13:26:52.269708+0000","last_became_active":"2024-03-21T13:26:17.305293+0000","last_became_peered":"2024-03-21T13:26:17.305293+0000","last_unstale":"2024-03-21T13:26:52.269708+0000","last_undegraded":"2024-03-21T13:26:52.269708+0000","last_fullsized":"2024-03-21T13:26:52.269708+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:42:58.999639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.375209+0000","last_change":"2024-03-21T13:26:15.238323+0000","last_active":"2024-03-21T13:26:52.375209+0000","last_peered":"2024-03-21T13:26:52.375209+0000","last_clean":"2024-03-21T13:26:52.375209+0000","last_became_active":"2024-03-21T13:26:15.238000+0000","last_became_peered":"2024-03-21T13:26:15.238000+0000","last_unstale":"2024-03-21T13:26:52.375209+0000","last_undegraded":"2024-03-21T13:26:52.375209+0000","last_fullsized":"2024-03-21T13:26:52.375209+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:42:57.216972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.271057+0000","last_change":"2024-03-21T13:26:11.196402+0000","last_active":"2024-03-21T13:26:52.271057+0000","last_peered":"2024-03-21T13:26:52.271057+0000","last_clean":"2024-03-21T13:26:52.271057+0000","last_became_active":"2024-03-21T13:26:11.195729+0000","last_became_peered":"2024-03-21T13:26:11.195729+0000","last_unstale":"2024-03-21T13:26:52.271057+0000","last_undegraded":"2024-03-21T13:26:52.271057+0000","last_fullsized":"2024-03-21T13:26:52.271057+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:00:40.926613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"67'11","reported_seq":48,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.272512+0000","last_change":"2024-03-21T13:26:13.222434+0000","last_active":"2024-03-21T13:26:52.272512+0000","last_peered":"2024-03-21T13:26:52.272512+0000","last_clean":"2024-03-21T13:26:52.272512+0000","last_became_active":"2024-03-21T13:26:13.221644+0000","last_became_peered":"2024-03-21T13:26:13.221644+0000","last_unstale":"2024-03-21T13:26:52.272512+0000","last_undegraded":"2024-03-21T13:26:52.272512+0000","last_fullsized":"2024-03-21T13:26:52.272512+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T00:18:55.479249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.383604+0000","last_change":"2024-03-21T13:26:17.251871+0000","last_active":"2024-03-21T13:26:52.383604+0000","last_peered":"2024-03-21T13:26:52.383604+0000","last_clean":"2024-03-21T13:26:52.383604+0000","last_became_active":"2024-03-21T13:26:17.251545+0000","last_became_peered":"2024-03-21T13:26:17.251545+0000","last_unstale":"2024-03-21T13:26:52.383604+0000","last_undegraded":"2024-03-21T13:26:52.383604+0000","last_fullsized":"2024-03-21T13:26:52.383604+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:57:07.626204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.641081+0000","last_change":"2024-03-21T13:26:15.226863+0000","last_active":"2024-03-21T13:26:52.641081+0000","last_peered":"2024-03-21T13:26:52.641081+0000","last_clean":"2024-03-21T13:26:52.641081+0000","last_became_active":"2024-03-21T13:26:15.226490+0000","last_became_peered":"2024-03-21T13:26:15.226490+0000","last_unstale":"2024-03-21T13:26:52.641081+0000","last_undegraded":"2024-03-21T13:26:52.641081+0000","last_fullsized":"2024-03-21T13:26:52.641081+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:07:37.450970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.547032+0000","last_change":"2024-03-21T13:26:11.196591+0000","last_active":"2024-03-21T13:26:52.547032+0000","last_peered":"2024-03-21T13:26:52.547032+0000","last_clean":"2024-03-21T13:26:52.547032+0000","last_became_active":"2024-03-21T13:26:11.196304+0000","last_became_peered":"2024-03-21T13:26:11.196304+0000","last_unstale":"2024-03-21T13:26:52.547032+0000","last_undegraded":"2024-03-21T13:26:52.547032+0000","last_fullsized":"2024-03-21T13:26:52.547032+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:55:25.839773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"67'17","reported_seq":57,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.272192+0000","last_change":"2024-03-21T13:26:13.222129+0000","last_active":"2024-03-21T13:26:52.272192+0000","last_peered":"2024-03-21T13:26:52.272192+0000","last_clean":"2024-03-21T13:26:52.272192+0000","last_became_active":"2024-03-21T13:26:13.221554+0000","last_became_peered":"2024-03-21T13:26:13.221554+0000","last_unstale":"2024-03-21T13:26:52.272192+0000","last_undegraded":"2024-03-21T13:26:52.272192+0000","last_fullsized":"2024-03-21T13:26:52.272192+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:30:27.492301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.272635+0000","last_change":"2024-03-21T13:26:17.248809+0000","last_active":"2024-03-21T13:26:52.272635+0000","last_peered":"2024-03-21T13:26:52.272635+0000","last_clean":"2024-03-21T13:26:52.272635+0000","last_became_active":"2024-03-21T13:26:17.248504+0000","last_became_peered":"2024-03-21T13:26:17.248504+0000","last_unstale":"2024-03-21T13:26:52.272635+0000","last_undegraded":"2024-03-21T13:26:52.272635+0000","last_fullsized":"2024-03-21T13:26:52.272635+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:16.213525+0000","last_clean_scrub_stamp":"2024-03-21T13:26:16.213525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T01:17:19.221467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"67'8","reported_seq":50,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:27:26.251456+0000","last_change":"2024-03-21T13:26:15.232591+0000","last_active":"2024-03-21T13:27:26.251456+0000","last_peered":"2024-03-21T13:27:26.251456+0000","last_clean":"2024-03-21T13:27:26.251456+0000","last_became_active":"2024-03-21T13:26:15.232239+0000","last_became_peered":"2024-03-21T13:26:15.232239+0000","last_unstale":"2024-03-21T13:27:26.251456+0000","last_undegraded":"2024-03-21T13:27:26.251456+0000","last_fullsized":"2024-03-21T13:27:26.251456+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:14.202222+0000","last_clean_scrub_stamp":"2024-03-21T13:26:14.202222+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:59:18.606505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.269921+0000","last_change":"2024-03-21T13:26:11.195917+0000","last_active":"2024-03-21T13:26:52.269921+0000","last_peered":"2024-03-21T13:26:52.269921+0000","last_clean":"2024-03-21T13:26:52.269921+0000","last_became_active":"2024-03-21T13:26:11.195581+0000","last_became_peered":"2024-03-21T13:26:11.195581+0000","last_unstale":"2024-03-21T13:26:52.269921+0000","last_undegraded":"2024-03-21T13:26:52.269921+0000","last_fullsized":"2024-03-21T13:26:52.269921+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:10.174287+0000","last_clean_scrub_stamp":"2024-03-21T13:26:10.174287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:17:56.216644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"67'10","reported_seq":44,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-03-21T13:26:52.271906+0000","last_change":"2024-03-21T13:26:13.224564+0000","last_active":"2024-03-21T13:26:52.271906+0000","last_peered":"2024-03-21T13:26:52.271906+0000","last_clean":"2024-03-21T13:26:52.271906+0000","last_became_active":"2024-03-21T13:26:13.224245+0000","last_became_peered":"2024-03-21T13:26:13.224245+0000","last_unstale":"2024-03-21T13:26:52.271906+0000","last_undegraded":"2024-03-21T13:26:52.271906+0000","last_fullsized":"2024-03-21T13:26:52.271906+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T13:26:12.184533+0000","last_clean_scrub_stamp":"2024-03-21T13:26:12.184533+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:46:05.005340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":383,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1531,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":75,"num_read_kb":70,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":9,"acting":9,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234005,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32448,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712704,"statfs":{"total":95995035648,"available":95961808896,"internally_reserved":0,"allocated":1302528,"data_stored":867673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495642,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32428,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712724,"statfs":{"total":95995035648,"available":95961829376,"internally_reserved":0,"allocated":1277952,"data_stored":866608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574900,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27752,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717400,"statfs":{"total":95995035648,"available":95966617600,"internally_reserved":0,"allocated":749568,"data_stored":279055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738426,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32460,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712692,"statfs":{"total":95995035648,"available":95961796608,"internally_reserved":0,"allocated":1310720,"data_stored":867851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411375,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27700,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717452,"statfs":{"total":95995035648,"available":95966670848,"internally_reserved":0,"allocated":696320,"data_stored":277596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182442,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31884,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713268,"statfs":{"total":95995035648,"available":95962386432,"internally_reserved":0,"allocated":720896,"data_stored":277131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018917,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31880,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713272,"statfs":{"total":95995035648,"available":95962390528,"internally_reserved":0,"allocated":716800,"data_stored":277281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757279,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31856,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713296,"statfs":{"total":95995035648,"available":95962415104,"internally_reserved":0,"allocated":692224,"data_stored":277477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-21T13:27:35.867 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-21T13:27:35.867 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-21T13:27:35.867 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-21T13:27:35.867 DEBUG:teuthology.orchestra.run.smithi097:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 -- ceph health --format=json 2024-03-21T13:27:36.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:36 smithi144 ceph-mon[33633]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:27:36.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:36 smithi144 ceph-mon[33633]: from='client.24629 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T13:27:36.447 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T13:27:36.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:36 smithi097 ceph-mon[30040]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:27:36.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:36 smithi097 ceph-mon[30040]: from='client.24629 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T13:27:36.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:36 smithi097 ceph-mon[39416]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:27:36.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:36 smithi097 ceph-mon[39416]: from='client.24629 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T13:27:38.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:38 smithi097 ceph-mon[30040]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:38.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:38 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:27:38.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:38 smithi097 ceph-mon[39416]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:38.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:38 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:27:38.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:38 smithi144 ceph-mon[33633]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:38.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:38 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:27:39.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:39.886+0000 7fc5c2dac700 1 -- 172.21.15.97:0/3784706312 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc5bc06b1b0 msgr2=0x7fc5bc10e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:39.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:39.886+0000 7fc5c2dac700 1 --2- 172.21.15.97:0/3784706312 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc5bc06b1b0 0x7fc5bc10e730 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac0098f0 tx=0x7fc5ac009c00 comp rx=0 tx=0).stop 2024-03-21T13:27:39.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:39.886+0000 7fc5c2dac700 1 -- 172.21.15.97:0/3784706312 shutdown_connections 2024-03-21T13:27:39.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:39.886+0000 7fc5c2dac700 1 --2- 172.21.15.97:0/3784706312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5bc10ec70 0x7fc5bc111060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:39.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:39.886+0000 7fc5c2dac700 1 --2- 172.21.15.97:0/3784706312 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5bc06a890 0x7fc5bc06ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:39.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:39.886+0000 7fc5c2dac700 1 --2- 172.21.15.97:0/3784706312 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc5bc06b1b0 0x7fc5bc10e730 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:39.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:39.886+0000 7fc5c2dac700 1 -- 172.21.15.97:0/3784706312 >> 172.21.15.97:0/3784706312 conn(0x7fc5bc0770b0 msgr2=0x7fc5bc0774c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:27:39.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:39.886+0000 7fc5c2dac700 1 -- 172.21.15.97:0/3784706312 shutdown_connections 2024-03-21T13:27:39.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:39.886+0000 7fc5c2dac700 1 -- 172.21.15.97:0/3784706312 wait complete. 2024-03-21T13:27:39.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:39.886+0000 7fc5c2dac700 1 Processor -- start 2024-03-21T13:27:39.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:39.887+0000 7fc5c2dac700 1 -- start start 2024-03-21T13:27:39.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:39.887+0000 7fc5c2dac700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5bc06a890 0x7fc5bc19a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:39.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:39.887+0000 7fc5c2dac700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc5bc06b1b0 0x7fc5bc19a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:39.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:39.887+0000 7fc5c2dac700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5bc10ec70 0x7fc5bc19eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:39.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:39.887+0000 7fc5c2dac700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5bc113d80 con 0x7fc5bc06a890 2024-03-21T13:27:39.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:39.887+0000 7fc5c2dac700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5bc113c00 con 0x7fc5bc10ec70 2024-03-21T13:27:39.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:39.887+0000 7fc5c2dac700 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5bc113f00 con 0x7fc5bc06b1b0 2024-03-21T13:27:39.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:39.887+0000 7fc5bbfff700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc5bc06b1b0 0x7fc5bc19a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:39.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:39.887+0000 7fc5c0b48700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5bc06a890 0x7fc5bc19a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:39.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:39.887+0000 7fc5bbfff700 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc5bc06b1b0 0x7fc5bc19a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.97:58510/0 (socket says 172.21.15.97:58510) 2024-03-21T13:27:39.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:39.887+0000 7fc5bbfff700 1 -- 172.21.15.97:0/2727120979 learned_addr learned my addr 172.21.15.97:0/2727120979 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:27:39.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:39.887+0000 7fc5c1349700 1 --2- 172.21.15.97:0/2727120979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5bc10ec70 0x7fc5bc19eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:39.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:39.887+0000 7fc5c0b48700 1 -- 172.21.15.97:0/2727120979 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc5bc06b1b0 msgr2=0x7fc5bc19a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:39.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:39.887+0000 7fc5c0b48700 1 --2- 172.21.15.97:0/2727120979 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc5bc06b1b0 0x7fc5bc19a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:39.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:39.887+0000 7fc5c0b48700 1 -- 172.21.15.97:0/2727120979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5bc10ec70 msgr2=0x7fc5bc19eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:39.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:39.887+0000 7fc5c0b48700 1 --2- 172.21.15.97:0/2727120979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5bc10ec70 0x7fc5bc19eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:39.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:39.887+0000 7fc5c0b48700 1 -- 172.21.15.97:0/2727120979 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5bc19f2a0 con 0x7fc5bc06a890 2024-03-21T13:27:39.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:39.887+0000 7fc5bbfff700 1 --2- 172.21.15.97:0/2727120979 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc5bc06b1b0 0x7fc5bc19a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T13:27:39.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:39.887+0000 7fc5c0b48700 1 --2- 172.21.15.97:0/2727120979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5bc06a890 0x7fc5bc19a2e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4009fd0 tx=0x7fc5b400c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:27:39.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:39.887+0000 7fc5b9ffb700 1 -- 172.21.15.97:0/2727120979 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b400ce40 con 0x7fc5bc06a890 2024-03-21T13:27:39.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:39.888+0000 7fc5c2dac700 1 -- 172.21.15.97:0/2727120979 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5bc19f530 con 0x7fc5bc06a890 2024-03-21T13:27:39.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:39.888+0000 7fc5b9ffb700 1 -- 172.21.15.97:0/2727120979 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fc5b4003a00 con 0x7fc5bc06a890 2024-03-21T13:27:39.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:39.888+0000 7fc5b9ffb700 1 -- 172.21.15.97:0/2727120979 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b4004650 con 0x7fc5bc06a890 2024-03-21T13:27:39.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:39.888+0000 7fc5c2dac700 1 -- 172.21.15.97:0/2727120979 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5bc1a6e10 con 0x7fc5bc06a890 2024-03-21T13:27:39.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:39.889+0000 7fc5c2dac700 1 -- 172.21.15.97:0/2727120979 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5bc1016a0 con 0x7fc5bc06a890 2024-03-21T13:27:39.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:39.889+0000 7fc5b9ffb700 1 -- 172.21.15.97:0/2727120979 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 22) v1 ==== 92152+0+0 (secure 0 0 0) 0x7fc5b401a020 con 0x7fc5bc06a890 2024-03-21T13:27:39.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:39.889+0000 7fc5b9ffb700 1 --2- 172.21.15.97:0/2727120979 >> v2:172.21.15.97:6800/12494470 conn(0x7fc5a806e300 0x7fc5a80707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:27:39.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:39.889+0000 7fc5b9ffb700 1 -- 172.21.15.97:0/2727120979 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6259+0+0 (secure 0 0 0) 0x7fc5b408df00 con 0x7fc5bc06a890 2024-03-21T13:27:39.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:39.889+0000 7fc5bbfff700 1 --2- 172.21.15.97:0/2727120979 >> v2:172.21.15.97:6800/12494470 conn(0x7fc5a806e300 0x7fc5a80707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:27:39.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:39.890+0000 7fc5bbfff700 1 --2- 172.21.15.97:0/2727120979 >> v2:172.21.15.97:6800/12494470 conn(0x7fc5a806e300 0x7fc5a80707c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac009680 tx=0x7fc5ac019040 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:27:39.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:39.896+0000 7fc5b9ffb700 1 -- 172.21.15.97:0/2727120979 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc5b4010040 con 0x7fc5bc06a890 2024-03-21T13:27:40.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:40.147+0000 7fc5c2dac700 1 -- 172.21.15.97:0/2727120979 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fc5bc19b5f0 con 0x7fc5bc06a890 2024-03-21T13:27:40.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:40.148+0000 7fc5b9ffb700 1 -- 172.21.15.97:0/2727120979 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fc5b405b650 con 0x7fc5bc06a890 2024-03-21T13:27:40.148 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-21T13:27:40.148 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-21T13:27:40.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:40.150+0000 7fc5c2dac700 1 -- 172.21.15.97:0/2727120979 >> v2:172.21.15.97:6800/12494470 conn(0x7fc5a806e300 msgr2=0x7fc5a80707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:40.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:40.150+0000 7fc5c2dac700 1 --2- 172.21.15.97:0/2727120979 >> v2:172.21.15.97:6800/12494470 conn(0x7fc5a806e300 0x7fc5a80707c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac009680 tx=0x7fc5ac019040 comp rx=0 tx=0).stop 2024-03-21T13:27:40.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:40.150+0000 7fc5c2dac700 1 -- 172.21.15.97:0/2727120979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5bc06a890 msgr2=0x7fc5bc19a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:40.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:40.150+0000 7fc5c2dac700 1 --2- 172.21.15.97:0/2727120979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5bc06a890 0x7fc5bc19a2e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4009fd0 tx=0x7fc5b400c620 comp rx=0 tx=0).stop 2024-03-21T13:27:40.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:40.150+0000 7fc5c2dac700 1 -- 172.21.15.97:0/2727120979 shutdown_connections 2024-03-21T13:27:40.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:40.150+0000 7fc5c2dac700 1 --2- 172.21.15.97:0/2727120979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5bc10ec70 0x7fc5bc19eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:40.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:40.150+0000 7fc5c2dac700 1 --2- 172.21.15.97:0/2727120979 >> v2:172.21.15.97:6800/12494470 conn(0x7fc5a806e300 0x7fc5a80707c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:40.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:40.150+0000 7fc5c2dac700 1 --2- 172.21.15.97:0/2727120979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5bc06a890 0x7fc5bc19a2e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:40.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:40.150+0000 7fc5c2dac700 1 --2- 172.21.15.97:0/2727120979 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc5bc06b1b0 0x7fc5bc19a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:27:40.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:40.150+0000 7fc5c2dac700 1 -- 172.21.15.97:0/2727120979 >> 172.21.15.97:0/2727120979 conn(0x7fc5bc0770b0 msgr2=0x7fc5bc1000b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:27:40.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:40.150+0000 7fc5c2dac700 1 -- 172.21.15.97:0/2727120979 shutdown_connections 2024-03-21T13:27:40.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-21T13:27:40.150+0000 7fc5c2dac700 1 -- 172.21.15.97:0/2727120979 wait complete. 2024-03-21T13:27:40.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:40 smithi144 ceph-mon[33633]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:27:40.390 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:40 smithi097 ceph-mon[39416]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:27:40.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:40 smithi097 ceph-mon[30040]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:27:41.062 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-21T13:27:41.062 INFO:tasks.cephadm:Setup complete, yielding 2024-03-21T13:27:41.063 INFO:teuthology.run_tasks:Running task workunit... 2024-03-21T13:27:41.076 INFO:tasks.workunit:Pulling workunits from ref 960fbbe4645c2becc8084669be194c95183c7753 2024-03-21T13:27:41.077 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-21T13:27:41.077 DEBUG:teuthology.orchestra.run.smithi097:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-21T13:27:41.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T13:27:41.094 INFO:teuthology.orchestra.run.smithi097.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-03-21T13:27:41.095 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-03-21T13:27:41.151 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-03-21T13:27:41.152 DEBUG:teuthology.orchestra.run.smithi097:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-03-21T13:27:41.208 INFO:tasks.workunit:timeout=3h 2024-03-21T13:27:41.208 INFO:tasks.workunit:cleanup=True 2024-03-21T13:27:41.209 DEBUG:teuthology.orchestra.run.smithi097:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 960fbbe4645c2becc8084669be194c95183c7753 2024-03-21T13:27:41.264 INFO:tasks.workunit.client.0.smithi097.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-21T13:27:41.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:41 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2727120979' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-21T13:27:41.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:41 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2727120979' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-21T13:27:41.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:41 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2727120979' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-21T13:27:42.053 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:27:41 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:27:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:27:42.053 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 13:27:41 smithi097 systemd[1]: Starting Ceph alertmanager.a for a5dd1f32-e785-11ee-95cd-87774f69a715... 2024-03-21T13:27:42.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:42 smithi144 ceph-mon[33633]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:42.498 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:42 smithi097 ceph-mon[39416]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:42.499 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:42 smithi097 ceph-mon[30040]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:42.499 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 13:27:42 smithi097 podman[77522]: 2024-03-21 13:27:42.090877729 +0000 UTC m=+0.017146847 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-03-21T13:27:42.499 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 13:27:42 smithi097 podman[77522]: 2024-03-21 13:27:42.201935145 +0000 UTC m=+0.128204264 volume create 577bc43a727b07df1264d94e41ee0195c1cf401e1bf8ebe7786a4a08e3d02f16 2024-03-21T13:27:42.499 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 13:27:42 smithi097 podman[77522]: 2024-03-21 13:27:42.285705197 +0000 UTC m=+0.211974315 container create 365241232e2e6e065d28e871cd085d3ef04dce642601aab28c5fe2f041e3ef4d (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-21T13:27:42.921 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 13:27:42 smithi097 podman[77522]: 2024-03-21 13:27:42.630297746 +0000 UTC m=+0.556566865 container init 365241232e2e6e065d28e871cd085d3ef04dce642601aab28c5fe2f041e3ef4d (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-21T13:27:42.921 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 13:27:42 smithi097 podman[77522]: 2024-03-21 13:27:42.636708296 +0000 UTC m=+0.562977423 container start 365241232e2e6e065d28e871cd085d3ef04dce642601aab28c5fe2f041e3ef4d (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-21T13:27:42.921 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 13:27:42 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-alertmanager-a[77537]: ts=2024-03-21T13:27:42.649Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-03-21T13:27:42.921 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 13:27:42 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-alertmanager-a[77537]: ts=2024-03-21T13:27:42.649Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-03-21T13:27:42.921 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 13:27:42 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-alertmanager-a[77537]: ts=2024-03-21T13:27:42.650Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.97 port=9094 2024-03-21T13:27:42.921 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 13:27:42 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-alertmanager-a[77537]: ts=2024-03-21T13:27:42.651Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-03-21T13:27:42.921 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 13:27:42 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-alertmanager-a[77537]: ts=2024-03-21T13:27:42.673Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-03-21T13:27:42.922 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 13:27:42 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-alertmanager-a[77537]: ts=2024-03-21T13:27:42.673Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-03-21T13:27:42.922 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 13:27:42 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-alertmanager-a[77537]: ts=2024-03-21T13:27:42.677Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-03-21T13:27:42.922 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 13:27:42 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-alertmanager-a[77537]: ts=2024-03-21T13:27:42.677Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-03-21T13:27:42.922 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 13:27:42 smithi097 bash[77522]: 365241232e2e6e065d28e871cd085d3ef04dce642601aab28c5fe2f041e3ef4d 2024-03-21T13:27:42.922 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 13:27:42 smithi097 systemd[1]: Started Ceph alertmanager.a for a5dd1f32-e785-11ee-95cd-87774f69a715. 2024-03-21T13:27:43.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:27:42 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: [21/Mar/2024:13:27:42] ENGINE Bus STOPPING 2024-03-21T13:27:43.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:27:42 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: [21/Mar/2024:13:27:42] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-03-21T13:27:43.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:27:42 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: [21/Mar/2024:13:27:42] ENGINE Bus STOPPED 2024-03-21T13:27:43.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:27:42 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: [21/Mar/2024:13:27:42] ENGINE Bus STARTING 2024-03-21T13:27:43.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:27:43 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: [21/Mar/2024:13:27:43] ENGINE Serving on http://:::9283 2024-03-21T13:27:43.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:27:43 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: [21/Mar/2024:13:27:43] ENGINE Bus STARTED 2024-03-21T13:27:43.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:27:43 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:27:44.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:43 smithi097 ceph-mon[39416]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:44.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:43 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:44.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:43 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:44.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:43 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:44.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:43 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:44.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:43 smithi097 ceph-mon[39416]: Regenerating cephadm self-signed grafana TLS certificates 2024-03-21T13:27:44.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:43 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:44.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:43 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:44.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:43 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-21T13:27:44.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:43 smithi097 ceph-mon[39416]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-21T13:27:44.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:43 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:44.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:43 smithi097 ceph-mon[39416]: Deploying daemon grafana.a on smithi144 2024-03-21T13:27:44.057 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:43 smithi097 ceph-mon[30040]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:44.058 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:43 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:44.058 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:43 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:44.058 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:43 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:44.058 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:43 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:44.058 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:43 smithi097 ceph-mon[30040]: Regenerating cephadm self-signed grafana TLS certificates 2024-03-21T13:27:44.058 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:43 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:44.058 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:43 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:44.058 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:43 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-21T13:27:44.058 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:43 smithi097 ceph-mon[30040]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-21T13:27:44.058 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:43 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:44.058 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:43 smithi097 ceph-mon[30040]: Deploying daemon grafana.a on smithi144 2024-03-21T13:27:44.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:43 smithi144 ceph-mon[33633]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:44.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:43 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:44.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:43 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:44.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:43 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:44.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:43 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:44.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:43 smithi144 ceph-mon[33633]: Regenerating cephadm self-signed grafana TLS certificates 2024-03-21T13:27:44.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:43 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:44.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:43 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:44.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:43 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-21T13:27:44.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:43 smithi144 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-21T13:27:44.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:43 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:44.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:43 smithi144 ceph-mon[33633]: Deploying daemon grafana.a on smithi144 2024-03-21T13:27:45.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:44 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:27:45.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:44 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:27:45.054 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 13:27:44 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-alertmanager-a[77537]: ts=2024-03-21T13:27:44.651Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000127331s 2024-03-21T13:27:45.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:44 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:27:46.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:45 smithi097 ceph-mon[39416]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:27:46.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:45 smithi097 ceph-mon[30040]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:27:46.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:45 smithi144 ceph-mon[33633]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:27:48.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:47 smithi097 ceph-mon[39416]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:48.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:47 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:48.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:47 smithi097 ceph-mon[30040]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:48.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:47 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:48.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:47 smithi144 ceph-mon[33633]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:48.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:47 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:50.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:49 smithi097 ceph-mon[39416]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:27:50.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:49 smithi097 ceph-mon[30040]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:27:50.073 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:49 smithi144 ceph-mon[33633]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:27:52.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:51 smithi097 ceph-mon[39416]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:52.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:51 smithi097 ceph-mon[30040]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:52.054 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:27:51 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:27:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:27:52.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:51 smithi144 ceph-mon[33633]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:52.800 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:52 smithi144 systemd[1]: Starting Ceph grafana.a for a5dd1f32-e785-11ee-95cd-87774f69a715... 2024-03-21T13:27:53.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:52 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:27:53.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:52 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:27:53.054 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 13:27:52 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-alertmanager-a[77537]: ts=2024-03-21T13:27:52.653Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.002351429s 2024-03-21T13:27:53.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:52 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:27:53.115 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:52 smithi144 podman[66262]: 2024-03-21 13:27:52.75822115 +0000 UTC m=+0.019518157 image pull 2bacad6d85d802cd176c146528adcc0e47a8a83e94bc4a8b7ba1a9b9ef2c65e7 quay.io/ceph/grafana:9.4.12 2024-03-21T13:27:53.115 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:52 smithi144 podman[66262]: 2024-03-21 13:27:52.888210743 +0000 UTC m=+0.149507744 container create a01d0c4fc05f5eb5e1a0aaee458e5ec42cbb796716f0f95f7d9d06977dc543cb (image=quay.io/ceph/grafana:9.4.12, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a, maintainer=Grafana Labs ) 2024-03-21T13:27:53.115 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 podman[66262]: 2024-03-21 13:27:53.115651682 +0000 UTC m=+0.376948690 container init a01d0c4fc05f5eb5e1a0aaee458e5ec42cbb796716f0f95f7d9d06977dc543cb (image=quay.io/ceph/grafana:9.4.12, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a, maintainer=Grafana Labs ) 2024-03-21T13:27:53.365 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:27:53.366 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 podman[66262]: 2024-03-21 13:27:53.12087472 +0000 UTC m=+0.382171719 container start a01d0c4fc05f5eb5e1a0aaee458e5ec42cbb796716f0f95f7d9d06977dc543cb (image=quay.io/ceph/grafana:9.4.12, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a, maintainer=Grafana Labs ) 2024-03-21T13:27:53.366 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 bash[66262]: a01d0c4fc05f5eb5e1a0aaee458e5ec42cbb796716f0f95f7d9d06977dc543cb 2024-03-21T13:27:53.366 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 systemd[1]: Started Ceph grafana.a for a5dd1f32-e785-11ee-95cd-87774f69a715. 2024-03-21T13:27:53.366 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=settings t=2024-03-21T13:27:53.268955277Z level=info msg="Starting Grafana" version=9.4.12 commit=37ee25ba46 branch=HEAD compiled=2023-05-22T15:58:05Z 2024-03-21T13:27:53.366 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=settings t=2024-03-21T13:27:53.269272287Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-03-21T13:27:53.366 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=settings t=2024-03-21T13:27:53.269289103Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-03-21T13:27:53.367 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=settings t=2024-03-21T13:27:53.269295047Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-03-21T13:27:53.367 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=settings t=2024-03-21T13:27:53.269299555Z level=info msg="Config overridden from command line" arg="default.paths.data=/var/lib/grafana" 2024-03-21T13:27:53.367 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=settings t=2024-03-21T13:27:53.269304095Z level=info msg="Config overridden from command line" arg="default.paths.logs=/var/log/grafana" 2024-03-21T13:27:53.367 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=settings t=2024-03-21T13:27:53.269308435Z level=info msg="Config overridden from command line" arg="default.paths.plugins=/var/lib/grafana/plugins" 2024-03-21T13:27:53.367 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=settings t=2024-03-21T13:27:53.269314444Z level=info msg="Config overridden from command line" arg="default.paths.provisioning=/etc/grafana/provisioning" 2024-03-21T13:27:53.367 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=settings t=2024-03-21T13:27:53.269318675Z level=info msg="Config overridden from command line" arg="default.log.mode=console" 2024-03-21T13:27:53.367 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=settings t=2024-03-21T13:27:53.269323585Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-03-21T13:27:53.367 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=settings t=2024-03-21T13:27:53.269328784Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-03-21T13:27:53.367 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=settings t=2024-03-21T13:27:53.269332642Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-03-21T13:27:53.367 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=settings t=2024-03-21T13:27:53.269337023Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-03-21T13:27:53.367 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=settings t=2024-03-21T13:27:53.269341615Z level=info msg="Path Home" path=/usr/share/grafana 2024-03-21T13:27:53.367 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=settings t=2024-03-21T13:27:53.269345939Z level=info msg="Path Data" path=/var/lib/grafana 2024-03-21T13:27:53.367 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=settings t=2024-03-21T13:27:53.269349772Z level=info msg="Path Logs" path=/var/log/grafana 2024-03-21T13:27:53.368 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=settings t=2024-03-21T13:27:53.269353704Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-03-21T13:27:53.368 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=settings t=2024-03-21T13:27:53.26935786Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-03-21T13:27:53.368 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=settings t=2024-03-21T13:27:53.269362603Z level=info msg="App mode production" 2024-03-21T13:27:53.368 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=sqlstore t=2024-03-21T13:27:53.269420684Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-03-21T13:27:53.368 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=sqlstore t=2024-03-21T13:27:53.269436462Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-03-21T13:27:53.368 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.270614693Z level=info msg="Starting DB migrations" 2024-03-21T13:27:53.368 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.270695429Z level=info msg="Executing migration" id="create migration_log table" 2024-03-21T13:27:53.368 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.271272971Z level=info msg="Executing migration" id="create user table" 2024-03-21T13:27:53.368 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.271746594Z level=info msg="Executing migration" id="add unique index user.login" 2024-03-21T13:27:53.368 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.274050836Z level=info msg="Executing migration" id="add unique index user.email" 2024-03-21T13:27:53.368 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.276835412Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-03-21T13:27:53.368 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.277429682Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-03-21T13:27:53.368 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.277842732Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-03-21T13:27:53.368 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.278609513Z level=info msg="Executing migration" id="create user table v2" 2024-03-21T13:27:53.369 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.279004654Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-03-21T13:27:53.369 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.281819062Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-03-21T13:27:53.369 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.286233535Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-03-21T13:27:53.369 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.286630632Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-03-21T13:27:53.369 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.287000808Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-03-21T13:27:53.370 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.287467778Z level=info msg="Executing migration" id="Update user table charset" 2024-03-21T13:27:53.370 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.287736334Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-03-21T13:27:53.370 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.288168352Z level=info msg="Executing migration" id="Add missing user data" 2024-03-21T13:27:53.370 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.288520582Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-03-21T13:27:53.370 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.288906291Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-03-21T13:27:53.370 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.289352575Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-03-21T13:27:53.370 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.289739313Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-03-21T13:27:53.370 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.291166328Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-03-21T13:27:53.370 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.291631971Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-03-21T13:27:53.370 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.292108714Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-03-21T13:27:53.370 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.292542411Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-03-21T13:27:53.370 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.295521259Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-03-21T13:27:53.371 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.296080619Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-03-21T13:27:53.371 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.296411428Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-03-21T13:27:53.371 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.296796276Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-03-21T13:27:53.371 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.297206249Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-03-21T13:27:53.371 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.297602553Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-03-21T13:27:53.371 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.297981644Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-03-21T13:27:53.371 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.29877943Z level=info msg="Executing migration" id="create temp_user v2" 2024-03-21T13:27:53.371 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.299201087Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-03-21T13:27:53.371 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.301969188Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-03-21T13:27:53.371 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.302435219Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-03-21T13:27:53.371 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.302812066Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-03-21T13:27:53.371 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.305422855Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-03-21T13:27:53.371 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.305790991Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-03-21T13:27:53.371 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.30616818Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-03-21T13:27:53.371 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.306514615Z level=info msg="Executing migration" id="create star table" 2024-03-21T13:27:53.371 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.306861801Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-03-21T13:27:53.372 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.3072781Z level=info msg="Executing migration" id="create org table v1" 2024-03-21T13:27:53.372 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.307669969Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-03-21T13:27:53.372 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.308216998Z level=info msg="Executing migration" id="create org_user table v1" 2024-03-21T13:27:53.372 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.30881691Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-03-21T13:27:53.372 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.309336392Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-03-21T13:27:53.372 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.309737846Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-03-21T13:27:53.372 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.310222188Z level=info msg="Executing migration" id="Update org table charset" 2024-03-21T13:27:53.372 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.312937599Z level=info msg="Executing migration" id="Update org_user table charset" 2024-03-21T13:27:53.372 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.315526262Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-03-21T13:27:53.372 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.318236224Z level=info msg="Executing migration" id="create dashboard table" 2024-03-21T13:27:53.372 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.318651576Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-03-21T13:27:53.372 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.321352342Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-03-21T13:27:53.372 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.324073457Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-03-21T13:27:53.372 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.326705869Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-03-21T13:27:53.372 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.327188188Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-03-21T13:27:53.373 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.329941194Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-03-21T13:27:53.373 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.332497382Z level=info msg="Executing migration" id="create dashboard v2" 2024-03-21T13:27:53.373 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.33520562Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-03-21T13:27:53.373 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.335649472Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-03-21T13:27:53.373 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.338263075Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-03-21T13:27:53.373 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.34097578Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-03-21T13:27:53.373 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.343722068Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-03-21T13:27:53.373 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.344043253Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-03-21T13:27:53.373 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.346832966Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-03-21T13:27:53.373 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.349288135Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-03-21T13:27:53.373 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.350210701Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-03-21T13:27:53.373 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.35076386Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-03-21T13:27:53.373 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.351590096Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-03-21T13:27:53.373 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.352458046Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-03-21T13:27:53.373 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.353246738Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-03-21T13:27:53.374 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.354044231Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-03-21T13:27:53.374 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.354855346Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-03-21T13:27:53.374 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.355593156Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-03-21T13:27:53.374 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.356103868Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-03-21T13:27:53.374 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.357054596Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-03-21T13:27:53.374 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.357792856Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-03-21T13:27:53.374 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.358716189Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-03-21T13:27:53.374 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.359497558Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-03-21T13:27:53.374 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.360343025Z level=info msg="Executing migration" id="Update dashboard title length" 2024-03-21T13:27:53.374 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.36102499Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-03-21T13:27:53.374 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.361895111Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-03-21T13:27:53.375 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.362868224Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-03-21T13:27:53.375 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.364034773Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-03-21T13:27:53.375 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.364992872Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-03-21T13:27:53.375 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.36571537Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-03-21T13:27:53.375 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.366495486Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-03-21T13:27:53.375 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.367355214Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-03-21T13:27:53.375 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.368257522Z level=info msg="Executing migration" id="Add check_sum column" 2024-03-21T13:27:53.375 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.369102499Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-03-21T13:27:53.618 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.369918458Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-03-21T13:27:53.619 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.370769032Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-03-21T13:27:53.619 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.371623473Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-03-21T13:27:53.619 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.372490665Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-03-21T13:27:53.619 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.373393414Z level=info msg="Executing migration" id="create data_source table" 2024-03-21T13:27:53.619 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.374265499Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-03-21T13:27:53.619 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.375092774Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-03-21T13:27:53.619 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.375531617Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-03-21T13:27:53.619 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.376382972Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-03-21T13:27:53.619 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.376810887Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-03-21T13:27:53.619 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.378034005Z level=info msg="Executing migration" id="create data_source table v2" 2024-03-21T13:27:53.619 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.37901123Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-03-21T13:27:53.620 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.379759811Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-03-21T13:27:53.620 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.382766738Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-03-21T13:27:53.620 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.38320237Z level=info msg="Executing migration" id="Add column with_credentials" 2024-03-21T13:27:53.620 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.386195482Z level=info msg="Executing migration" id="Add secure json data column" 2024-03-21T13:27:53.620 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.389372097Z level=info msg="Executing migration" id="Update data_source table charset" 2024-03-21T13:27:53.620 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.392907096Z level=info msg="Executing migration" id="Update initial version to 1" 2024-03-21T13:27:53.620 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.395775846Z level=info msg="Executing migration" id="Add read_only data column" 2024-03-21T13:27:53.620 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.398452651Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-03-21T13:27:53.620 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.401245548Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-03-21T13:27:53.620 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.403885362Z level=info msg="Executing migration" id="Add uid column" 2024-03-21T13:27:53.620 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.40678377Z level=info msg="Executing migration" id="Update uid value" 2024-03-21T13:27:53.620 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.409696215Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-03-21T13:27:53.620 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.412165425Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-03-21T13:27:53.620 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.41499943Z level=info msg="Executing migration" id="create api_key table" 2024-03-21T13:27:53.621 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.417606391Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-03-21T13:27:53.621 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.420215029Z level=info msg="Executing migration" id="add index api_key.key" 2024-03-21T13:27:53.621 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.42302609Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-03-21T13:27:53.621 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.425593702Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-03-21T13:27:53.621 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.428194148Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-03-21T13:27:53.621 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.430908144Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-03-21T13:27:53.621 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.433552124Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-03-21T13:27:53.621 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.435085094Z level=info msg="Executing migration" id="create api_key table v2" 2024-03-21T13:27:53.621 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.437749948Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-03-21T13:27:53.621 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.440384137Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-03-21T13:27:53.621 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.44298674Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-03-21T13:27:53.621 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.443443223Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-03-21T13:27:53.621 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.446027469Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-03-21T13:27:53.622 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.446416826Z level=info msg="Executing migration" id="Update api_key table charset" 2024-03-21T13:27:53.623 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.44852301Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-03-21T13:27:53.623 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.449123171Z level=info msg="Executing migration" id="Add service account foreign key" 2024-03-21T13:27:53.623 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.449700165Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-03-21T13:27:53.623 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.449996878Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-03-21T13:27:53.623 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.450581088Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-03-21T13:27:53.623 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.451179038Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-03-21T13:27:53.623 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.452181116Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-03-21T13:27:53.623 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.45286913Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-03-21T13:27:53.623 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.453280326Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-03-21T13:27:53.623 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.454089918Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-03-21T13:27:53.623 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.455081156Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-03-21T13:27:53.623 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.455922074Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-03-21T13:27:53.623 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.456404363Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-03-21T13:27:53.624 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.457226953Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-03-21T13:27:53.624 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.45787626Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-03-21T13:27:53.624 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.458506931Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-03-21T13:27:53.624 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.459329059Z level=info msg="Executing migration" id="create quota table v1" 2024-03-21T13:27:53.624 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.459755419Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-03-21T13:27:53.624 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.460195841Z level=info msg="Executing migration" id="Update quota table charset" 2024-03-21T13:27:53.624 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.46104426Z level=info msg="Executing migration" id="create plugin_setting table" 2024-03-21T13:27:53.624 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.461920853Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-03-21T13:27:53.624 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.46289491Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-03-21T13:27:53.624 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.463600328Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-03-21T13:27:53.624 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.464419729Z level=info msg="Executing migration" id="create session table" 2024-03-21T13:27:53.624 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.464874395Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-03-21T13:27:53.624 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.465440622Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-03-21T13:27:53.624 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.466223681Z level=info msg="Executing migration" id="create playlist table v2" 2024-03-21T13:27:53.624 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.466641492Z level=info msg="Executing migration" id="create playlist item table v2" 2024-03-21T13:27:53.624 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.467514989Z level=info msg="Executing migration" id="Update playlist table charset" 2024-03-21T13:27:53.625 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.468496952Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-03-21T13:27:53.625 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.469247504Z level=info msg="Executing migration" id="drop preferences table v2" 2024-03-21T13:27:53.625 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.469574529Z level=info msg="Executing migration" id="drop preferences table v3" 2024-03-21T13:27:53.625 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.470424736Z level=info msg="Executing migration" id="create preferences table v3" 2024-03-21T13:27:53.625 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.470865407Z level=info msg="Executing migration" id="Update preferences table charset" 2024-03-21T13:27:53.625 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.471287938Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-03-21T13:27:53.625 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.472177885Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-03-21T13:27:53.625 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.4724991Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-03-21T13:27:53.625 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.475502726Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-03-21T13:27:53.625 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.478351857Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-03-21T13:27:53.625 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.481294328Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-03-21T13:27:53.625 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.481741508Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-03-21T13:27:53.625 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.484752568Z level=info msg="Executing migration" id="create alert table v1" 2024-03-21T13:27:53.625 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.487315459Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-03-21T13:27:53.625 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.490076989Z level=info msg="Executing migration" id="add index alert state" 2024-03-21T13:27:53.625 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.490515848Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-03-21T13:27:53.626 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.493481065Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-03-21T13:27:53.626 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.496359442Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-03-21T13:27:53.626 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.499179216Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-03-21T13:27:53.626 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.499596596Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-03-21T13:27:53.626 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.502355673Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-03-21T13:27:53.626 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.505324344Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-03-21T13:27:53.626 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.508208204Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-03-21T13:27:53.626 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.511099466Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-03-21T13:27:53.626 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.513840793Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-03-21T13:27:53.626 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.5163357Z level=info msg="Executing migration" id="Add column is_default" 2024-03-21T13:27:53.626 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.518880331Z level=info msg="Executing migration" id="Add column frequency" 2024-03-21T13:27:53.626 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.521381242Z level=info msg="Executing migration" id="Add column send_reminder" 2024-03-21T13:27:53.626 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.524256968Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-03-21T13:27:53.626 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.526867837Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-03-21T13:27:53.627 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.52934598Z level=info msg="Executing migration" id="Update alert table charset" 2024-03-21T13:27:53.627 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.532029559Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-03-21T13:27:53.627 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.534754798Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-03-21T13:27:53.627 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.537255599Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-03-21T13:27:53.627 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.540072824Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-03-21T13:27:53.627 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.542580196Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-03-21T13:27:53.627 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.545165424Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-03-21T13:27:53.627 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.547666799Z level=info msg="Executing migration" id="Add for to alert table" 2024-03-21T13:27:53.627 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.550147422Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-03-21T13:27:53.627 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.552766683Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-03-21T13:27:53.627 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.555309076Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-03-21T13:27:53.627 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.558025817Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-03-21T13:27:53.627 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.558672993Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-03-21T13:27:53.627 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.559568262Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-03-21T13:27:53.627 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.560472617Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-03-21T13:27:53.627 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.561295009Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-03-21T13:27:53.628 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.562178832Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-03-21T13:27:53.628 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.563033528Z level=info msg="Executing migration" id="create annotation table v5" 2024-03-21T13:27:53.628 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.563878864Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-03-21T13:27:53.628 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.564729509Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-03-21T13:27:53.628 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.565545351Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-03-21T13:27:53.628 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.566557711Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-03-21T13:27:53.628 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.567291004Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-03-21T13:27:53.628 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.568135071Z level=info msg="Executing migration" id="Update annotation table charset" 2024-03-21T13:27:53.628 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.568999657Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-03-21T13:27:53.628 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.569834017Z level=info msg="Executing migration" id="Drop category_id index" 2024-03-21T13:27:53.628 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.570799112Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-03-21T13:27:53.628 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.571673796Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-03-21T13:27:53.628 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.572682964Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-03-21T13:27:53.628 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.57352097Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-03-21T13:27:53.628 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.574258816Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-03-21T13:27:53.629 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.576266413Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-03-21T13:27:53.629 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.577119669Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-03-21T13:27:53.629 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.578122769Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-03-21T13:27:53.629 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.578875433Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-03-21T13:27:53.629 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.579745146Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-03-21T13:27:53.629 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.580750387Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-03-21T13:27:53.629 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.581587681Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-03-21T13:27:53.629 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.582433533Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-03-21T13:27:53.629 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.583322845Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-03-21T13:27:53.629 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.584286364Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-03-21T13:27:53.629 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.585038349Z level=info msg="Executing migration" id="Add epoch_end column" 2024-03-21T13:27:53.629 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.585898254Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-03-21T13:27:53.629 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.586759164Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-03-21T13:27:53.629 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.587738646Z level=info msg="Executing migration" id="Move region to single row" 2024-03-21T13:27:53.630 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.58850983Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-03-21T13:27:53.630 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.589368689Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-03-21T13:27:53.630 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.592171991Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-03-21T13:27:53.630 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.595050688Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-03-21T13:27:53.630 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.595509113Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-03-21T13:27:53.630 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.596379936Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-03-21T13:27:53.630 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.596841507Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-03-21T13:27:53.630 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.599437523Z level=info msg="Executing migration" id="create test_data table" 2024-03-21T13:27:53.630 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.600039665Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-03-21T13:27:53.630 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.602234684Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-03-21T13:27:53.630 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.605057981Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-03-21T13:27:53.630 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.607714277Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-03-21T13:27:53.630 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.610678256Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-03-21T13:27:53.630 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.613431361Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-03-21T13:27:53.631 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.616198365Z level=info msg="Executing migration" id="create team table" 2024-03-21T13:27:53.631 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.619035239Z level=info msg="Executing migration" id="add index team.org_id" 2024-03-21T13:27:53.631 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.621961413Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-03-21T13:27:53.868 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-mon[33633]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:53.869 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:53.869 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:53.869 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:53.869 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:53.870 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.62477649Z level=info msg="Executing migration" id="create team member table" 2024-03-21T13:27:53.870 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.627597382Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-03-21T13:27:53.870 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.630137132Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-03-21T13:27:53.870 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.63281154Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-03-21T13:27:53.870 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.635604349Z level=info msg="Executing migration" id="Add column email to team table" 2024-03-21T13:27:53.870 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.638239624Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-03-21T13:27:53.870 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.640949755Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-03-21T13:27:53.870 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.643657236Z level=info msg="Executing migration" id="create dashboard acl table" 2024-03-21T13:27:53.870 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.64643981Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-03-21T13:27:53.870 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.649089582Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-03-21T13:27:53.870 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.651850742Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-03-21T13:27:53.871 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.652323856Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-03-21T13:27:53.871 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.655088443Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-03-21T13:27:53.871 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.6578231Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-03-21T13:27:53.871 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.658281503Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-03-21T13:27:53.871 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.658703789Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-03-21T13:27:53.871 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.659115258Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-03-21T13:27:53.871 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.65940533Z level=info msg="Executing migration" id="create tag table" 2024-03-21T13:27:53.871 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.659766372Z level=info msg="Executing migration" id="add index tag.key_value" 2024-03-21T13:27:53.871 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.66019656Z level=info msg="Executing migration" id="create login attempt table" 2024-03-21T13:27:53.871 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.660627034Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-03-21T13:27:53.871 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.661029903Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-03-21T13:27:53.871 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.661444485Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-03-21T13:27:53.871 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.663755318Z level=info msg="Executing migration" id="create login_attempt v2" 2024-03-21T13:27:53.871 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.664150925Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-03-21T13:27:53.871 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.664566377Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-03-21T13:27:53.872 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.664839035Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-03-21T13:27:53.872 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.665191786Z level=info msg="Executing migration" id="create user auth table" 2024-03-21T13:27:53.872 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.665551093Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-03-21T13:27:53.872 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.665951152Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-03-21T13:27:53.872 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.666227251Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-03-21T13:27:53.872 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.666992838Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-03-21T13:27:53.872 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.667802853Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-03-21T13:27:53.872 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.668653608Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-03-21T13:27:53.872 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.669540974Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-03-21T13:27:53.872 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.669980928Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-03-21T13:27:53.872 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.670832381Z level=info msg="Executing migration" id="create server_lock table" 2024-03-21T13:27:53.872 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.671208689Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-03-21T13:27:53.872 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.671622989Z level=info msg="Executing migration" id="create user auth token table" 2024-03-21T13:27:53.872 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.672025146Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-03-21T13:27:53.873 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.672434822Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-03-21T13:27:53.873 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.67283326Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-03-21T13:27:53.873 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.673239875Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-03-21T13:27:53.873 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.674122142Z level=info msg="Executing migration" id="create cache_data table" 2024-03-21T13:27:53.873 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.674514003Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-03-21T13:27:53.873 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.674917545Z level=info msg="Executing migration" id="create short_url table v1" 2024-03-21T13:27:53.873 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.675330893Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-03-21T13:27:53.873 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.677885278Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-03-21T13:27:53.873 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.67819857Z level=info msg="Executing migration" id="delete alert_definition table" 2024-03-21T13:27:53.873 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.678457919Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-03-21T13:27:53.873 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.678875374Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-03-21T13:27:53.873 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.681654778Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-03-21T13:27:53.873 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.682245898Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-03-21T13:27:53.873 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.682530292Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-03-21T13:27:53.874 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.682939359Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-03-21T13:27:53.874 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.683339691Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-03-21T13:27:53.875 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.683720283Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-03-21T13:27:53.875 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.684124506Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-03-21T13:27:53.875 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.684989152Z level=info msg="Executing migration" id="drop alert_definition table" 2024-03-21T13:27:53.875 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.685403976Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-03-21T13:27:53.875 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.685661351Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-03-21T13:27:53.875 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.686265851Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-03-21T13:27:53.875 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.688862259Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-03-21T13:27:53.875 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.691549956Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-03-21T13:27:53.875 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.691930173Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-03-21T13:27:53.875 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.694159942Z level=info msg="Executing migration" id="create alert_instance table" 2024-03-21T13:27:53.875 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.697055846Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-03-21T13:27:53.876 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.699735995Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-03-21T13:27:53.876 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.700171715Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-03-21T13:27:53.876 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.70104451Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-03-21T13:27:53.876 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.702342161Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-03-21T13:27:53.876 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.705065801Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-03-21T13:27:53.876 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.709623067Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-03-21T13:27:53.876 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.713860662Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-03-21T13:27:53.876 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.714299033Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-03-21T13:27:53.876 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.715272401Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-03-21T13:27:53.876 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.716200864Z level=info msg="Executing migration" id="create alert_rule table" 2024-03-21T13:27:53.876 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.716652912Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-03-21T13:27:53.876 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.717518439Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-03-21T13:27:53.876 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.718483005Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-03-21T13:27:53.877 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.71914892Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-03-21T13:27:53.877 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.720091116Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-03-21T13:27:53.877 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.721122968Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-03-21T13:27:53.877 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.722044547Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-03-21T13:27:53.877 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.722945132Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-03-21T13:27:53.877 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.723823041Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-03-21T13:27:53.877 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.724605145Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-03-21T13:27:53.877 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.72552334Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-03-21T13:27:53.877 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.726450433Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-03-21T13:27:53.877 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.727253651Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-03-21T13:27:53.877 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.728173899Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-03-21T13:27:53.877 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.729080418Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-03-21T13:27:53.877 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.729950565Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-03-21T13:27:53.877 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.730764304Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-03-21T13:27:53.877 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.7316486Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-03-21T13:27:53.878 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.732519512Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-03-21T13:27:53.878 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.733021849Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-03-21T13:27:53.878 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.733995461Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-03-21T13:27:53.878 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.734908775Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-03-21T13:27:53.878 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.735825671Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-03-21T13:27:53.878 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.736747588Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-03-21T13:27:53.878 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.737698527Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-03-21T13:27:53.878 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.738554225Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-03-21T13:27:53.878 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.739441652Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-03-21T13:27:53.878 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.740396355Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-03-21T13:27:53.878 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.741405003Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-03-21T13:27:53.878 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.742535903Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-03-21T13:27:53.878 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.742972841Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-03-21T13:27:53.878 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.743907106Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-03-21T13:27:53.879 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.744865534Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-03-21T13:27:53.879 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.745625111Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-03-21T13:27:53.879 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.746596173Z level=info msg="Executing migration" id="create provenance_type table" 2024-03-21T13:27:53.879 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.74948673Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-03-21T13:27:53.879 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.751993597Z level=info msg="Executing migration" id="create alert_image table" 2024-03-21T13:27:53.879 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.754678622Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-03-21T13:27:53.879 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.755139663Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-03-21T13:27:53.879 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.757860919Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-03-21T13:27:53.879 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.760917371Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-03-21T13:27:53.879 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.761544287Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-03-21T13:27:53.879 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.761620886Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-03-21T13:27:53.879 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.761855394Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-03-21T13:27:53.879 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.76221245Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-03-21T13:27:53.879 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.762723501Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-03-21T13:27:53.879 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.762829096Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-03-21T13:27:53.879 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.762927606Z level=info msg="alerts found to migrate" alerts=0 2024-03-21T13:27:53.880 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.763617396Z level=info msg="Executing migration" id="create library_element table v1" 2024-03-21T13:27:53.880 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.76449415Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-03-21T13:27:53.880 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.764927115Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-03-21T13:27:53.880 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.767709826Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-03-21T13:27:53.880 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.770639995Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-03-21T13:27:53.880 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.773395036Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-03-21T13:27:53.880 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.773691301Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-03-21T13:27:53.880 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.776592811Z level=info msg="Executing migration" id="create data_keys table" 2024-03-21T13:27:53.880 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.777148208Z level=info msg="Executing migration" id="create secrets table" 2024-03-21T13:27:53.880 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.779364339Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-03-21T13:27:53.880 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.784755811Z level=info msg="Executing migration" id="add name column into data_keys" 2024-03-21T13:27:53.880 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.785799271Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-03-21T13:27:53.880 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.78860288Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-03-21T13:27:53.880 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.794074939Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-03-21T13:27:53.880 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.799757852Z level=info msg="Executing migration" id="create kv_store table v1" 2024-03-21T13:27:53.881 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.800170282Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-03-21T13:27:53.881 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.80580748Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-03-21T13:27:53.881 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.806162219Z level=info msg="Executing migration" id="create permission table" 2024-03-21T13:27:53.881 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.80928748Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-03-21T13:27:53.881 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.811889087Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-03-21T13:27:53.881 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.814614194Z level=info msg="Executing migration" id="create role table" 2024-03-21T13:27:53.881 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.817152578Z level=info msg="Executing migration" id="add column display_name" 2024-03-21T13:27:53.881 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.819791597Z level=info msg="Executing migration" id="add column group_name" 2024-03-21T13:27:53.881 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.822331383Z level=info msg="Executing migration" id="add index role.org_id" 2024-03-21T13:27:53.881 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.8250126Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-03-21T13:27:53.881 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.827958701Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-03-21T13:27:53.881 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.830447281Z level=info msg="Executing migration" id="create team role table" 2024-03-21T13:27:53.881 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.833000748Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-03-21T13:27:53.881 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.835784213Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-03-21T13:27:53.881 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.838120534Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-03-21T13:27:53.882 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.840753251Z level=info msg="Executing migration" id="create user role table" 2024-03-21T13:27:53.882 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.84346313Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-03-21T13:27:53.882 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.8458799Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-03-21T13:27:53.882 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.84847882Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-03-21T13:27:53.882 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.851124945Z level=info msg="Executing migration" id="create builtin role table" 2024-03-21T13:27:53.882 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.853732885Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-03-21T13:27:53.882 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.856259179Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-03-21T13:27:53.882 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.858798302Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-03-21T13:27:53.882 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.859960633Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-03-21T13:27:53.882 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.860971397Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-03-21T13:27:53.882 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.861682019Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-03-21T13:27:53.882 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.862627993Z level=info msg="Executing migration" id="add unique index role.uid" 2024-03-21T13:27:53.882 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.863349151Z level=info msg="Executing migration" id="create seed assignment table" 2024-03-21T13:27:53.882 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.864315229Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-03-21T13:27:53.882 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.864982861Z level=info msg="Executing migration" id="add column hidden to role table" 2024-03-21T13:27:53.883 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.86614188Z level=info msg="Executing migration" id="create query_history table v1" 2024-03-21T13:27:53.883 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.866898983Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-03-21T13:27:53.883 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.867884511Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-03-21T13:27:53.883 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.868594739Z level=info msg="Executing migration" id="teams permissions migration" 2024-03-21T13:27:53.883 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.869422155Z level=info msg="Executing migration" id="dashboard permissions" 2024-03-21T13:27:53.883 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.869970169Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-03-21T13:27:53.883 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.870393537Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-03-21T13:27:53.883 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.871199427Z level=info msg="Executing migration" id="alerting notification permissions" 2024-03-21T13:27:53.883 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.871545587Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-03-21T13:27:53.883 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.87209395Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-03-21T13:27:53.883 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.872928369Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-03-21T13:27:54.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:53 smithi097 ceph-mon[39416]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:54.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:53 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:54.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:53 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:54.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:53 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:54.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:53 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:54.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:53 smithi097 ceph-mon[30040]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:54.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:53 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:54.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:53 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:54.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:53 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:54.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:53 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:54.121 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.874272929Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-03-21T13:27:54.122 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.875162993Z level=info msg="Executing migration" id="create correlation table v1" 2024-03-21T13:27:54.122 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.876029573Z level=info msg="Executing migration" id="add index correlations.uid" 2024-03-21T13:27:54.122 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.876890896Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-03-21T13:27:54.122 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.877777715Z level=info msg="Executing migration" id="add correlation config column" 2024-03-21T13:27:54.122 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.879166227Z level=info msg="Executing migration" id="create entity_events table" 2024-03-21T13:27:54.122 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.880021833Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-03-21T13:27:54.122 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.880957214Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-03-21T13:27:54.122 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.881053195Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-03-21T13:27:54.122 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.881831544Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-03-21T13:27:54.122 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.881917952Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-03-21T13:27:54.122 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.88271477Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-03-21T13:27:54.123 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.883705152Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-03-21T13:27:54.123 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.884458826Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-03-21T13:27:54.123 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.885429007Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-03-21T13:27:54.123 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.8861408Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-03-21T13:27:54.123 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.887019079Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-03-21T13:27:54.123 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.888016414Z level=info msg="Executing migration" id="Drop public config table" 2024-03-21T13:27:54.123 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.888799682Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-03-21T13:27:54.123 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.891621125Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-03-21T13:27:54.123 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.894625724Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-03-21T13:27:54.123 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.897490943Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-03-21T13:27:54.123 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.900431838Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-03-21T13:27:54.123 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.904240127Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-03-21T13:27:54.123 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.907145235Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-03-21T13:27:54.123 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.910031691Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-03-21T13:27:54.124 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.913016833Z level=info msg="Executing migration" id="add share column" 2024-03-21T13:27:54.124 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.915965072Z level=info msg="Executing migration" id="create default alerting folders" 2024-03-21T13:27:54.124 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.918767938Z level=info msg="Executing migration" id="create file table" 2024-03-21T13:27:54.124 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.921674343Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-03-21T13:27:54.124 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.924568731Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-03-21T13:27:54.124 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.927333911Z level=info msg="Executing migration" id="create file_meta table" 2024-03-21T13:27:54.124 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.930141252Z level=info msg="Executing migration" id="file table idx: path key" 2024-03-21T13:27:54.124 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.93307527Z level=info msg="Executing migration" id="set path collation in file table" 2024-03-21T13:27:54.124 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.93577142Z level=info msg="Executing migration" id="managed permissions migration" 2024-03-21T13:27:54.124 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.938894852Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-03-21T13:27:54.124 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.941820741Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-03-21T13:27:54.125 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.944417551Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-03-21T13:27:54.125 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.946985284Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-03-21T13:27:54.125 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.949563771Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-03-21T13:27:54.125 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.952302329Z level=info msg="Executing migration" id="update group index for alert rules" 2024-03-21T13:27:54.125 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.955040304Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-03-21T13:27:54.125 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.957817149Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-03-21T13:27:54.125 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.960191813Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-03-21T13:27:54.125 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.962800458Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-03-21T13:27:54.125 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.965578345Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-03-21T13:27:54.125 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.968041731Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-03-21T13:27:54.125 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.980928753Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-03-21T13:27:54.126 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.983591474Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-03-21T13:27:54.126 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.984056667Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-03-21T13:27:54.126 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.988326851Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-03-21T13:27:54.126 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.991100304Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-03-21T13:27:54.126 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.99380006Z level=info msg="Executing migration" id="create folder table" 2024-03-21T13:27:54.126 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.994241012Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-03-21T13:27:54.126 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.9950507Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-03-21T13:27:54.126 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.995926816Z level=info msg="Executing migration" id="Update folder title length" 2024-03-21T13:27:54.126 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.996770413Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-03-21T13:27:54.126 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=migrator t=2024-03-21T13:27:53.997219672Z level=info msg="migrations completed" performed=485 skipped=0 duration=726.552294ms 2024-03-21T13:27:54.126 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=sqlstore t=2024-03-21T13:27:53.997639796Z level=info msg="Created default organization" 2024-03-21T13:27:54.126 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:54 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=plugin.finder t=2024-03-21T13:27:54.028936797Z level=warn msg="Skipping finding plugins as directory does not exist" path=/usr/share/grafana/plugins-bundled 2024-03-21T13:27:54.126 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:54 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=secrets t=2024-03-21T13:27:54.029020717Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-03-21T13:27:54.126 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:54 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=query_data t=2024-03-21T13:27:54.031182551Z level=info msg="Query Service initialization" 2024-03-21T13:27:54.127 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:54 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=live.push_http t=2024-03-21T13:27:54.03530894Z level=info msg="Live Push Gateway initialization" 2024-03-21T13:27:54.459 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:54 smithi097 systemd[1]: Starting Ceph node-exporter.a for a5dd1f32-e785-11ee-95cd-87774f69a715... 2024-03-21T13:27:54.803 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:54 smithi097 bash[77816]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-03-21T13:27:55.097 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:54 smithi097 ceph-mon[39416]: Deploying daemon node-exporter.a on smithi097 2024-03-21T13:27:55.097 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:54 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:27:55.097 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:54 smithi097 ceph-mon[30040]: Deploying daemon node-exporter.a on smithi097 2024-03-21T13:27:55.097 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:54 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:27:55.120 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:54 smithi144 ceph-mon[33633]: Deploying daemon node-exporter.a on smithi097 2024-03-21T13:27:55.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:54 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:27:55.121 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:54 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=infra.usagestats.collector t=2024-03-21T13:27:54.633194756Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-03-21T13:27:55.121 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:54 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=provisioning.datasources t=2024-03-21T13:27:54.634175505Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-03-21T13:27:55.121 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:54 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=provisioning.datasources t=2024-03-21T13:27:54.645081714Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-03-21T13:27:55.121 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:54 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=provisioning.alerting t=2024-03-21T13:27:54.655490483Z level=info msg="starting to provision alerting" 2024-03-21T13:27:55.121 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:54 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=provisioning.alerting t=2024-03-21T13:27:54.655503555Z level=info msg="finished to provision alerting" 2024-03-21T13:27:55.121 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:54 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=grafanaStorageLogger t=2024-03-21T13:27:54.655610604Z level=info msg="storage starting" 2024-03-21T13:27:55.121 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:54 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=ngalert.state.manager t=2024-03-21T13:27:54.655910243Z level=info msg="Warming state cache for startup" 2024-03-21T13:27:55.122 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:54 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=http.server t=2024-03-21T13:27:54.657456079Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-03-21T13:27:55.122 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:54 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=ngalert.state.manager t=2024-03-21T13:27:54.664487629Z level=info msg="State cache has been initialized" states=0 duration=8.57666ms 2024-03-21T13:27:55.122 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:54 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=ticker t=2024-03-21T13:27:54.664531617Z level=info msg=starting first_tick=2024-03-21T13:28:00Z 2024-03-21T13:27:55.122 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:54 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=ngalert.multiorg.alertmanager t=2024-03-21T13:27:54.664539468Z level=info msg="starting MultiOrg Alertmanager" 2024-03-21T13:27:55.122 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:27:54 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=sqlstore.transactions t=2024-03-21T13:27:54.670080946Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-03-21T13:27:55.553 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:55 smithi097 bash[77816]: Getting image source signatures 2024-03-21T13:27:55.553 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:55 smithi097 bash[77816]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-03-21T13:27:55.553 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:55 smithi097 bash[77816]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-03-21T13:27:55.553 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:55 smithi097 bash[77816]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-03-21T13:27:56.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:55 smithi144 ceph-mon[33633]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:27:56.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:55 smithi097 ceph-mon[39416]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:27:56.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:55 smithi097 ceph-mon[30040]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:27:57.303 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:56 smithi097 bash[77816]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-03-21T13:27:57.303 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 bash[77816]: Writing manifest to image destination 2024-03-21T13:27:57.709 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 podman[77816]: 2024-03-21 13:27:57.344211215 +0000 UTC m=+2.977129793 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-03-21T13:27:57.709 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 podman[77816]: 2024-03-21 13:27:57.484209759 +0000 UTC m=+3.117128334 container create d296a13aefa1785643bc968f14c17d81489509ee19be68945462d80fe93ba404 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-03-21T13:27:57.709 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 podman[77816]: 2024-03-21 13:27:57.709561751 +0000 UTC m=+3.342480326 container init d296a13aefa1785643bc968f14c17d81489509ee19be68945462d80fe93ba404 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-03-21T13:27:58.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-mon[39416]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:58.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:58.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-mon[30040]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:58.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:58.054 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 podman[77816]: 2024-03-21 13:27:57.715256408 +0000 UTC m=+3.348174984 container start d296a13aefa1785643bc968f14c17d81489509ee19be68945462d80fe93ba404 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-03-21T13:27:58.054 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.719Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-03-21T13:27:58.054 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.719Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-03-21T13:27:58.054 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.719Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-03-21T13:27:58.054 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.719Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-03-21T13:27:58.054 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.720Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-03-21T13:27:58.054 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.720Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-03-21T13:27:58.054 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.720Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-03-21T13:27:58.055 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.720Z caller=node_exporter.go:117 level=info collector=arp 2024-03-21T13:27:58.055 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.720Z caller=node_exporter.go:117 level=info collector=bcache 2024-03-21T13:27:58.055 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.720Z caller=node_exporter.go:117 level=info collector=bonding 2024-03-21T13:27:58.055 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.720Z caller=node_exporter.go:117 level=info collector=btrfs 2024-03-21T13:27:58.055 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.720Z caller=node_exporter.go:117 level=info collector=conntrack 2024-03-21T13:27:58.055 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.720Z caller=node_exporter.go:117 level=info collector=cpu 2024-03-21T13:27:58.055 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.720Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-03-21T13:27:58.055 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.720Z caller=node_exporter.go:117 level=info collector=diskstats 2024-03-21T13:27:58.055 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.720Z caller=node_exporter.go:117 level=info collector=dmi 2024-03-21T13:27:58.055 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.720Z caller=node_exporter.go:117 level=info collector=edac 2024-03-21T13:27:58.055 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.720Z caller=node_exporter.go:117 level=info collector=entropy 2024-03-21T13:27:58.055 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.720Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-03-21T13:27:58.055 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.720Z caller=node_exporter.go:117 level=info collector=filefd 2024-03-21T13:27:58.055 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.720Z caller=node_exporter.go:117 level=info collector=filesystem 2024-03-21T13:27:58.056 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.720Z caller=node_exporter.go:117 level=info collector=hwmon 2024-03-21T13:27:58.056 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.720Z caller=node_exporter.go:117 level=info collector=infiniband 2024-03-21T13:27:58.056 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.720Z caller=node_exporter.go:117 level=info collector=ipvs 2024-03-21T13:27:58.056 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.720Z caller=node_exporter.go:117 level=info collector=loadavg 2024-03-21T13:27:58.056 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.720Z caller=node_exporter.go:117 level=info collector=mdadm 2024-03-21T13:27:58.056 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.720Z caller=node_exporter.go:117 level=info collector=meminfo 2024-03-21T13:27:58.056 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.720Z caller=node_exporter.go:117 level=info collector=netclass 2024-03-21T13:27:58.056 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.720Z caller=node_exporter.go:117 level=info collector=netdev 2024-03-21T13:27:58.056 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.720Z caller=node_exporter.go:117 level=info collector=netstat 2024-03-21T13:27:58.056 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.720Z caller=node_exporter.go:117 level=info collector=nfs 2024-03-21T13:27:58.056 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.720Z caller=node_exporter.go:117 level=info collector=nfsd 2024-03-21T13:27:58.056 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.720Z caller=node_exporter.go:117 level=info collector=nvme 2024-03-21T13:27:58.056 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.720Z caller=node_exporter.go:117 level=info collector=os 2024-03-21T13:27:58.056 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.720Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-03-21T13:27:58.057 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.720Z caller=node_exporter.go:117 level=info collector=pressure 2024-03-21T13:27:58.057 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.720Z caller=node_exporter.go:117 level=info collector=rapl 2024-03-21T13:27:58.057 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.720Z caller=node_exporter.go:117 level=info collector=schedstat 2024-03-21T13:27:58.057 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.720Z caller=node_exporter.go:117 level=info collector=selinux 2024-03-21T13:27:58.057 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.720Z caller=node_exporter.go:117 level=info collector=sockstat 2024-03-21T13:27:58.057 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.720Z caller=node_exporter.go:117 level=info collector=softnet 2024-03-21T13:27:58.057 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.720Z caller=node_exporter.go:117 level=info collector=stat 2024-03-21T13:27:58.057 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.720Z caller=node_exporter.go:117 level=info collector=tapestats 2024-03-21T13:27:58.057 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.720Z caller=node_exporter.go:117 level=info collector=textfile 2024-03-21T13:27:58.057 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.720Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-03-21T13:27:58.058 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.720Z caller=node_exporter.go:117 level=info collector=time 2024-03-21T13:27:58.058 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.720Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-03-21T13:27:58.058 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.720Z caller=node_exporter.go:117 level=info collector=uname 2024-03-21T13:27:58.058 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.720Z caller=node_exporter.go:117 level=info collector=vmstat 2024-03-21T13:27:58.058 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.720Z caller=node_exporter.go:117 level=info collector=xfs 2024-03-21T13:27:58.058 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.720Z caller=node_exporter.go:117 level=info collector=zfs 2024-03-21T13:27:58.058 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.721Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-03-21T13:27:58.058 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a[77866]: ts=2024-03-21T13:27:57.721Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-03-21T13:27:58.058 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 bash[77816]: d296a13aefa1785643bc968f14c17d81489509ee19be68945462d80fe93ba404 2024-03-21T13:27:58.058 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 13:27:57 smithi097 systemd[1]: Started Ceph node-exporter.a for a5dd1f32-e785-11ee-95cd-87774f69a715. 2024-03-21T13:27:58.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:57 smithi144 ceph-mon[33633]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:58.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:57 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:59.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:58 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:59.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:58 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:59.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:58 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:59.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:58 smithi144 ceph-mon[33633]: Deploying daemon node-exporter.b on smithi144 2024-03-21T13:27:59.122 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:27:59 smithi144 bash[66561]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-03-21T13:27:59.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:58 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:59.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:58 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:59.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:58 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:59.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:58 smithi097 ceph-mon[39416]: Deploying daemon node-exporter.b on smithi144 2024-03-21T13:27:59.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:58 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:59.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:58 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:59.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:58 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:27:59.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:58 smithi097 ceph-mon[30040]: Deploying daemon node-exporter.b on smithi144 2024-03-21T13:27:59.871 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:27:59 smithi144 bash[66561]: Getting image source signatures 2024-03-21T13:27:59.871 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:27:59 smithi144 bash[66561]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-03-21T13:27:59.871 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:27:59 smithi144 bash[66561]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-03-21T13:27:59.871 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:27:59 smithi144 bash[66561]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-03-21T13:28:00.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:27:59 smithi097 ceph-mon[30040]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:00.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:27:59 smithi097 ceph-mon[39416]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:00.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:27:59 smithi144 ceph-mon[33633]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:01.121 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:00 smithi144 bash[66561]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-03-21T13:28:01.121 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 bash[66561]: Writing manifest to image destination 2024-03-21T13:28:01.621 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 podman[66561]: 2024-03-21 13:28:01.324548052 +0000 UTC m=+2.379726908 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-03-21T13:28:01.621 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 podman[66561]: 2024-03-21 13:28:01.431466485 +0000 UTC m=+2.486645329 container create 4c4e84e2fcaed915f9b00d05fd0021adb18fb96101c3cbc0473757d4594b379f (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-03-21T13:28:02.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-mon[33633]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:02.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:02.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:02.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:02.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:02.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:02.122 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 podman[66561]: 2024-03-21 13:28:01.683677152 +0000 UTC m=+2.738856006 container init 4c4e84e2fcaed915f9b00d05fd0021adb18fb96101c3cbc0473757d4594b379f (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-03-21T13:28:02.122 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 podman[66561]: 2024-03-21 13:28:01.689859256 +0000 UTC m=+2.745038107 container start 4c4e84e2fcaed915f9b00d05fd0021adb18fb96101c3cbc0473757d4594b379f (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-03-21T13:28:02.122 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.694Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-03-21T13:28:02.122 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.694Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-03-21T13:28:02.122 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.694Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-03-21T13:28:02.122 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.694Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-03-21T13:28:02.122 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.695Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-03-21T13:28:02.122 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.695Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-03-21T13:28:02.122 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.695Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-03-21T13:28:02.122 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.695Z caller=node_exporter.go:117 level=info collector=arp 2024-03-21T13:28:02.122 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.695Z caller=node_exporter.go:117 level=info collector=bcache 2024-03-21T13:28:02.122 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.695Z caller=node_exporter.go:117 level=info collector=bonding 2024-03-21T13:28:02.123 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.695Z caller=node_exporter.go:117 level=info collector=btrfs 2024-03-21T13:28:02.123 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.695Z caller=node_exporter.go:117 level=info collector=conntrack 2024-03-21T13:28:02.123 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.695Z caller=node_exporter.go:117 level=info collector=cpu 2024-03-21T13:28:02.123 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.695Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-03-21T13:28:02.123 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.695Z caller=node_exporter.go:117 level=info collector=diskstats 2024-03-21T13:28:02.123 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.695Z caller=node_exporter.go:117 level=info collector=dmi 2024-03-21T13:28:02.123 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.695Z caller=node_exporter.go:117 level=info collector=edac 2024-03-21T13:28:02.123 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.695Z caller=node_exporter.go:117 level=info collector=entropy 2024-03-21T13:28:02.124 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.695Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-03-21T13:28:02.124 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.695Z caller=node_exporter.go:117 level=info collector=filefd 2024-03-21T13:28:02.124 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.695Z caller=node_exporter.go:117 level=info collector=filesystem 2024-03-21T13:28:02.124 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.695Z caller=node_exporter.go:117 level=info collector=hwmon 2024-03-21T13:28:02.124 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.695Z caller=node_exporter.go:117 level=info collector=infiniband 2024-03-21T13:28:02.124 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.695Z caller=node_exporter.go:117 level=info collector=ipvs 2024-03-21T13:28:02.124 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.695Z caller=node_exporter.go:117 level=info collector=loadavg 2024-03-21T13:28:02.124 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.695Z caller=node_exporter.go:117 level=info collector=mdadm 2024-03-21T13:28:02.124 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.695Z caller=node_exporter.go:117 level=info collector=meminfo 2024-03-21T13:28:02.124 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.695Z caller=node_exporter.go:117 level=info collector=netclass 2024-03-21T13:28:02.124 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.695Z caller=node_exporter.go:117 level=info collector=netdev 2024-03-21T13:28:02.124 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.695Z caller=node_exporter.go:117 level=info collector=netstat 2024-03-21T13:28:02.124 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.695Z caller=node_exporter.go:117 level=info collector=nfs 2024-03-21T13:28:02.124 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.695Z caller=node_exporter.go:117 level=info collector=nfsd 2024-03-21T13:28:02.124 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.695Z caller=node_exporter.go:117 level=info collector=nvme 2024-03-21T13:28:02.124 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.695Z caller=node_exporter.go:117 level=info collector=os 2024-03-21T13:28:02.125 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.695Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-03-21T13:28:02.125 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.695Z caller=node_exporter.go:117 level=info collector=pressure 2024-03-21T13:28:02.125 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.695Z caller=node_exporter.go:117 level=info collector=rapl 2024-03-21T13:28:02.125 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.695Z caller=node_exporter.go:117 level=info collector=schedstat 2024-03-21T13:28:02.125 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.695Z caller=node_exporter.go:117 level=info collector=selinux 2024-03-21T13:28:02.125 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.695Z caller=node_exporter.go:117 level=info collector=sockstat 2024-03-21T13:28:02.125 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.695Z caller=node_exporter.go:117 level=info collector=softnet 2024-03-21T13:28:02.125 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.695Z caller=node_exporter.go:117 level=info collector=stat 2024-03-21T13:28:02.125 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.695Z caller=node_exporter.go:117 level=info collector=tapestats 2024-03-21T13:28:02.125 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.695Z caller=node_exporter.go:117 level=info collector=textfile 2024-03-21T13:28:02.125 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.695Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-03-21T13:28:02.125 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.695Z caller=node_exporter.go:117 level=info collector=time 2024-03-21T13:28:02.125 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.695Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-03-21T13:28:02.125 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.695Z caller=node_exporter.go:117 level=info collector=uname 2024-03-21T13:28:02.125 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.695Z caller=node_exporter.go:117 level=info collector=vmstat 2024-03-21T13:28:02.125 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.695Z caller=node_exporter.go:117 level=info collector=xfs 2024-03-21T13:28:02.126 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.695Z caller=node_exporter.go:117 level=info collector=zfs 2024-03-21T13:28:02.126 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.696Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-03-21T13:28:02.126 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b[66611]: ts=2024-03-21T13:28:01.696Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-03-21T13:28:02.126 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 bash[66561]: 4c4e84e2fcaed915f9b00d05fd0021adb18fb96101c3cbc0473757d4594b379f 2024-03-21T13:28:02.126 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 13:28:01 smithi144 systemd[1]: Started Ceph node-exporter.b for a5dd1f32-e785-11ee-95cd-87774f69a715. 2024-03-21T13:28:02.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:28:01 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:28:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:28:02.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:01 smithi097 ceph-mon[30040]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:02.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:01 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:02.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:01 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:02.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:01 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:02.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:01 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:02.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:01 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:02.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:01 smithi097 ceph-mon[39416]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:02.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:01 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:02.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:01 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:02.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:01 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:02.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:01 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:02.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:01 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:03.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:02 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:28:03.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:02 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:03.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:02 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:28:03.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:02 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:03.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:02 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:28:03.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:02 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:03.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:28:03 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:28:04.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:03 smithi144 ceph-mon[33633]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:04.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:03 smithi097 ceph-mon[39416]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:04.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:03 smithi097 ceph-mon[30040]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:05.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:04 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:28:05.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:04 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:28:05.265 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:04 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:28:06.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:05 smithi097 ceph-mon[39416]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:06.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:05 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:06.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:05 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:06.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:05 smithi097 ceph-mon[30040]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:06.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:05 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:06.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:05 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:06.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:05 smithi144 ceph-mon[33633]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:06.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:05 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:06.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:05 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:07.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:07 smithi097 ceph-mon[30040]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:07.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:07 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:07.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:07 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:07.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:07 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:28:07.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:07 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:28:07.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:07 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:07.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:07 smithi097 ceph-mon[30040]: Reconfiguring alertmanager.a (dependencies changed)... 2024-03-21T13:28:07.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:07 smithi097 ceph-mon[30040]: Reconfiguring daemon alertmanager.a on smithi097 2024-03-21T13:28:07.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:07 smithi097 ceph-mon[39416]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:07.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:07 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:07.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:07 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:07.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:07 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:28:07.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:07 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:28:07.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:07 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:07.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:07 smithi097 ceph-mon[39416]: Reconfiguring alertmanager.a (dependencies changed)... 2024-03-21T13:28:07.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:07 smithi097 ceph-mon[39416]: Reconfiguring daemon alertmanager.a on smithi097 2024-03-21T13:28:07.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:07 smithi144 ceph-mon[33633]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:07.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:07 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:07.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:07 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:07.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:07 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:28:07.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:07 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:28:07.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:07 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:07.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:07 smithi144 ceph-mon[33633]: Reconfiguring alertmanager.a (dependencies changed)... 2024-03-21T13:28:07.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:07 smithi144 ceph-mon[33633]: Reconfiguring daemon alertmanager.a on smithi097 2024-03-21T13:28:08.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:08 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:28:08.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:08 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:28:08.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:08 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:28:09.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:09 smithi097 ceph-mon[30040]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:09.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:09 smithi097 ceph-mon[39416]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:09.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:09 smithi144 ceph-mon[33633]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:10.803 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 13:28:10 smithi097 systemd[1]: Stopping Ceph alertmanager.a for a5dd1f32-e785-11ee-95cd-87774f69a715... 2024-03-21T13:28:11.132 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 13:28:10 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-alertmanager-a[77537]: ts=2024-03-21T13:28:10.833Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-03-21T13:28:11.132 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 13:28:10 smithi097 podman[78698]: 2024-03-21 13:28:10.864481293 +0000 UTC m=+0.156272017 container died 365241232e2e6e065d28e871cd085d3ef04dce642601aab28c5fe2f041e3ef4d (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-21T13:28:11.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:11 smithi097 ceph-mon[30040]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:11.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:11 smithi097 ceph-mon[39416]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:11.554 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 13:28:11 smithi097 podman[78698]: 2024-03-21 13:28:11.315325085 +0000 UTC m=+0.607115851 container cleanup 365241232e2e6e065d28e871cd085d3ef04dce642601aab28c5fe2f041e3ef4d (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-21T13:28:11.554 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 13:28:11 smithi097 bash[78698]: ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-alertmanager-a 2024-03-21T13:28:11.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:11 smithi144 ceph-mon[33633]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:11.866 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 13:28:11 smithi097 podman[78717]: 2024-03-21 13:28:11.601288834 +0000 UTC m=+0.741257847 container remove 365241232e2e6e065d28e871cd085d3ef04dce642601aab28c5fe2f041e3ef4d (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-21T13:28:11.866 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 13:28:11 smithi097 podman[78717]: 2024-03-21 13:28:11.700617338 +0000 UTC m=+0.840586359 volume remove 577bc43a727b07df1264d94e41ee0195c1cf401e1bf8ebe7786a4a08e3d02f16 2024-03-21T13:28:12.226 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:28:11 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:28:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:28:12.227 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 13:28:11 smithi097 systemd[1]: ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@alertmanager.a.service: Deactivated successfully. 2024-03-21T13:28:12.227 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 13:28:11 smithi097 systemd[1]: Stopped Ceph alertmanager.a for a5dd1f32-e785-11ee-95cd-87774f69a715. 2024-03-21T13:28:12.227 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 13:28:11 smithi097 systemd[1]: Starting Ceph alertmanager.a for a5dd1f32-e785-11ee-95cd-87774f69a715... 2024-03-21T13:28:12.227 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 13:28:12 smithi097 podman[78818]: 2024-03-21 13:28:11.998076727 +0000 UTC m=+0.017585312 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-03-21T13:28:12.227 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 13:28:12 smithi097 podman[78818]: 2024-03-21 13:28:12.109939372 +0000 UTC m=+0.129447954 volume create c9e3d9c764aa2dcec73073d6fb6b5396b6c8829a571d2af65b31b2234cf35685 2024-03-21T13:28:12.227 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 13:28:12 smithi097 podman[78818]: 2024-03-21 13:28:12.227294289 +0000 UTC m=+0.246802871 container create e5a584b4baa96dfbaa4832d154f4578ecb535cf27e1a397735fef7cdfb2e573a (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-21T13:28:12.803 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 13:28:12 smithi097 podman[78818]: 2024-03-21 13:28:12.513485558 +0000 UTC m=+0.532994140 container init e5a584b4baa96dfbaa4832d154f4578ecb535cf27e1a397735fef7cdfb2e573a (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-21T13:28:12.803 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 13:28:12 smithi097 podman[78818]: 2024-03-21 13:28:12.518837386 +0000 UTC m=+0.538345968 container start e5a584b4baa96dfbaa4832d154f4578ecb535cf27e1a397735fef7cdfb2e573a (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-21T13:28:12.803 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 13:28:12 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-alertmanager-a[78832]: ts=2024-03-21T13:28:12.531Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-03-21T13:28:12.804 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 13:28:12 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-alertmanager-a[78832]: ts=2024-03-21T13:28:12.531Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-03-21T13:28:12.804 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 13:28:12 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-alertmanager-a[78832]: ts=2024-03-21T13:28:12.532Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.97 port=9094 2024-03-21T13:28:12.804 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 13:28:12 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-alertmanager-a[78832]: ts=2024-03-21T13:28:12.533Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-03-21T13:28:12.804 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 13:28:12 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-alertmanager-a[78832]: ts=2024-03-21T13:28:12.556Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-03-21T13:28:12.804 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 13:28:12 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-alertmanager-a[78832]: ts=2024-03-21T13:28:12.556Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-03-21T13:28:12.804 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 13:28:12 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-alertmanager-a[78832]: ts=2024-03-21T13:28:12.559Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-03-21T13:28:12.804 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 13:28:12 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-alertmanager-a[78832]: ts=2024-03-21T13:28:12.559Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-03-21T13:28:12.804 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 13:28:12 smithi097 bash[78818]: e5a584b4baa96dfbaa4832d154f4578ecb535cf27e1a397735fef7cdfb2e573a 2024-03-21T13:28:12.804 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 13:28:12 smithi097 systemd[1]: Started Ceph alertmanager.a for a5dd1f32-e785-11ee-95cd-87774f69a715. 2024-03-21T13:28:13.653 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:28:13 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:28:13.967 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:13 smithi144 ceph-mon[33633]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:13.967 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:13 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:13.967 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:13 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:13.967 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:13 smithi144 ceph-mon[33633]: Reconfiguring prometheus.a (dependencies changed)... 2024-03-21T13:28:13.968 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:13 smithi144 ceph-mon[33633]: Reconfiguring daemon prometheus.a on smithi144 2024-03-21T13:28:14.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:13 smithi097 ceph-mon[39416]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:14.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:13 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:14.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:13 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:14.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:13 smithi097 ceph-mon[39416]: Reconfiguring prometheus.a (dependencies changed)... 2024-03-21T13:28:14.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:13 smithi097 ceph-mon[39416]: Reconfiguring daemon prometheus.a on smithi144 2024-03-21T13:28:14.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:13 smithi097 ceph-mon[30040]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:14.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:13 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:14.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:13 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:14.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:13 smithi097 ceph-mon[30040]: Reconfiguring prometheus.a (dependencies changed)... 2024-03-21T13:28:14.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:13 smithi097 ceph-mon[30040]: Reconfiguring daemon prometheus.a on smithi144 2024-03-21T13:28:14.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:14 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:28:14.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:14 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:28:14.804 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 13:28:14 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-alertmanager-a[78832]: ts=2024-03-21T13:28:14.534Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000692551s 2024-03-21T13:28:14.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:14 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:28:16.002 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:15 smithi144 ceph-mon[33633]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:16.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:15 smithi097 ceph-mon[39416]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:16.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:15 smithi097 ceph-mon[30040]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:16.371 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:28:16 smithi144 systemd[1]: Stopping Ceph prometheus.a for a5dd1f32-e785-11ee-95cd-87774f69a715... 2024-03-21T13:28:16.700 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:28:16 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[64191]: ts=2024-03-21T13:28:16.411Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-03-21T13:28:16.700 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:28:16 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[64191]: ts=2024-03-21T13:28:16.411Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-03-21T13:28:16.700 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:28:16 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[64191]: ts=2024-03-21T13:28:16.411Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-03-21T13:28:16.700 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:28:16 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[64191]: ts=2024-03-21T13:28:16.411Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-03-21T13:28:16.700 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:28:16 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[64191]: ts=2024-03-21T13:28:16.411Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-03-21T13:28:16.700 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:28:16 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[64191]: ts=2024-03-21T13:28:16.411Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-03-21T13:28:16.700 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:28:16 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[64191]: ts=2024-03-21T13:28:16.411Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-03-21T13:28:16.700 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:28:16 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[64191]: ts=2024-03-21T13:28:16.411Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-03-21T13:28:16.700 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:28:16 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[64191]: ts=2024-03-21T13:28:16.411Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-03-21T13:28:16.701 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:28:16 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[64191]: ts=2024-03-21T13:28:16.411Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-03-21T13:28:16.701 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:28:16 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[64191]: ts=2024-03-21T13:28:16.411Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-03-21T13:28:16.701 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:28:16 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[64191]: ts=2024-03-21T13:28:16.411Z caller=main.go:1147 level=info msg="See you next time!" 2024-03-21T13:28:16.701 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:28:16 smithi144 podman[67659]: 2024-03-21 13:28:16.421553355 +0000 UTC m=+0.133591520 container died 6f097bbd054bd04ec33556acd6c318241eb390e8fbd68fe916de113dbe4f7935 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-21T13:28:17.015 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:28:16 smithi144 podman[67659]: 2024-03-21 13:28:16.790260117 +0000 UTC m=+0.502298279 container cleanup 6f097bbd054bd04ec33556acd6c318241eb390e8fbd68fe916de113dbe4f7935 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-21T13:28:17.016 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:28:16 smithi144 bash[67659]: ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a 2024-03-21T13:28:17.366 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:28:17 smithi144 podman[67678]: 2024-03-21 13:28:17.016473977 +0000 UTC m=+0.592043512 container remove 6f097bbd054bd04ec33556acd6c318241eb390e8fbd68fe916de113dbe4f7935 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-21T13:28:17.366 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:28:17 smithi144 systemd[1]: ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@prometheus.a.service: Deactivated successfully. 2024-03-21T13:28:17.367 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:28:17 smithi144 systemd[1]: Stopped Ceph prometheus.a for a5dd1f32-e785-11ee-95cd-87774f69a715. 2024-03-21T13:28:17.367 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:28:17 smithi144 systemd[1]: ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@prometheus.a.service: Consumed 1.174s CPU time. 2024-03-21T13:28:17.367 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:28:17 smithi144 systemd[1]: Starting Ceph prometheus.a for a5dd1f32-e785-11ee-95cd-87774f69a715... 2024-03-21T13:28:17.621 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:28:17 smithi144 podman[67780]: 2024-03-21 13:28:17.27089227 +0000 UTC m=+0.019592250 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-03-21T13:28:17.621 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:28:17 smithi144 podman[67780]: 2024-03-21 13:28:17.375981558 +0000 UTC m=+0.124681584 container create 1a9823cc5c75409c5d0b588f283632a600da23b051281af5aba6ca8c5998dd84 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-21T13:28:17.621 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:28:17 smithi144 podman[67780]: 2024-03-21 13:28:17.561454698 +0000 UTC m=+0.310154688 container init 1a9823cc5c75409c5d0b588f283632a600da23b051281af5aba6ca8c5998dd84 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-21T13:28:17.622 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:28:17 smithi144 podman[67780]: 2024-03-21 13:28:17.567227654 +0000 UTC m=+0.315927644 container start 1a9823cc5c75409c5d0b588f283632a600da23b051281af5aba6ca8c5998dd84 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-21T13:28:17.622 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:28:17 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[67795]: ts=2024-03-21T13:28:17.591Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-03-21T13:28:17.622 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:28:17 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[67795]: ts=2024-03-21T13:28:17.591Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-03-21T13:28:17.622 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:28:17 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[67795]: ts=2024-03-21T13:28:17.591Z caller=main.go:570 level=info host_details="(Linux 5.14.0-430.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024 x86_64 smithi144 (none))" 2024-03-21T13:28:17.622 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:28:17 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[67795]: ts=2024-03-21T13:28:17.591Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-03-21T13:28:17.622 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:28:17 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[67795]: ts=2024-03-21T13:28:17.591Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-03-21T13:28:17.622 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:28:17 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[67795]: ts=2024-03-21T13:28:17.592Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-03-21T13:28:17.622 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:28:17 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[67795]: ts=2024-03-21T13:28:17.593Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-03-21T13:28:17.622 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:28:17 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[67795]: ts=2024-03-21T13:28:17.595Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-03-21T13:28:17.622 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:28:17 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[67795]: ts=2024-03-21T13:28:17.595Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-03-21T13:28:17.622 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:28:17 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[67795]: ts=2024-03-21T13:28:17.596Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-03-21T13:28:17.622 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:28:17 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[67795]: ts=2024-03-21T13:28:17.596Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=2.093µs 2024-03-21T13:28:17.622 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:28:17 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[67795]: ts=2024-03-21T13:28:17.596Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-03-21T13:28:17.623 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:28:17 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[67795]: ts=2024-03-21T13:28:17.596Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-03-21T13:28:17.623 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:28:17 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[67795]: ts=2024-03-21T13:28:17.596Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-03-21T13:28:17.623 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:28:17 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[67795]: ts=2024-03-21T13:28:17.596Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=28.186µs wal_replay_duration=544.132µs wbl_replay_duration=116ns total_replay_duration=600.51µs 2024-03-21T13:28:17.623 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:28:17 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[67795]: ts=2024-03-21T13:28:17.597Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-03-21T13:28:17.623 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:28:17 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[67795]: ts=2024-03-21T13:28:17.597Z caller=main.go:1029 level=info msg="TSDB started" 2024-03-21T13:28:17.623 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:28:17 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[67795]: ts=2024-03-21T13:28:17.597Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-03-21T13:28:17.623 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:28:17 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[67795]: ts=2024-03-21T13:28:17.616Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=19.091026ms db_storage=769ns remote_storage=1.01µs web_handler=273ns query_engine=549ns scrape=394.716µs scrape_sd=84.878µs notify=10.191µs notify_sd=12.681µs rules=18.296343ms tracing=5.279µs 2024-03-21T13:28:17.623 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:28:17 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[67795]: ts=2024-03-21T13:28:17.616Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-03-21T13:28:17.623 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:28:17 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[67795]: ts=2024-03-21T13:28:17.616Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-03-21T13:28:18.004 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:17 smithi097 ceph-mon[39416]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:18.004 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:17 smithi097 ceph-mon[30040]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:18.004 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:28:17 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: [21/Mar/2024:13:28:17] ENGINE Bus STOPPING 2024-03-21T13:28:18.004 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:28:17 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: [21/Mar/2024:13:28:17] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-03-21T13:28:18.004 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:28:17 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: [21/Mar/2024:13:28:17] ENGINE Bus STOPPED 2024-03-21T13:28:18.004 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:28:17 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: [21/Mar/2024:13:28:17] ENGINE Bus STARTING 2024-03-21T13:28:18.004 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:28:17 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: [21/Mar/2024:13:28:17] ENGINE Serving on http://:::9283 2024-03-21T13:28:18.005 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:28:17 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: [21/Mar/2024:13:28:17] ENGINE Bus STARTED 2024-03-21T13:28:18.005 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:28:17 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: [21/Mar/2024:13:28:17] ENGINE Bus STOPPING 2024-03-21T13:28:18.005 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:28:17 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: [21/Mar/2024:13:28:17] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-03-21T13:28:18.005 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:28:17 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: [21/Mar/2024:13:28:17] ENGINE Bus STOPPED 2024-03-21T13:28:18.005 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:28:17 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: [21/Mar/2024:13:28:17] ENGINE Bus STARTING 2024-03-21T13:28:18.005 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:28:18 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: [21/Mar/2024:13:28:18] ENGINE Serving on http://:::9283 2024-03-21T13:28:18.005 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:28:18 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: [21/Mar/2024:13:28:18] ENGINE Bus STARTED 2024-03-21T13:28:18.005 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:28:18 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: [21/Mar/2024:13:28:18] ENGINE Bus STOPPING 2024-03-21T13:28:18.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:17 smithi144 ceph-mon[33633]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:18.121 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:28:17 smithi144 bash[67780]: 1a9823cc5c75409c5d0b588f283632a600da23b051281af5aba6ca8c5998dd84 2024-03-21T13:28:18.121 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 13:28:17 smithi144 systemd[1]: Started Ceph prometheus.a for a5dd1f32-e785-11ee-95cd-87774f69a715. 2024-03-21T13:28:18.267 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:28:18 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: [21/Mar/2024:13:28:18] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-03-21T13:28:18.267 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:28:18 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: [21/Mar/2024:13:28:18] ENGINE Bus STOPPED 2024-03-21T13:28:18.267 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:28:18 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: [21/Mar/2024:13:28:18] ENGINE Bus STARTING 2024-03-21T13:28:18.267 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:28:18 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: [21/Mar/2024:13:28:18] ENGINE Serving on http://:::9283 2024-03-21T13:28:18.267 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:28:18 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: [21/Mar/2024:13:28:18] ENGINE Bus STARTED 2024-03-21T13:28:19.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:18 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:19.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:18 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:19.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:18 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-21T13:28:19.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:18 smithi097 ceph-mon[30040]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-21T13:28:19.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:18 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi097.front.sepia.ceph.com:9093"}]: dispatch 2024-03-21T13:28:19.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:18 smithi097 ceph-mon[30040]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi097.front.sepia.ceph.com:9093"}]: dispatch 2024-03-21T13:28:19.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:18 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:19.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:18 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-21T13:28:19.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:18 smithi097 ceph-mon[30040]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-21T13:28:19.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:18 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi144.front.sepia.ceph.com:9095"}]: dispatch 2024-03-21T13:28:19.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:18 smithi097 ceph-mon[30040]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi144.front.sepia.ceph.com:9095"}]: dispatch 2024-03-21T13:28:19.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:18 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:19.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:18 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-21T13:28:19.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:18 smithi097 ceph-mon[30040]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-21T13:28:19.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:18 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi144.front.sepia.ceph.com:3000"}]: dispatch 2024-03-21T13:28:19.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:18 smithi097 ceph-mon[30040]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi144.front.sepia.ceph.com:3000"}]: dispatch 2024-03-21T13:28:19.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:18 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:19.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:18 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:28:19.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:18 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:19.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:18 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:19.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:18 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-21T13:28:19.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:18 smithi097 ceph-mon[39416]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-21T13:28:19.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:18 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi097.front.sepia.ceph.com:9093"}]: dispatch 2024-03-21T13:28:19.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:18 smithi097 ceph-mon[39416]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi097.front.sepia.ceph.com:9093"}]: dispatch 2024-03-21T13:28:19.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:18 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:19.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:18 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-21T13:28:19.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:18 smithi097 ceph-mon[39416]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-21T13:28:19.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:18 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi144.front.sepia.ceph.com:9095"}]: dispatch 2024-03-21T13:28:19.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:18 smithi097 ceph-mon[39416]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi144.front.sepia.ceph.com:9095"}]: dispatch 2024-03-21T13:28:19.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:18 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:19.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:18 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-21T13:28:19.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:18 smithi097 ceph-mon[39416]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-21T13:28:19.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:18 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi144.front.sepia.ceph.com:3000"}]: dispatch 2024-03-21T13:28:19.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:18 smithi097 ceph-mon[39416]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi144.front.sepia.ceph.com:3000"}]: dispatch 2024-03-21T13:28:19.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:18 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:19.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:18 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:28:19.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:18 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:19.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:18 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:19.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:18 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-21T13:28:19.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:18 smithi144 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-21T13:28:19.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:18 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi097.front.sepia.ceph.com:9093"}]: dispatch 2024-03-21T13:28:19.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:18 smithi144 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi097.front.sepia.ceph.com:9093"}]: dispatch 2024-03-21T13:28:19.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:18 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:19.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:18 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-21T13:28:19.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:18 smithi144 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-21T13:28:19.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:18 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi144.front.sepia.ceph.com:9095"}]: dispatch 2024-03-21T13:28:19.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:18 smithi144 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi144.front.sepia.ceph.com:9095"}]: dispatch 2024-03-21T13:28:19.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:18 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:19.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:18 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-21T13:28:19.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:18 smithi144 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-21T13:28:19.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:18 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi144.front.sepia.ceph.com:3000"}]: dispatch 2024-03-21T13:28:19.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:18 smithi144 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi144.front.sepia.ceph.com:3000"}]: dispatch 2024-03-21T13:28:19.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:18 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:19.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:18 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:28:20.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:19 smithi097 ceph-mon[39416]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:20.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:19 smithi097 ceph-mon[30040]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:20.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:19 smithi144 ceph-mon[33633]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:22.374 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:22 smithi144 ceph-mon[33633]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:22.374 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:22 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:22.374 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:22 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:22.536 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:22 smithi097 ceph-mon[30040]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:22.536 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:22 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:22.536 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:22 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:22.536 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:22 smithi097 ceph-mon[39416]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:22.536 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:22 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:22.536 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:22 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:22.803 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 13:28:22 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-alertmanager-a[78832]: ts=2024-03-21T13:28:22.536Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.002740406s 2024-03-21T13:28:23.445 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:23 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:28:23.445 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:28:23 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:28:23.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:23 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:28:23.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:23 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:28:24.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:24 smithi144 ceph-mon[33633]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:24.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:24 smithi097 ceph-mon[39416]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:24.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:24 smithi097 ceph-mon[30040]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:25.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:25 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:28:25.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:25 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:25.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:25 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:25.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:25 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:28:25.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:25 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:28:25.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:25 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:25.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:25 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:28:25.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:25 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:25.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:25 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:25.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:25 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:28:25.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:25 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:28:25.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:25 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:25.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:25 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:28:25.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:25 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:25.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:25 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:25.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:25 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:28:25.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:25 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:28:25.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:25 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:28:26.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:26 smithi097 ceph-mon[39416]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:26.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:26 smithi097 ceph-mon[30040]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:26.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:26 smithi144 ceph-mon[33633]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:27.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:27 smithi097 ceph-mon[30040]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:27.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:27 smithi097 ceph-mon[39416]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:27.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:27 smithi144 ceph-mon[33633]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:29.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:29 smithi144 ceph-mon[33633]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:29.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:29 smithi097 ceph-mon[39416]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:29.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:29 smithi097 ceph-mon[30040]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:31.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:31 smithi144 ceph-mon[33633]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:31.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:31 smithi097 ceph-mon[39416]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:31.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:31 smithi097 ceph-mon[30040]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:32.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:28:31 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:28:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:28:33.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:33 smithi144 ceph-mon[33633]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:33.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:28:33 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:28:33.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:33 smithi097 ceph-mon[39416]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:33.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:33 smithi097 ceph-mon[30040]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:34.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:34 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:28:34.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:34 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:28:34.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:34 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:28:35.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:35 smithi144 ceph-mon[33633]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:35.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:35 smithi097 ceph-mon[39416]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:35.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:35 smithi097 ceph-mon[30040]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:38.505 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:38 smithi097 ceph-mon[39416]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:38.505 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:38 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:28:38.506 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:38 smithi097 ceph-mon[30040]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:38.506 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:38 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:28:38.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:38 smithi144 ceph-mon[33633]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:38.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:38 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:28:40.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:40 smithi097 ceph-mon[39416]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:40.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:40 smithi097 ceph-mon[30040]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:40.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:40 smithi144 ceph-mon[33633]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:42.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:42 smithi097 ceph-mon[39416]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:42.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:42 smithi097 ceph-mon[30040]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:42.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:28:41 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:28:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:28:42.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:42 smithi144 ceph-mon[33633]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:43.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:43 smithi097 ceph-mon[39416]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:43.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:43 smithi097 ceph-mon[30040]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:43.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:43 smithi144 ceph-mon[33633]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:43.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:28:43 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:28:44.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:44 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:28:44.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:44 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:28:44.620 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:44 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:28:45.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:45 smithi097 ceph-mon[39416]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:45.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:45 smithi097 ceph-mon[30040]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:45.620 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:45 smithi144 ceph-mon[33633]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:47.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:47 smithi144 ceph-mon[33633]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:47.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:47 smithi097 ceph-mon[39416]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:47.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:47 smithi097 ceph-mon[30040]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:49.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:49 smithi144 ceph-mon[33633]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:49.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:49 smithi097 ceph-mon[39416]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:49.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:49 smithi097 ceph-mon[30040]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:51.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:51 smithi144 ceph-mon[33633]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:51.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:51 smithi097 ceph-mon[39416]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:51.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:51 smithi097 ceph-mon[30040]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:52.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:28:51 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:28:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:28:53.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:53 smithi144 ceph-mon[33633]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:53.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:53 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:28:53.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:28:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:28:53.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:53 smithi097 ceph-mon[39416]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:53.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:53 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:28:53.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:53 smithi097 ceph-mon[30040]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:53.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:53 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:28:54.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:54 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:28:54.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:54 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:28:54.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:54 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:28:55.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:55 smithi097 ceph-mon[39416]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:55.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:55 smithi097 ceph-mon[30040]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:55.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:55 smithi144 ceph-mon[33633]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:57.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:57 smithi097 ceph-mon[39416]: pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:57.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:57 smithi097 ceph-mon[30040]: pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:57.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:57 smithi144 ceph-mon[33633]: pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:58.299 INFO:tasks.workunit.client.0.smithi097.stderr:Note: switching to '960fbbe4645c2becc8084669be194c95183c7753'. 2024-03-21T13:28:58.299 INFO:tasks.workunit.client.0.smithi097.stderr: 2024-03-21T13:28:58.299 INFO:tasks.workunit.client.0.smithi097.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-21T13:28:58.299 INFO:tasks.workunit.client.0.smithi097.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-21T13:28:58.299 INFO:tasks.workunit.client.0.smithi097.stderr:state without impacting any branches by switching back to a branch. 2024-03-21T13:28:58.299 INFO:tasks.workunit.client.0.smithi097.stderr: 2024-03-21T13:28:58.299 INFO:tasks.workunit.client.0.smithi097.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-21T13:28:58.300 INFO:tasks.workunit.client.0.smithi097.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-21T13:28:58.300 INFO:tasks.workunit.client.0.smithi097.stderr: 2024-03-21T13:28:58.300 INFO:tasks.workunit.client.0.smithi097.stderr: git switch -c 2024-03-21T13:28:58.300 INFO:tasks.workunit.client.0.smithi097.stderr: 2024-03-21T13:28:58.300 INFO:tasks.workunit.client.0.smithi097.stderr:Or undo this operation with: 2024-03-21T13:28:58.300 INFO:tasks.workunit.client.0.smithi097.stderr: 2024-03-21T13:28:58.300 INFO:tasks.workunit.client.0.smithi097.stderr: git switch - 2024-03-21T13:28:58.300 INFO:tasks.workunit.client.0.smithi097.stderr: 2024-03-21T13:28:58.300 INFO:tasks.workunit.client.0.smithi097.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-21T13:28:58.300 INFO:tasks.workunit.client.0.smithi097.stderr: 2024-03-21T13:28:58.300 INFO:tasks.workunit.client.0.smithi097.stderr:HEAD is now at 960fbbe4645 Merge branch 'cv-refactor-osd-objectstore' of https://github.com/guits/ceph into wip-guits-testing-2024-03-21-0818 2024-03-21T13:28:58.304 DEBUG:teuthology.orchestra.run.smithi097:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-21T13:28:58.331 INFO:tasks.workunit.client.0.smithi097.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-21T13:28:58.333 INFO:tasks.workunit.client.0.smithi097.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-21T13:28:58.334 INFO:tasks.workunit.client.0.smithi097.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-21T13:28:59.110 INFO:tasks.workunit.client.0.smithi097.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-21T13:28:59.184 INFO:tasks.workunit.client.0.smithi097.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-21T13:28:59.227 INFO:tasks.workunit.client.0.smithi097.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-21T13:28:59.229 INFO:tasks.workunit.client.0.smithi097.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-21T13:28:59.229 INFO:tasks.workunit.client.0.smithi097.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-21T13:28:59.269 INFO:tasks.workunit.client.0.smithi097.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-21T13:28:59.272 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-21T13:28:59.272 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-21T13:28:59.327 INFO:tasks.workunit:Running workunits matching rados/test.sh on client.0... 2024-03-21T13:28:59.328 INFO:tasks.workunit:Running workunit rados/test.sh... 2024-03-21T13:28:59.328 DEBUG:teuthology.orchestra.run.smithi097:workunit test rados/test.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=960fbbe4645c2becc8084669be194c95183c7753 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh 2024-03-21T13:28:59.393 INFO:tasks.workunit.client.0.smithi097.stderr:+ parallel=1 2024-03-21T13:28:59.394 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' '' = --serial ']' 2024-03-21T13:28:59.394 INFO:tasks.workunit.client.0.smithi097.stderr:+ crimson=0 2024-03-21T13:28:59.394 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' '' = --crimson ']' 2024-03-21T13:28:59.394 INFO:tasks.workunit.client.0.smithi097.stderr:+ color= 2024-03-21T13:28:59.394 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -t 1 ']' 2024-03-21T13:28:59.394 INFO:tasks.workunit.client.0.smithi097.stderr:+ trap cleanup EXIT ERR HUP INT QUIT 2024-03-21T13:28:59.394 INFO:tasks.workunit.client.0.smithi097.stderr:+ GTEST_OUTPUT_DIR=/home/ubuntu/cephtest/archive/unit_test_xml_report 2024-03-21T13:28:59.394 INFO:tasks.workunit.client.0.smithi097.stderr:+ mkdir -p /home/ubuntu/cephtest/archive/unit_test_xml_report 2024-03-21T13:28:59.395 INFO:tasks.workunit.client.0.smithi097.stderr:+ declare -A pids 2024-03-21T13:28:59.395 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T13:28:59.395 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T13:28:59.395 INFO:tasks.workunit.client.0.smithi097.stderr:++ printf %25s api_aio 2024-03-21T13:28:59.395 INFO:tasks.workunit.client.0.smithi097.stderr:+ r=' api_aio' 2024-03-21T13:28:59.395 INFO:tasks.workunit.client.0.smithi097.stderr:++ echo api_aio 2024-03-21T13:28:59.395 INFO:tasks.workunit.client.0.smithi097.stderr:++ awk '{print $1}' 2024-03-21T13:28:59.396 INFO:tasks.workunit.client.0.smithi097.stderr:+ ff=api_aio 2024-03-21T13:28:59.396 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=79551 2024-03-21T13:28:59.396 INFO:tasks.workunit.client.0.smithi097.stderr:+ echo 'test api_aio on pid 79551' 2024-03-21T13:28:59.397 INFO:tasks.workunit.client.0.smithi097.stdout:test api_aio on pid 79551 2024-03-21T13:28:59.399 INFO:tasks.workunit.client.0.smithi097.stderr:+ pids[$f]=79551 2024-03-21T13:28:59.399 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T13:28:59.399 INFO:tasks.workunit.client.0.smithi097.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio.xml 2>&1 | tee ceph_test_rados_api_aio.log | sed "s/^/ api_aio: /"' 2024-03-21T13:28:59.399 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T13:28:59.399 INFO:tasks.workunit.client.0.smithi097.stderr:++ printf %25s api_aio_pp 2024-03-21T13:28:59.399 INFO:tasks.workunit.client.0.smithi097.stderr:+ r=' api_aio_pp' 2024-03-21T13:28:59.399 INFO:tasks.workunit.client.0.smithi097.stderr:++ echo api_aio_pp 2024-03-21T13:28:59.400 INFO:tasks.workunit.client.0.smithi097.stderr:++ awk '{print $1}' 2024-03-21T13:28:59.400 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T13:28:59.400 INFO:tasks.workunit.client.0.smithi097.stderr:+ . /etc/bashrc 2024-03-21T13:28:59.400 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -z '' ']' 2024-03-21T13:28:59.400 INFO:tasks.workunit.client.0.smithi097.stderr:++ BASHRCSOURCED=Y 2024-03-21T13:28:59.400 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.400 INFO:tasks.workunit.client.0.smithi097.stderr:++ shopt -q login_shell 2024-03-21T13:28:59.400 INFO:tasks.workunit.client.0.smithi097.stderr:+++ umask 2024-03-21T13:28:59.400 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T13:28:59.400 INFO:tasks.workunit.client.0.smithi097.stderr:++ SHELL=/bin/bash 2024-03-21T13:28:59.400 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.400 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T13:28:59.400 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.400 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T13:28:59.401 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T13:28:59.401 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.401 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T13:28:59.401 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.401 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T13:28:59.401 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.401 INFO:tasks.workunit.client.0.smithi097.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.401 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.401 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T13:28:59.401 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.401 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T13:28:59.401 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.401 INFO:tasks.workunit.client.0.smithi097.stderr:+ ff=api_aio_pp 2024-03-21T13:28:59.401 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=79558 2024-03-21T13:28:59.401 INFO:tasks.workunit.client.0.smithi097.stderr:+ echo 'test api_aio_pp on pid 79558' 2024-03-21T13:28:59.402 INFO:tasks.workunit.client.0.smithi097.stdout:test api_aio_pp on pid 79558 2024-03-21T13:28:59.405 INFO:tasks.workunit.client.0.smithi097.stderr:+ pids[$f]=79558 2024-03-21T13:28:59.406 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T13:28:59.406 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T13:28:59.406 INFO:tasks.workunit.client.0.smithi097.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio_pp.xml 2>&1 | tee ceph_test_rados_api_aio_pp.log | sed "s/^/ api_aio_pp: /"' 2024-03-21T13:28:59.406 INFO:tasks.workunit.client.0.smithi097.stderr:++ printf %25s api_io 2024-03-21T13:28:59.406 INFO:tasks.workunit.client.0.smithi097.stderr:+ r=' api_io' 2024-03-21T13:28:59.406 INFO:tasks.workunit.client.0.smithi097.stderr:++ echo api_io 2024-03-21T13:28:59.406 INFO:tasks.workunit.client.0.smithi097.stderr:++ awk '{print $1}' 2024-03-21T13:28:59.406 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T13:28:59.406 INFO:tasks.workunit.client.0.smithi097.stderr:+ . /etc/bashrc 2024-03-21T13:28:59.406 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -z '' ']' 2024-03-21T13:28:59.406 INFO:tasks.workunit.client.0.smithi097.stderr:++ BASHRCSOURCED=Y 2024-03-21T13:28:59.406 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.407 INFO:tasks.workunit.client.0.smithi097.stderr:++ shopt -q login_shell 2024-03-21T13:28:59.407 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T13:28:59.407 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T13:28:59.407 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T13:28:59.407 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.407 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T13:28:59.407 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.407 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T13:28:59.407 INFO:tasks.workunit.client.0.smithi097.stderr:+++ umask 2024-03-21T13:28:59.407 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T13:28:59.407 INFO:tasks.workunit.client.0.smithi097.stderr:+++ return 2024-03-21T13:28:59.407 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.407 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T13:28:59.407 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.407 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T13:28:59.407 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export S_COLORS=auto 2024-03-21T13:28:59.408 INFO:tasks.workunit.client.0.smithi097.stderr:+++ S_COLORS=auto 2024-03-21T13:28:59.408 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.408 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T13:28:59.408 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.408 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T13:28:59.408 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T13:28:59.408 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.408 INFO:tasks.workunit.client.0.smithi097.stderr:++ SHELL=/bin/bash 2024-03-21T13:28:59.408 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.408 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T13:28:59.408 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.408 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T13:28:59.408 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T13:28:59.408 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.408 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T13:28:59.409 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.409 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T13:28:59.409 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.409 INFO:tasks.workunit.client.0.smithi097.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.409 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.409 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T13:28:59.409 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.409 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T13:28:59.409 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.409 INFO:tasks.workunit.client.0.smithi097.stderr:+ ff=api_io 2024-03-21T13:28:59.409 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=79567 2024-03-21T13:28:59.409 INFO:tasks.workunit.client.0.smithi097.stdout:test api_io on pid 79567 2024-03-21T13:28:59.414 INFO:tasks.workunit.client.0.smithi097.stderr:+ echo 'test api_io on pid 79567' 2024-03-21T13:28:59.414 INFO:tasks.workunit.client.0.smithi097.stderr:+ pids[$f]=79567 2024-03-21T13:28:59.414 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T13:28:59.414 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T13:28:59.414 INFO:tasks.workunit.client.0.smithi097.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io.xml 2>&1 | tee ceph_test_rados_api_io.log | sed "s/^/ api_io: /"' 2024-03-21T13:28:59.414 INFO:tasks.workunit.client.0.smithi097.stderr:++ printf %25s api_io_pp 2024-03-21T13:28:59.415 INFO:tasks.workunit.client.0.smithi097.stderr:+ r=' api_io_pp' 2024-03-21T13:28:59.415 INFO:tasks.workunit.client.0.smithi097.stderr:++ echo api_io_pp 2024-03-21T13:28:59.415 INFO:tasks.workunit.client.0.smithi097.stderr:++ awk '{print $1}' 2024-03-21T13:28:59.415 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T13:28:59.415 INFO:tasks.workunit.client.0.smithi097.stderr:+ . /etc/bashrc 2024-03-21T13:28:59.415 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -z '' ']' 2024-03-21T13:28:59.415 INFO:tasks.workunit.client.0.smithi097.stderr:++ BASHRCSOURCED=Y 2024-03-21T13:28:59.415 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.415 INFO:tasks.workunit.client.0.smithi097.stderr:++ shopt -q login_shell 2024-03-21T13:28:59.415 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T13:28:59.415 INFO:tasks.workunit.client.0.smithi097.stderr:+++ umask 2024-03-21T13:28:59.415 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T13:28:59.415 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T13:28:59.415 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.415 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T13:28:59.416 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.416 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T13:28:59.416 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T13:28:59.416 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.416 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T13:28:59.416 INFO:tasks.workunit.client.0.smithi097.stderr:++ SHELL=/bin/bash 2024-03-21T13:28:59.416 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T13:28:59.416 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T13:28:59.416 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.416 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T13:28:59.416 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T13:28:59.416 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.416 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T13:28:59.416 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.416 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T13:28:59.416 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.417 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T13:28:59.417 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T13:28:59.417 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T13:28:59.417 INFO:tasks.workunit.client.0.smithi097.stderr:+++ return 2024-03-21T13:28:59.417 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.417 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T13:28:59.417 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.417 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.417 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T13:28:59.417 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T13:28:59.417 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.417 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.417 INFO:tasks.workunit.client.0.smithi097.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.417 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T13:28:59.417 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.418 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T13:28:59.418 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.418 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T13:28:59.418 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export S_COLORS=auto 2024-03-21T13:28:59.418 INFO:tasks.workunit.client.0.smithi097.stderr:+++ S_COLORS=auto 2024-03-21T13:28:59.418 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.418 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.418 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T13:28:59.418 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.418 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T13:28:59.418 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.418 INFO:tasks.workunit.client.0.smithi097.stderr:+ ff=api_io_pp 2024-03-21T13:28:59.418 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=79579 2024-03-21T13:28:59.419 INFO:tasks.workunit.client.0.smithi097.stdout:test api_io_pp on pid 79579 2024-03-21T13:28:59.423 INFO:tasks.workunit.client.0.smithi097.stderr:+ echo 'test api_io_pp on pid 79579' 2024-03-21T13:28:59.424 INFO:tasks.workunit.client.0.smithi097.stderr:+ pids[$f]=79579 2024-03-21T13:28:59.424 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T13:28:59.424 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T13:28:59.424 INFO:tasks.workunit.client.0.smithi097.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io_pp.xml 2>&1 | tee ceph_test_rados_api_io_pp.log | sed "s/^/ api_io_pp: /"' 2024-03-21T13:28:59.424 INFO:tasks.workunit.client.0.smithi097.stderr:++ printf %25s api_asio 2024-03-21T13:28:59.424 INFO:tasks.workunit.client.0.smithi097.stderr:+ r=' api_asio' 2024-03-21T13:28:59.424 INFO:tasks.workunit.client.0.smithi097.stderr:++ echo api_asio 2024-03-21T13:28:59.424 INFO:tasks.workunit.client.0.smithi097.stderr:++ awk '{print $1}' 2024-03-21T13:28:59.424 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T13:28:59.424 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T13:28:59.424 INFO:tasks.workunit.client.0.smithi097.stderr:+ . /etc/bashrc 2024-03-21T13:28:59.424 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T13:28:59.424 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T13:28:59.424 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.425 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T13:28:59.425 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.425 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T13:28:59.425 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T13:28:59.425 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T13:28:59.425 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T13:28:59.425 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T13:28:59.425 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -z '' ']' 2024-03-21T13:28:59.425 INFO:tasks.workunit.client.0.smithi097.stderr:++ BASHRCSOURCED=Y 2024-03-21T13:28:59.425 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.425 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.425 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T13:28:59.425 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.425 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T13:28:59.426 INFO:tasks.workunit.client.0.smithi097.stderr:++ shopt -q login_shell 2024-03-21T13:28:59.426 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.426 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T13:28:59.426 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.426 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T13:28:59.426 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T13:28:59.426 INFO:tasks.workunit.client.0.smithi097.stderr:+++ return 2024-03-21T13:28:59.426 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.426 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T13:28:59.426 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.426 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T13:28:59.426 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T13:28:59.426 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.427 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T13:28:59.427 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T13:28:59.427 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export S_COLORS=auto 2024-03-21T13:28:59.427 INFO:tasks.workunit.client.0.smithi097.stderr:+++ S_COLORS=auto 2024-03-21T13:28:59.427 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.427 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T13:28:59.427 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T13:28:59.427 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.427 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T13:28:59.427 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.427 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.427 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T13:28:59.427 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T13:28:59.427 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.427 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T13:28:59.428 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup 2024-03-21T13:28:59.428 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T13:28:59.428 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.428 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.428 INFO:tasks.workunit.client.0.smithi097.stderr:+++ umask 2024-03-21T13:28:59.428 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T13:28:59.428 INFO:tasks.workunit.client.0.smithi097.stderr:++ SHELL=/bin/bash 2024-03-21T13:28:59.428 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/locale 2024-03-21T13:28:59.428 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.428 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T13:28:59.428 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.428 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T13:28:59.428 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T13:28:59.429 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.429 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T13:28:59.429 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.429 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T13:28:59.429 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.429 INFO:tasks.workunit.client.0.smithi097.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.429 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.429 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T13:28:59.429 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.429 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T13:28:59.429 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.429 INFO:tasks.workunit.client.0.smithi097.stderr:+ ff=api_asio 2024-03-21T13:28:59.429 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.429 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T13:28:59.429 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.430 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T13:28:59.430 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T13:28:59.430 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=79594 2024-03-21T13:28:59.430 INFO:tasks.workunit.client.0.smithi097.stderr:+ echo 'test api_asio on pid 79594' 2024-03-21T13:28:59.430 INFO:tasks.workunit.client.0.smithi097.stdout:test api_asio on pid 79594 2024-03-21T13:28:59.434 INFO:tasks.workunit.client.0.smithi097.stderr:+ pids[$f]=79594 2024-03-21T13:28:59.434 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T13:28:59.434 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T13:28:59.435 INFO:tasks.workunit.client.0.smithi097.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_asio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_asio.xml 2>&1 | tee ceph_test_rados_api_asio.log | sed "s/^/ api_asio: /"' 2024-03-21T13:28:59.435 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T13:28:59.435 INFO:tasks.workunit.client.0.smithi097.stderr:++ printf %25s api_list 2024-03-21T13:28:59.435 INFO:tasks.workunit.client.0.smithi097.stderr:+ r=' api_list' 2024-03-21T13:28:59.435 INFO:tasks.workunit.client.0.smithi097.stderr:++ echo api_list 2024-03-21T13:28:59.435 INFO:tasks.workunit.client.0.smithi097.stderr:++ awk '{print $1}' 2024-03-21T13:28:59.435 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T13:28:59.435 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T13:28:59.435 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T13:28:59.435 INFO:tasks.workunit.client.0.smithi097.stderr:+ . /etc/bashrc 2024-03-21T13:28:59.435 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T13:28:59.435 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T13:28:59.436 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.436 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T13:28:59.436 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T13:28:59.436 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.436 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T13:28:59.436 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T13:28:59.436 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.436 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T13:28:59.436 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.436 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T13:28:59.436 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T13:28:59.436 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.436 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T13:28:59.437 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -z '' ']' 2024-03-21T13:28:59.437 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T13:28:59.437 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.437 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T13:28:59.437 INFO:tasks.workunit.client.0.smithi097.stderr:++ BASHRCSOURCED=Y 2024-03-21T13:28:59.437 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.437 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.437 INFO:tasks.workunit.client.0.smithi097.stderr:++ shopt -q login_shell 2024-03-21T13:28:59.437 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T13:28:59.437 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.437 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T13:28:59.437 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.438 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T13:28:59.438 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T13:28:59.438 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup 2024-03-21T13:28:59.438 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T13:28:59.438 INFO:tasks.workunit.client.0.smithi097.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T13:28:59.438 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.438 INFO:tasks.workunit.client.0.smithi097.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T13:28:59.438 INFO:tasks.workunit.client.0.smithi097.stderr:++++ LANG=en_US.UTF-8 2024-03-21T13:28:59.438 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T13:28:59.438 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.438 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T13:28:59.438 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.438 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T13:28:59.438 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.439 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG=en_US.UTF-8 2024-03-21T13:28:59.439 INFO:tasks.workunit.client.0.smithi097.stderr:+++ umask 2024-03-21T13:28:59.439 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup config 2024-03-21T13:28:59.439 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T13:28:59.439 INFO:tasks.workunit.client.0.smithi097.stderr:+++ return 2024-03-21T13:28:59.439 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.439 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.439 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T13:28:59.439 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.439 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T13:28:59.439 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.439 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export S_COLORS=auto 2024-03-21T13:28:59.440 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' dumb = linux ']' 2024-03-21T13:28:59.440 INFO:tasks.workunit.client.0.smithi097.stderr:+++ S_COLORS=auto 2024-03-21T13:28:59.440 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/locale 2024-03-21T13:28:59.440 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.440 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.440 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T13:28:59.440 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T13:28:59.440 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.440 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.440 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/less.sh 2024-03-21T13:28:59.440 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T13:28:59.440 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.441 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T13:28:59.441 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T13:28:59.441 INFO:tasks.workunit.client.0.smithi097.stderr:++ SHELL=/bin/bash 2024-03-21T13:28:59.441 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.441 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T13:28:59.441 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.441 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T13:28:59.441 INFO:tasks.workunit.client.0.smithi097.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T13:28:59.441 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.441 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T13:28:59.441 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.441 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T13:28:59.441 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T13:28:59.441 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.441 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T13:28:59.442 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.442 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T13:28:59.442 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.442 INFO:tasks.workunit.client.0.smithi097.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.442 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.442 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T13:28:59.442 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.442 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T13:28:59.442 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.442 INFO:tasks.workunit.client.0.smithi097.stderr:+++++ readlink /proc/79551/exe 2024-03-21T13:28:59.442 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.442 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T13:28:59.442 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.442 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T13:28:59.442 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T13:28:59.443 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T13:28:59.443 INFO:tasks.workunit.client.0.smithi097.stderr:+ ff=api_list 2024-03-21T13:28:59.443 INFO:tasks.workunit.client.0.smithi097.stdout:test api_list on pid 79611 2024-03-21T13:28:59.448 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=79611 2024-03-21T13:28:59.448 INFO:tasks.workunit.client.0.smithi097.stderr:++++ basename /usr/bin/bash 2024-03-21T13:28:59.448 INFO:tasks.workunit.client.0.smithi097.stderr:+ echo 'test api_list on pid 79611' 2024-03-21T13:28:59.448 INFO:tasks.workunit.client.0.smithi097.stderr:+ pids[$f]=79611 2024-03-21T13:28:59.448 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T13:28:59.448 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T13:28:59.448 INFO:tasks.workunit.client.0.smithi097.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_list --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_list.xml 2>&1 | tee ceph_test_rados_api_list.log | sed "s/^/ api_list: /"' 2024-03-21T13:28:59.448 INFO:tasks.workunit.client.0.smithi097.stderr:++ printf %25s api_lock 2024-03-21T13:28:59.449 INFO:tasks.workunit.client.0.smithi097.stderr:+ r=' api_lock' 2024-03-21T13:28:59.449 INFO:tasks.workunit.client.0.smithi097.stderr:++ echo api_lock 2024-03-21T13:28:59.449 INFO:tasks.workunit.client.0.smithi097.stderr:++ awk '{print $1}' 2024-03-21T13:28:59.449 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_declare='declare -f' 2024-03-21T13:28:59.449 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_opt=-f 2024-03-21T13:28:59.449 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export which_declare 2024-03-21T13:28:59.449 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export -f which 2024-03-21T13:28:59.449 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset i 2024-03-21T13:28:59.449 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset -f pathmunge 2024-03-21T13:28:59.449 INFO:tasks.workunit.client.0.smithi097.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T13:28:59.449 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T13:28:59.449 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T13:28:59.449 INFO:tasks.workunit.client.0.smithi097.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T13:28:59.449 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T13:28:59.449 INFO:tasks.workunit.client.0.smithi097.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T13:28:59.450 INFO:tasks.workunit.client.0.smithi097.stderr:++++ LANG=en_US.UTF-8 2024-03-21T13:28:59.450 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.450 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T13:28:59.450 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.450 INFO:tasks.workunit.client.0.smithi097.stderr:+ export PATH 2024-03-21T13:28:59.450 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.450 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T13:28:59.450 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T13:28:59.450 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.450 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T13:28:59.450 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG=en_US.UTF-8 2024-03-21T13:28:59.450 INFO:tasks.workunit.client.0.smithi097.stderr:+ unset rc 2024-03-21T13:28:59.450 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup config 2024-03-21T13:28:59.450 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T13:28:59.450 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.451 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.451 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T13:28:59.451 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.451 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T13:28:59.451 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T13:28:59.451 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.451 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' dumb = linux ']' 2024-03-21T13:28:59.451 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T13:28:59.451 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.451 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T13:28:59.451 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.451 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/less.sh 2024-03-21T13:28:59.451 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T13:28:59.451 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.451 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T13:28:59.452 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T13:28:59.452 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.452 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.452 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T13:28:59.452 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.452 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T13:28:59.452 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T13:28:59.452 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.452 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T13:28:59.452 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.452 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T13:28:59.452 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T13:28:59.452 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T13:28:59.452 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.452 INFO:tasks.workunit.client.0.smithi097.stderr:+ . /etc/bashrc 2024-03-21T13:28:59.452 INFO:tasks.workunit.client.0.smithi097.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T13:28:59.453 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup 2024-03-21T13:28:59.453 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_api_aio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio.xml 2024-03-21T13:28:59.453 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -z '' ']' 2024-03-21T13:28:59.453 INFO:tasks.workunit.client.0.smithi097.stderr:+ tee ceph_test_rados_api_aio.log 2024-03-21T13:28:59.453 INFO:tasks.workunit.client.0.smithi097.stderr:++ BASHRCSOURCED=Y 2024-03-21T13:28:59.453 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.453 INFO:tasks.workunit.client.0.smithi097.stderr:+ sed 's/^/ api_aio: /' 2024-03-21T13:28:59.453 INFO:tasks.workunit.client.0.smithi097.stderr:++ shopt -q login_shell 2024-03-21T13:28:59.453 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/locale 2024-03-21T13:28:59.453 INFO:tasks.workunit.client.0.smithi097.stderr:+++++ readlink /proc/79558/exe 2024-03-21T13:28:59.453 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T13:28:59.453 INFO:tasks.workunit.client.0.smithi097.stderr:+++ umask 2024-03-21T13:28:59.453 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T13:28:59.453 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T13:28:59.453 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.454 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T13:28:59.454 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.454 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T13:28:59.454 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T13:28:59.454 INFO:tasks.workunit.client.0.smithi097.stderr:+++ return 2024-03-21T13:28:59.454 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.454 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T13:28:59.454 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.454 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T13:28:59.454 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T13:28:59.454 INFO:tasks.workunit.client.0.smithi097.stderr:++ SHELL=/bin/bash 2024-03-21T13:28:59.454 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export S_COLORS=auto 2024-03-21T13:28:59.454 INFO:tasks.workunit.client.0.smithi097.stderr:+++ S_COLORS=auto 2024-03-21T13:28:59.454 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.455 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T13:28:59.455 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.455 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T13:28:59.455 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.455 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T13:28:59.455 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.455 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.455 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T13:28:59.455 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T13:28:59.455 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.455 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T13:28:59.455 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.455 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T13:28:59.455 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.455 INFO:tasks.workunit.client.0.smithi097.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.455 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.456 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T13:28:59.456 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.456 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T13:28:59.456 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.456 INFO:tasks.workunit.client.0.smithi097.stderr:++++ basename /usr/bin/bash 2024-03-21T13:28:59.456 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.456 INFO:tasks.workunit.client.0.smithi097.stderr:+ ff=api_lock 2024-03-21T13:28:59.456 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T13:28:59.456 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.456 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T13:28:59.456 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T13:28:59.456 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=79629 2024-03-21T13:28:59.456 INFO:tasks.workunit.client.0.smithi097.stderr:+ echo 'test api_lock on pid 79629' 2024-03-21T13:28:59.457 INFO:tasks.workunit.client.0.smithi097.stdout:test api_lock on pid 79629 2024-03-21T13:28:59.462 INFO:tasks.workunit.client.0.smithi097.stderr:+ pids[$f]=79629 2024-03-21T13:28:59.462 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T13:28:59.462 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T13:28:59.462 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T13:28:59.462 INFO:tasks.workunit.client.0.smithi097.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock.xml 2>&1 | tee ceph_test_rados_api_lock.log | sed "s/^/ api_lock: /"' 2024-03-21T13:28:59.462 INFO:tasks.workunit.client.0.smithi097.stderr:++ printf %25s api_lock_pp 2024-03-21T13:28:59.462 INFO:tasks.workunit.client.0.smithi097.stderr:+ r=' api_lock_pp' 2024-03-21T13:28:59.462 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_declare='declare -f' 2024-03-21T13:28:59.462 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_opt=-f 2024-03-21T13:28:59.462 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export which_declare 2024-03-21T13:28:59.462 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export -f which 2024-03-21T13:28:59.462 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset i 2024-03-21T13:28:59.463 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset -f pathmunge 2024-03-21T13:28:59.463 INFO:tasks.workunit.client.0.smithi097.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T13:28:59.463 INFO:tasks.workunit.client.0.smithi097.stderr:+ export PATH 2024-03-21T13:28:59.463 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T13:28:59.463 INFO:tasks.workunit.client.0.smithi097.stderr:+ unset rc 2024-03-21T13:28:59.463 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_api_aio_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio_pp.xml 2024-03-21T13:28:59.463 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T13:28:59.463 INFO:tasks.workunit.client.0.smithi097.stderr:+ sed 's/^/ api_aio_pp: /' 2024-03-21T13:28:59.463 INFO:tasks.workunit.client.0.smithi097.stderr:+ . /etc/bashrc 2024-03-21T13:28:59.463 INFO:tasks.workunit.client.0.smithi097.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T13:28:59.463 INFO:tasks.workunit.client.0.smithi097.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T13:28:59.463 INFO:tasks.workunit.client.0.smithi097.stderr:++++ LANG=en_US.UTF-8 2024-03-21T13:28:59.463 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.463 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T13:28:59.463 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.464 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG=en_US.UTF-8 2024-03-21T13:28:59.464 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -z '' ']' 2024-03-21T13:28:59.464 INFO:tasks.workunit.client.0.smithi097.stderr:++ BASHRCSOURCED=Y 2024-03-21T13:28:59.464 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.464 INFO:tasks.workunit.client.0.smithi097.stderr:++ shopt -q login_shell 2024-03-21T13:28:59.464 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T13:28:59.464 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup config 2024-03-21T13:28:59.464 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T13:28:59.464 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.464 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T13:28:59.464 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.464 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T13:28:59.464 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.464 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T13:28:59.464 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.465 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' dumb = linux ']' 2024-03-21T13:28:59.465 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.465 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T13:28:59.465 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T13:28:59.465 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.465 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/less.sh 2024-03-21T13:28:59.465 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.465 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T13:28:59.465 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.465 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T13:28:59.465 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T13:28:59.465 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.465 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T13:28:59.465 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.466 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T13:28:59.466 INFO:tasks.workunit.client.0.smithi097.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T13:28:59.466 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.466 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T13:28:59.466 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.466 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T13:28:59.466 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup 2024-03-21T13:28:59.466 INFO:tasks.workunit.client.0.smithi097.stderr:+++ umask 2024-03-21T13:28:59.466 INFO:tasks.workunit.client.0.smithi097.stderr:++ awk '{print $1}' 2024-03-21T13:28:59.466 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T13:28:59.466 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T13:28:59.466 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T13:28:59.466 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T13:28:59.466 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T13:28:59.466 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T13:28:59.467 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.467 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T13:28:59.467 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.467 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.467 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T13:28:59.467 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T13:28:59.467 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.467 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T13:28:59.467 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T13:28:59.467 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T13:28:59.467 INFO:tasks.workunit.client.0.smithi097.stderr:++ SHELL=/bin/bash 2024-03-21T13:28:59.467 INFO:tasks.workunit.client.0.smithi097.stderr:+ tee ceph_test_rados_api_aio_pp.log 2024-03-21T13:28:59.467 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.467 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T13:28:59.467 INFO:tasks.workunit.client.0.smithi097.stderr:+++ return 2024-03-21T13:28:59.467 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.468 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T13:28:59.468 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.468 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T13:28:59.468 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.468 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T13:28:59.468 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.468 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T13:28:59.468 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export S_COLORS=auto 2024-03-21T13:28:59.468 INFO:tasks.workunit.client.0.smithi097.stderr:+++ S_COLORS=auto 2024-03-21T13:28:59.468 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.468 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T13:28:59.468 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.468 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T13:28:59.468 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.468 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T13:28:59.469 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.469 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T13:28:59.469 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.469 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T13:28:59.469 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.469 INFO:tasks.workunit.client.0.smithi097.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.469 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.469 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T13:28:59.469 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.469 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T13:28:59.469 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/locale 2024-03-21T13:28:59.469 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.469 INFO:tasks.workunit.client.0.smithi097.stderr:++ echo api_lock_pp 2024-03-21T13:28:59.469 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.469 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T13:28:59.470 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.470 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T13:28:59.470 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T13:28:59.470 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T13:28:59.470 INFO:tasks.workunit.client.0.smithi097.stderr:+ ff=api_lock_pp 2024-03-21T13:28:59.470 INFO:tasks.workunit.client.0.smithi097.stderr:+++++ readlink /proc/79567/exe 2024-03-21T13:28:59.470 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=79653 2024-03-21T13:28:59.470 INFO:tasks.workunit.client.0.smithi097.stdout:test api_lock_pp on pid 79653 2024-03-21T13:28:59.473 INFO:tasks.workunit.client.0.smithi097.stderr:+ echo 'test api_lock_pp on pid 79653' 2024-03-21T13:28:59.473 INFO:tasks.workunit.client.0.smithi097.stderr:+ pids[$f]=79653 2024-03-21T13:28:59.474 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T13:28:59.474 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T13:28:59.474 INFO:tasks.workunit.client.0.smithi097.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock_pp.xml 2>&1 | tee ceph_test_rados_api_lock_pp.log | sed "s/^/ api_lock_pp: /"' 2024-03-21T13:28:59.474 INFO:tasks.workunit.client.0.smithi097.stderr:++ printf %25s api_misc 2024-03-21T13:28:59.474 INFO:tasks.workunit.client.0.smithi097.stderr:+ r=' api_misc' 2024-03-21T13:28:59.474 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T13:28:59.474 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T13:28:59.474 INFO:tasks.workunit.client.0.smithi097.stderr:++++ basename /usr/bin/bash 2024-03-21T13:28:59.474 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T13:28:59.474 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.474 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T13:28:59.474 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.474 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T13:28:59.475 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T13:28:59.475 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.475 INFO:tasks.workunit.client.0.smithi097.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T13:28:59.475 INFO:tasks.workunit.client.0.smithi097.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T13:28:59.475 INFO:tasks.workunit.client.0.smithi097.stderr:++ echo api_misc 2024-03-21T13:28:59.475 INFO:tasks.workunit.client.0.smithi097.stderr:++++ LANG=en_US.UTF-8 2024-03-21T13:28:59.475 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.475 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T13:28:59.475 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.475 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG=en_US.UTF-8 2024-03-21T13:28:59.475 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup config 2024-03-21T13:28:59.475 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T13:28:59.475 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.475 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T13:28:59.475 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T13:28:59.476 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.476 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T13:28:59.476 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.476 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.476 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' dumb = linux ']' 2024-03-21T13:28:59.476 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T13:28:59.476 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T13:28:59.476 INFO:tasks.workunit.client.0.smithi097.stderr:++ awk '{print $1}' 2024-03-21T13:28:59.476 INFO:tasks.workunit.client.0.smithi097.stderr:+ . /etc/bashrc 2024-03-21T13:28:59.476 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.476 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T13:28:59.476 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.476 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/less.sh 2024-03-21T13:28:59.476 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T13:28:59.477 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.477 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T13:28:59.477 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.477 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T13:28:59.477 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T13:28:59.477 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.477 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T13:28:59.477 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.477 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T13:28:59.477 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -z '' ']' 2024-03-21T13:28:59.477 INFO:tasks.workunit.client.0.smithi097.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T13:28:59.477 INFO:tasks.workunit.client.0.smithi097.stderr:++ BASHRCSOURCED=Y 2024-03-21T13:28:59.477 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.477 INFO:tasks.workunit.client.0.smithi097.stderr:++ shopt -q login_shell 2024-03-21T13:28:59.477 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.478 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T13:28:59.478 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.478 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T13:28:59.478 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup 2024-03-21T13:28:59.478 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_declare='declare -f' 2024-03-21T13:28:59.478 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_opt=-f 2024-03-21T13:28:59.478 INFO:tasks.workunit.client.0.smithi097.stderr:+++ umask 2024-03-21T13:28:59.478 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export which_declare 2024-03-21T13:28:59.478 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export -f which 2024-03-21T13:28:59.478 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset i 2024-03-21T13:28:59.478 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset -f pathmunge 2024-03-21T13:28:59.478 INFO:tasks.workunit.client.0.smithi097.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T13:28:59.478 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/locale 2024-03-21T13:28:59.478 INFO:tasks.workunit.client.0.smithi097.stderr:+ export PATH 2024-03-21T13:28:59.478 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T13:28:59.479 INFO:tasks.workunit.client.0.smithi097.stderr:+ unset rc 2024-03-21T13:28:59.479 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T13:28:59.479 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T13:28:59.479 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T13:28:59.479 INFO:tasks.workunit.client.0.smithi097.stderr:++ SHELL=/bin/bash 2024-03-21T13:28:59.479 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T13:28:59.479 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.479 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T13:28:59.479 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.479 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T13:28:59.479 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.479 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T13:28:59.479 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.479 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T13:28:59.479 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T13:28:59.480 INFO:tasks.workunit.client.0.smithi097.stderr:+++ return 2024-03-21T13:28:59.480 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.480 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T13:28:59.480 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.480 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T13:28:59.480 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export S_COLORS=auto 2024-03-21T13:28:59.480 INFO:tasks.workunit.client.0.smithi097.stderr:+++ S_COLORS=auto 2024-03-21T13:28:59.480 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T13:28:59.480 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.480 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T13:28:59.480 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.480 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.480 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T13:28:59.480 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T13:28:59.480 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.481 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T13:28:59.481 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.481 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.481 INFO:tasks.workunit.client.0.smithi097.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.481 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.481 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T13:28:59.481 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.481 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T13:28:59.481 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.481 INFO:tasks.workunit.client.0.smithi097.stderr:+++++ readlink /proc/79579/exe 2024-03-21T13:28:59.481 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_api_io --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io.xml 2024-03-21T13:28:59.481 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.481 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T13:28:59.481 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.482 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T13:28:59.482 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T13:28:59.482 INFO:tasks.workunit.client.0.smithi097.stderr:++++ basename /usr/bin/bash 2024-03-21T13:28:59.482 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T13:28:59.482 INFO:tasks.workunit.client.0.smithi097.stderr:+ ff=api_misc 2024-03-21T13:28:59.482 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=79672 2024-03-21T13:28:59.482 INFO:tasks.workunit.client.0.smithi097.stdout:test api_misc on pid 79672 2024-03-21T13:28:59.485 INFO:tasks.workunit.client.0.smithi097.stderr:+ echo 'test api_misc on pid 79672' 2024-03-21T13:28:59.485 INFO:tasks.workunit.client.0.smithi097.stderr:+ pids[$f]=79672 2024-03-21T13:28:59.485 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T13:28:59.485 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T13:28:59.485 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_declare='declare -f' 2024-03-21T13:28:59.485 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_opt=-f 2024-03-21T13:28:59.485 INFO:tasks.workunit.client.0.smithi097.stderr:++ printf %25s api_misc_pp 2024-03-21T13:28:59.485 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export which_declare 2024-03-21T13:28:59.485 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T13:28:59.485 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export -f which 2024-03-21T13:28:59.485 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T13:28:59.485 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset i 2024-03-21T13:28:59.486 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset -f pathmunge 2024-03-21T13:28:59.486 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T13:28:59.486 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.486 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T13:28:59.486 INFO:tasks.workunit.client.0.smithi097.stderr:+ tee ceph_test_rados_api_io.log 2024-03-21T13:28:59.486 INFO:tasks.workunit.client.0.smithi097.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T13:28:59.486 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.486 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T13:28:59.486 INFO:tasks.workunit.client.0.smithi097.stderr:+ r=' api_misc_pp' 2024-03-21T13:28:59.486 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T13:28:59.486 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.486 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T13:28:59.486 INFO:tasks.workunit.client.0.smithi097.stderr:+ export PATH 2024-03-21T13:28:59.486 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.487 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T13:28:59.487 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T13:28:59.487 INFO:tasks.workunit.client.0.smithi097.stderr:+ unset rc 2024-03-21T13:28:59.487 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.487 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T13:28:59.487 INFO:tasks.workunit.client.0.smithi097.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc.xml 2>&1 | tee ceph_test_rados_api_misc.log | sed "s/^/ api_misc: /"' 2024-03-21T13:28:59.487 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.487 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T13:28:59.487 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup 2024-03-21T13:28:59.487 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_api_io_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io_pp.xml 2024-03-21T13:28:59.487 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/locale 2024-03-21T13:28:59.487 INFO:tasks.workunit.client.0.smithi097.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T13:28:59.487 INFO:tasks.workunit.client.0.smithi097.stderr:+ tee ceph_test_rados_api_io_pp.log 2024-03-21T13:28:59.487 INFO:tasks.workunit.client.0.smithi097.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T13:28:59.487 INFO:tasks.workunit.client.0.smithi097.stderr:++++ LANG=en_US.UTF-8 2024-03-21T13:28:59.487 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.488 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T13:28:59.488 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.488 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG=en_US.UTF-8 2024-03-21T13:28:59.488 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup config 2024-03-21T13:28:59.488 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.488 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.488 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' dumb = linux ']' 2024-03-21T13:28:59.488 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.488 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T13:28:59.488 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.488 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/less.sh 2024-03-21T13:28:59.488 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T13:28:59.488 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.488 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T13:28:59.489 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.489 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T13:28:59.489 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T13:28:59.489 INFO:tasks.workunit.client.0.smithi097.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T13:28:59.489 INFO:tasks.workunit.client.0.smithi097.stderr:+ . /etc/bashrc 2024-03-21T13:28:59.489 INFO:tasks.workunit.client.0.smithi097.stderr:+ sed 's/^/ api_io: /' 2024-03-21T13:28:59.489 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -z '' ']' 2024-03-21T13:28:59.489 INFO:tasks.workunit.client.0.smithi097.stderr:++ BASHRCSOURCED=Y 2024-03-21T13:28:59.489 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.489 INFO:tasks.workunit.client.0.smithi097.stderr:++ shopt -q login_shell 2024-03-21T13:28:59.489 INFO:tasks.workunit.client.0.smithi097.stderr:+++ umask 2024-03-21T13:28:59.489 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T13:28:59.489 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T13:28:59.489 INFO:tasks.workunit.client.0.smithi097.stderr:++ SHELL=/bin/bash 2024-03-21T13:28:59.489 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T13:28:59.490 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T13:28:59.490 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.490 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.490 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T13:28:59.490 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T13:28:59.490 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.490 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.490 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T13:28:59.490 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T13:28:59.490 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T13:28:59.490 INFO:tasks.workunit.client.0.smithi097.stderr:+++ return 2024-03-21T13:28:59.490 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.490 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T13:28:59.490 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.490 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T13:28:59.491 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T13:28:59.491 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.491 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export S_COLORS=auto 2024-03-21T13:28:59.491 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T13:28:59.491 INFO:tasks.workunit.client.0.smithi097.stderr:+++ S_COLORS=auto 2024-03-21T13:28:59.491 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.491 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.491 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T13:28:59.491 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T13:28:59.491 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.491 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T13:28:59.491 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.491 INFO:tasks.workunit.client.0.smithi097.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.491 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.492 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.492 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T13:28:59.492 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.492 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T13:28:59.492 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.492 INFO:tasks.workunit.client.0.smithi097.stderr:+++++ readlink /proc/79594/exe 2024-03-21T13:28:59.492 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.492 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T13:28:59.492 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.492 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T13:28:59.492 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T13:28:59.492 INFO:tasks.workunit.client.0.smithi097.stderr:++ awk '{print $1}' 2024-03-21T13:28:59.492 INFO:tasks.workunit.client.0.smithi097.stderr:++ echo api_misc_pp 2024-03-21T13:28:59.492 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T13:28:59.492 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T13:28:59.493 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T13:28:59.493 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.493 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T13:28:59.493 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.493 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T13:28:59.493 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T13:28:59.493 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.493 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T13:28:59.493 INFO:tasks.workunit.client.0.smithi097.stderr:++++ basename /usr/bin/bash 2024-03-21T13:28:59.493 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_declare='declare -f' 2024-03-21T13:28:59.493 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_opt=-f 2024-03-21T13:28:59.493 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export which_declare 2024-03-21T13:28:59.493 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export -f which 2024-03-21T13:28:59.493 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset i 2024-03-21T13:28:59.493 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset -f pathmunge 2024-03-21T13:28:59.494 INFO:tasks.workunit.client.0.smithi097.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T13:28:59.494 INFO:tasks.workunit.client.0.smithi097.stderr:+ ff=api_misc_pp 2024-03-21T13:28:59.494 INFO:tasks.workunit.client.0.smithi097.stderr:+ export PATH 2024-03-21T13:28:59.494 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T13:28:59.494 INFO:tasks.workunit.client.0.smithi097.stderr:+ unset rc 2024-03-21T13:28:59.494 INFO:tasks.workunit.client.0.smithi097.stdout:test api_misc_pp on pid 79691 2024-03-21T13:28:59.496 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=79691 2024-03-21T13:28:59.496 INFO:tasks.workunit.client.0.smithi097.stderr:+ echo 'test api_misc_pp on pid 79691' 2024-03-21T13:28:59.496 INFO:tasks.workunit.client.0.smithi097.stderr:+ pids[$f]=79691 2024-03-21T13:28:59.497 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T13:28:59.497 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T13:28:59.497 INFO:tasks.workunit.client.0.smithi097.stderr:+ sed 's/^/ api_io_pp: /' 2024-03-21T13:28:59.497 INFO:tasks.workunit.client.0.smithi097.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc_pp.xml 2>&1 | tee ceph_test_rados_api_misc_pp.log | sed "s/^/ api_misc_pp: /"' 2024-03-21T13:28:59.497 INFO:tasks.workunit.client.0.smithi097.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T13:28:59.497 INFO:tasks.workunit.client.0.smithi097.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T13:28:59.497 INFO:tasks.workunit.client.0.smithi097.stderr:++++ LANG=en_US.UTF-8 2024-03-21T13:28:59.497 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.497 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T13:28:59.497 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.497 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG=en_US.UTF-8 2024-03-21T13:28:59.497 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup config 2024-03-21T13:28:59.497 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.497 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.498 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' dumb = linux ']' 2024-03-21T13:28:59.498 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.498 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T13:28:59.498 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.498 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/less.sh 2024-03-21T13:28:59.498 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T13:28:59.498 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.498 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T13:28:59.498 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.498 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T13:28:59.498 INFO:tasks.workunit.client.0.smithi097.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T13:28:59.498 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_api_asio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_asio.xml 2024-03-21T13:28:59.498 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T13:28:59.498 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T13:28:59.498 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T13:28:59.499 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T13:28:59.499 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T13:28:59.499 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T13:28:59.499 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.499 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.499 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T13:28:59.499 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T13:28:59.499 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T13:28:59.499 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.499 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.499 INFO:tasks.workunit.client.0.smithi097.stderr:+ . /etc/bashrc 2024-03-21T13:28:59.499 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T13:28:59.499 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T13:28:59.499 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T13:28:59.499 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T13:28:59.500 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.500 INFO:tasks.workunit.client.0.smithi097.stderr:+++ return 2024-03-21T13:28:59.500 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.500 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T13:28:59.500 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.500 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T13:28:59.500 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export S_COLORS=auto 2024-03-21T13:28:59.500 INFO:tasks.workunit.client.0.smithi097.stderr:+++ S_COLORS=auto 2024-03-21T13:28:59.500 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.500 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T13:28:59.500 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -z '' ']' 2024-03-21T13:28:59.500 INFO:tasks.workunit.client.0.smithi097.stderr:++ BASHRCSOURCED=Y 2024-03-21T13:28:59.500 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.500 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T13:28:59.500 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.501 INFO:tasks.workunit.client.0.smithi097.stderr:++ shopt -q login_shell 2024-03-21T13:28:59.501 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.501 INFO:tasks.workunit.client.0.smithi097.stderr:+ sed 's/^/ api_asio: /' 2024-03-21T13:28:59.501 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T13:28:59.501 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T13:28:59.501 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T13:28:59.501 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.501 INFO:tasks.workunit.client.0.smithi097.stderr:++ printf %25s api_tier_pp 2024-03-21T13:28:59.501 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T13:28:59.501 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.501 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T13:28:59.501 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T13:28:59.501 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.501 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T13:28:59.502 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.502 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T13:28:59.502 INFO:tasks.workunit.client.0.smithi097.stderr:+++++ readlink /proc/79611/exe 2024-03-21T13:28:59.502 INFO:tasks.workunit.client.0.smithi097.stderr:+ r=' api_tier_pp' 2024-03-21T13:28:59.502 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.502 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T13:28:59.502 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.502 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T13:28:59.502 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup 2024-03-21T13:28:59.502 INFO:tasks.workunit.client.0.smithi097.stderr:+ tee ceph_test_rados_api_asio.log 2024-03-21T13:28:59.502 INFO:tasks.workunit.client.0.smithi097.stderr:++++ basename /usr/bin/bash 2024-03-21T13:28:59.502 INFO:tasks.workunit.client.0.smithi097.stderr:+++ umask 2024-03-21T13:28:59.502 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/locale 2024-03-21T13:28:59.502 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_declare='declare -f' 2024-03-21T13:28:59.502 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_opt=-f 2024-03-21T13:28:59.503 INFO:tasks.workunit.client.0.smithi097.stderr:++ echo api_tier_pp 2024-03-21T13:28:59.503 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export which_declare 2024-03-21T13:28:59.503 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export -f which 2024-03-21T13:28:59.503 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset i 2024-03-21T13:28:59.503 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset -f pathmunge 2024-03-21T13:28:59.503 INFO:tasks.workunit.client.0.smithi097.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T13:28:59.503 INFO:tasks.workunit.client.0.smithi097.stderr:+ export PATH 2024-03-21T13:28:59.503 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T13:28:59.503 INFO:tasks.workunit.client.0.smithi097.stderr:+ unset rc 2024-03-21T13:28:59.503 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T13:28:59.503 INFO:tasks.workunit.client.0.smithi097.stderr:++ SHELL=/bin/bash 2024-03-21T13:28:59.503 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.503 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T13:28:59.503 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.503 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T13:28:59.503 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T13:28:59.504 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T13:28:59.504 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T13:28:59.504 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T13:28:59.504 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.504 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.504 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T13:28:59.504 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T13:28:59.504 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.504 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.504 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T13:28:59.504 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T13:28:59.504 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.504 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T13:28:59.504 INFO:tasks.workunit.client.0.smithi097.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.504 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.505 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.505 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T13:28:59.505 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.505 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T13:28:59.505 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.505 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_api_list --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_list.xml 2024-03-21T13:28:59.505 INFO:tasks.workunit.client.0.smithi097.stderr:+ tee ceph_test_rados_api_list.log 2024-03-21T13:28:59.505 INFO:tasks.workunit.client.0.smithi097.stderr:+ sed 's/^/ api_list: /' 2024-03-21T13:28:59.505 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T13:28:59.505 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T13:28:59.505 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T13:28:59.505 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.505 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T13:28:59.505 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.505 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T13:28:59.506 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T13:28:59.506 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.506 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T13:28:59.506 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.506 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T13:28:59.506 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.506 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T13:28:59.506 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.506 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T13:28:59.506 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup 2024-03-21T13:28:59.506 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/locale 2024-03-21T13:28:59.506 INFO:tasks.workunit.client.0.smithi097.stderr:++ awk '{print $1}' 2024-03-21T13:28:59.506 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.506 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T13:28:59.507 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.507 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T13:28:59.507 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T13:28:59.507 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T13:28:59.507 INFO:tasks.workunit.client.0.smithi097.stdout:test api_tier_pp on pid 79719 2024-03-21T13:28:59.509 INFO:tasks.workunit.client.0.smithi097.stderr:+ ff=api_tier_pp 2024-03-21T13:28:59.509 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=79719 2024-03-21T13:28:59.509 INFO:tasks.workunit.client.0.smithi097.stderr:+ echo 'test api_tier_pp on pid 79719' 2024-03-21T13:28:59.509 INFO:tasks.workunit.client.0.smithi097.stderr:+ pids[$f]=79719 2024-03-21T13:28:59.509 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T13:28:59.509 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T13:28:59.509 INFO:tasks.workunit.client.0.smithi097.stderr:++ printf %25s api_pool 2024-03-21T13:28:59.509 INFO:tasks.workunit.client.0.smithi097.stderr:+ r=' api_pool' 2024-03-21T13:28:59.509 INFO:tasks.workunit.client.0.smithi097.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T13:28:59.509 INFO:tasks.workunit.client.0.smithi097.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T13:28:59.509 INFO:tasks.workunit.client.0.smithi097.stderr:++++ LANG=en_US.UTF-8 2024-03-21T13:28:59.509 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.509 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T13:28:59.510 INFO:tasks.workunit.client.0.smithi097.stderr:++ awk '{print $1}' 2024-03-21T13:28:59.510 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.510 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG=en_US.UTF-8 2024-03-21T13:28:59.510 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup config 2024-03-21T13:28:59.510 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.510 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.510 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' dumb = linux ']' 2024-03-21T13:28:59.510 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.510 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T13:28:59.510 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.510 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/less.sh 2024-03-21T13:28:59.510 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T13:28:59.510 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.510 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T13:28:59.511 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.511 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T13:28:59.511 INFO:tasks.workunit.client.0.smithi097.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T13:28:59.511 INFO:tasks.workunit.client.0.smithi097.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_tier_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_tier_pp.xml 2>&1 | tee ceph_test_rados_api_tier_pp.log | sed "s/^/ api_tier_pp: /"' 2024-03-21T13:28:59.511 INFO:tasks.workunit.client.0.smithi097.stderr:++ echo api_pool 2024-03-21T13:28:59.511 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T13:28:59.511 INFO:tasks.workunit.client.0.smithi097.stderr:+ . /etc/bashrc 2024-03-21T13:28:59.511 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -z '' ']' 2024-03-21T13:28:59.511 INFO:tasks.workunit.client.0.smithi097.stderr:++ BASHRCSOURCED=Y 2024-03-21T13:28:59.511 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.511 INFO:tasks.workunit.client.0.smithi097.stderr:++ shopt -q login_shell 2024-03-21T13:28:59.511 INFO:tasks.workunit.client.0.smithi097.stderr:+ ff=api_pool 2024-03-21T13:28:59.512 INFO:tasks.workunit.client.0.smithi097.stdout:test api_pool on pid 79725 2024-03-21T13:28:59.514 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=79725 2024-03-21T13:28:59.514 INFO:tasks.workunit.client.0.smithi097.stderr:+ echo 'test api_pool on pid 79725' 2024-03-21T13:28:59.514 INFO:tasks.workunit.client.0.smithi097.stderr:+ pids[$f]=79725 2024-03-21T13:28:59.514 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T13:28:59.514 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T13:28:59.514 INFO:tasks.workunit.client.0.smithi097.stderr:++ printf %25s api_snapshots 2024-03-21T13:28:59.514 INFO:tasks.workunit.client.0.smithi097.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_pool --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_pool.xml 2>&1 | tee ceph_test_rados_api_pool.log | sed "s/^/ api_pool: /"' 2024-03-21T13:28:59.514 INFO:tasks.workunit.client.0.smithi097.stderr:+ r=' api_snapshots' 2024-03-21T13:28:59.514 INFO:tasks.workunit.client.0.smithi097.stderr:+++ umask 2024-03-21T13:28:59.515 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T13:28:59.515 INFO:tasks.workunit.client.0.smithi097.stderr:++ SHELL=/bin/bash 2024-03-21T13:28:59.515 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.515 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T13:28:59.515 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.515 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T13:28:59.515 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T13:28:59.515 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.515 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T13:28:59.515 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.515 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T13:28:59.515 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.515 INFO:tasks.workunit.client.0.smithi097.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.515 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.516 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T13:28:59.516 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.516 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T13:28:59.516 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.516 INFO:tasks.workunit.client.0.smithi097.stderr:++ awk '{print $1}' 2024-03-21T13:28:59.516 INFO:tasks.workunit.client.0.smithi097.stderr:++ echo api_snapshots 2024-03-21T13:28:59.516 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T13:28:59.516 INFO:tasks.workunit.client.0.smithi097.stderr:+ . /etc/bashrc 2024-03-21T13:28:59.516 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -z '' ']' 2024-03-21T13:28:59.516 INFO:tasks.workunit.client.0.smithi097.stderr:+++++ readlink /proc/79653/exe 2024-03-21T13:28:59.516 INFO:tasks.workunit.client.0.smithi097.stderr:++++ basename /usr/bin/bash 2024-03-21T13:28:59.516 INFO:tasks.workunit.client.0.smithi097.stderr:++ BASHRCSOURCED=Y 2024-03-21T13:28:59.516 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.516 INFO:tasks.workunit.client.0.smithi097.stderr:++ shopt -q login_shell 2024-03-21T13:28:59.516 INFO:tasks.workunit.client.0.smithi097.stderr:+++ umask 2024-03-21T13:28:59.517 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T13:28:59.517 INFO:tasks.workunit.client.0.smithi097.stderr:++ SHELL=/bin/bash 2024-03-21T13:28:59.517 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.517 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_declare='declare -f' 2024-03-21T13:28:59.517 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_opt=-f 2024-03-21T13:28:59.517 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T13:28:59.517 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.517 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T13:28:59.517 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export which_declare 2024-03-21T13:28:59.517 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export -f which 2024-03-21T13:28:59.517 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset i 2024-03-21T13:28:59.517 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset -f pathmunge 2024-03-21T13:28:59.517 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T13:28:59.517 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.517 INFO:tasks.workunit.client.0.smithi097.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T13:28:59.518 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T13:28:59.518 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.518 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T13:28:59.518 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.518 INFO:tasks.workunit.client.0.smithi097.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.518 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.518 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T13:28:59.518 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.518 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T13:28:59.518 INFO:tasks.workunit.client.0.smithi097.stderr:+ export PATH 2024-03-21T13:28:59.518 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.518 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T13:28:59.518 INFO:tasks.workunit.client.0.smithi097.stderr:+ unset rc 2024-03-21T13:28:59.518 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T13:28:59.518 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T13:28:59.519 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T13:28:59.519 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_api_lock_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock_pp.xml 2024-03-21T13:28:59.519 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.519 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T13:28:59.519 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.519 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T13:28:59.519 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T13:28:59.519 INFO:tasks.workunit.client.0.smithi097.stderr:+++ return 2024-03-21T13:28:59.519 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.519 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T13:28:59.519 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.519 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T13:28:59.519 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export S_COLORS=auto 2024-03-21T13:28:59.519 INFO:tasks.workunit.client.0.smithi097.stderr:+++ S_COLORS=auto 2024-03-21T13:28:59.519 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.520 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T13:28:59.520 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.520 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T13:28:59.520 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.520 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T13:28:59.520 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T13:28:59.520 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T13:28:59.520 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.520 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T13:28:59.520 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.520 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T13:28:59.520 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T13:28:59.520 INFO:tasks.workunit.client.0.smithi097.stderr:+++ return 2024-03-21T13:28:59.520 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.520 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T13:28:59.520 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.521 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T13:28:59.521 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export S_COLORS=auto 2024-03-21T13:28:59.521 INFO:tasks.workunit.client.0.smithi097.stderr:+++ S_COLORS=auto 2024-03-21T13:28:59.521 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.521 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T13:28:59.521 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.521 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T13:28:59.521 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.521 INFO:tasks.workunit.client.0.smithi097.stderr:+ tee ceph_test_rados_api_lock_pp.log 2024-03-21T13:28:59.521 INFO:tasks.workunit.client.0.smithi097.stdout:test api_snapshots on pid 79746 2024-03-21T13:28:59.523 INFO:tasks.workunit.client.0.smithi097.stderr:+ ff=api_snapshots 2024-03-21T13:28:59.523 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=79746 2024-03-21T13:28:59.523 INFO:tasks.workunit.client.0.smithi097.stderr:+ echo 'test api_snapshots on pid 79746' 2024-03-21T13:28:59.524 INFO:tasks.workunit.client.0.smithi097.stderr:+ pids[$f]=79746 2024-03-21T13:28:59.524 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T13:28:59.524 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T13:28:59.524 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.524 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T13:28:59.524 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.524 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T13:28:59.524 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T13:28:59.524 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T13:28:59.524 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T13:28:59.524 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T13:28:59.524 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.524 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T13:28:59.524 INFO:tasks.workunit.client.0.smithi097.stderr:++ printf %25s api_snapshots_pp 2024-03-21T13:28:59.524 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.525 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T13:28:59.525 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T13:28:59.525 INFO:tasks.workunit.client.0.smithi097.stderr:+++ return 2024-03-21T13:28:59.525 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.525 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T13:28:59.525 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.525 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T13:28:59.525 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export S_COLORS=auto 2024-03-21T13:28:59.525 INFO:tasks.workunit.client.0.smithi097.stderr:+++ S_COLORS=auto 2024-03-21T13:28:59.525 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.525 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T13:28:59.525 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.525 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T13:28:59.525 INFO:tasks.workunit.client.0.smithi097.stderr:+ r=' api_snapshots_pp' 2024-03-21T13:28:59.525 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.525 INFO:tasks.workunit.client.0.smithi097.stderr:++ awk '{print $1}' 2024-03-21T13:28:59.526 INFO:tasks.workunit.client.0.smithi097.stderr:++ echo api_snapshots_pp 2024-03-21T13:28:59.526 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T13:28:59.526 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T13:28:59.526 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T13:28:59.526 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.526 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T13:28:59.526 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.526 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T13:28:59.526 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T13:28:59.526 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.526 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T13:28:59.526 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T13:28:59.526 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T13:28:59.526 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.527 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T13:28:59.527 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.527 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T13:28:59.527 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T13:28:59.527 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.527 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T13:28:59.527 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.527 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T13:28:59.527 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.527 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T13:28:59.527 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.527 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T13:28:59.527 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup 2024-03-21T13:28:59.527 INFO:tasks.workunit.client.0.smithi097.stderr:+ ff=api_snapshots_pp 2024-03-21T13:28:59.527 INFO:tasks.workunit.client.0.smithi097.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots.xml 2>&1 | tee ceph_test_rados_api_snapshots.log | sed "s/^/ api_snapshots: /"' 2024-03-21T13:28:59.528 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=79767 2024-03-21T13:28:59.528 INFO:tasks.workunit.client.0.smithi097.stdout:test api_snapshots_pp on pid 79767 2024-03-21T13:28:59.531 INFO:tasks.workunit.client.0.smithi097.stderr:+ echo 'test api_snapshots_pp on pid 79767' 2024-03-21T13:28:59.531 INFO:tasks.workunit.client.0.smithi097.stderr:+ pids[$f]=79767 2024-03-21T13:28:59.531 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T13:28:59.531 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T13:28:59.532 INFO:tasks.workunit.client.0.smithi097.stderr:+ sed 's/^/ api_lock_pp: /' 2024-03-21T13:28:59.532 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/locale 2024-03-21T13:28:59.532 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T13:28:59.532 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T13:28:59.532 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T13:28:59.532 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.532 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T13:28:59.532 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.532 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T13:28:59.532 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T13:28:59.532 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.532 INFO:tasks.workunit.client.0.smithi097.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots_pp.xml 2>&1 | tee ceph_test_rados_api_snapshots_pp.log | sed "s/^/ api_snapshots_pp: /"' 2024-03-21T13:28:59.532 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T13:28:59.532 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.532 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T13:28:59.533 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.533 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T13:28:59.533 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T13:28:59.533 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T13:28:59.533 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T13:28:59.533 INFO:tasks.workunit.client.0.smithi097.stderr:++ printf %25s api_stat 2024-03-21T13:28:59.533 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T13:28:59.533 INFO:tasks.workunit.client.0.smithi097.stderr:+ . /etc/bashrc 2024-03-21T13:28:59.533 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -z '' ']' 2024-03-21T13:28:59.533 INFO:tasks.workunit.client.0.smithi097.stderr:++ BASHRCSOURCED=Y 2024-03-21T13:28:59.533 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T13:28:59.533 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.533 INFO:tasks.workunit.client.0.smithi097.stderr:++ shopt -q login_shell 2024-03-21T13:28:59.533 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T13:28:59.533 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T13:28:59.534 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.534 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T13:28:59.534 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.534 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T13:28:59.534 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T13:28:59.534 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.534 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T13:28:59.534 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.534 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T13:28:59.534 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.534 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T13:28:59.534 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.534 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T13:28:59.534 INFO:tasks.workunit.client.0.smithi097.stderr:+ r=' api_stat' 2024-03-21T13:28:59.535 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup 2024-03-21T13:28:59.535 INFO:tasks.workunit.client.0.smithi097.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T13:28:59.535 INFO:tasks.workunit.client.0.smithi097.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T13:28:59.535 INFO:tasks.workunit.client.0.smithi097.stderr:++++ LANG=en_US.UTF-8 2024-03-21T13:28:59.535 INFO:tasks.workunit.client.0.smithi097.stderr:+++ umask 2024-03-21T13:28:59.535 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.535 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T13:28:59.535 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/locale 2024-03-21T13:28:59.535 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.535 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG=en_US.UTF-8 2024-03-21T13:28:59.535 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup config 2024-03-21T13:28:59.535 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.535 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.535 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' dumb = linux ']' 2024-03-21T13:28:59.535 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T13:28:59.536 INFO:tasks.workunit.client.0.smithi097.stderr:++ SHELL=/bin/bash 2024-03-21T13:28:59.536 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.536 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T13:28:59.536 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.536 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/less.sh 2024-03-21T13:28:59.536 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T13:28:59.536 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.536 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.536 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T13:28:59.536 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T13:28:59.536 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.536 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.536 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T13:28:59.536 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T13:28:59.537 INFO:tasks.workunit.client.0.smithi097.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T13:28:59.537 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T13:28:59.537 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.537 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T13:28:59.537 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.537 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T13:28:59.537 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.537 INFO:tasks.workunit.client.0.smithi097.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.537 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.537 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T13:28:59.537 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.537 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T13:28:59.537 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.537 INFO:tasks.workunit.client.0.smithi097.stderr:++ echo api_stat 2024-03-21T13:28:59.537 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.537 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T13:28:59.537 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.538 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T13:28:59.538 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T13:28:59.538 INFO:tasks.workunit.client.0.smithi097.stderr:++ awk '{print $1}' 2024-03-21T13:28:59.538 INFO:tasks.workunit.client.0.smithi097.stderr:+ . /etc/bashrc 2024-03-21T13:28:59.538 INFO:tasks.workunit.client.0.smithi097.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T13:28:59.538 INFO:tasks.workunit.client.0.smithi097.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T13:28:59.538 INFO:tasks.workunit.client.0.smithi097.stderr:++++ LANG=en_US.UTF-8 2024-03-21T13:28:59.538 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.538 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T13:28:59.538 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.538 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG=en_US.UTF-8 2024-03-21T13:28:59.538 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup config 2024-03-21T13:28:59.538 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T13:28:59.538 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -z '' ']' 2024-03-21T13:28:59.538 INFO:tasks.workunit.client.0.smithi097.stderr:++ BASHRCSOURCED=Y 2024-03-21T13:28:59.539 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.539 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.539 INFO:tasks.workunit.client.0.smithi097.stderr:++ shopt -q login_shell 2024-03-21T13:28:59.539 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.539 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' dumb = linux ']' 2024-03-21T13:28:59.539 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.539 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T13:28:59.539 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.539 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/less.sh 2024-03-21T13:28:59.539 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T13:28:59.539 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.539 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T13:28:59.539 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.539 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T13:28:59.540 INFO:tasks.workunit.client.0.smithi097.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T13:28:59.540 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T13:28:59.540 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T13:28:59.540 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T13:28:59.540 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.540 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T13:28:59.540 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.540 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T13:28:59.540 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T13:28:59.540 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.540 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T13:28:59.540 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T13:28:59.540 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T13:28:59.540 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.540 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T13:28:59.540 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.540 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T13:28:59.541 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T13:28:59.541 INFO:tasks.workunit.client.0.smithi097.stderr:+++++ readlink /proc/79629/exe 2024-03-21T13:28:59.541 INFO:tasks.workunit.client.0.smithi097.stderr:+++ return 2024-03-21T13:28:59.541 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.541 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T13:28:59.541 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.541 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T13:28:59.541 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export S_COLORS=auto 2024-03-21T13:28:59.541 INFO:tasks.workunit.client.0.smithi097.stderr:+++ S_COLORS=auto 2024-03-21T13:28:59.541 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.541 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T13:28:59.541 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.541 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T13:28:59.541 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.541 INFO:tasks.workunit.client.0.smithi097.stderr:++++ basename /usr/bin/bash 2024-03-21T13:28:59.541 INFO:tasks.workunit.client.0.smithi097.stderr:+ ff=api_stat 2024-03-21T13:28:59.542 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=79819 2024-03-21T13:28:59.542 INFO:tasks.workunit.client.0.smithi097.stdout:test api_stat on pid 79819 2024-03-21T13:28:59.544 INFO:tasks.workunit.client.0.smithi097.stderr:+ echo 'test api_stat on pid 79819' 2024-03-21T13:28:59.544 INFO:tasks.workunit.client.0.smithi097.stderr:+ pids[$f]=79819 2024-03-21T13:28:59.544 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T13:28:59.544 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T13:28:59.544 INFO:tasks.workunit.client.0.smithi097.stderr:++ printf %25s api_stat_pp 2024-03-21T13:28:59.544 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_declare='declare -f' 2024-03-21T13:28:59.544 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_opt=-f 2024-03-21T13:28:59.544 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export which_declare 2024-03-21T13:28:59.545 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export -f which 2024-03-21T13:28:59.545 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset i 2024-03-21T13:28:59.545 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset -f pathmunge 2024-03-21T13:28:59.545 INFO:tasks.workunit.client.0.smithi097.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T13:28:59.545 INFO:tasks.workunit.client.0.smithi097.stderr:+ r=' api_stat_pp' 2024-03-21T13:28:59.545 INFO:tasks.workunit.client.0.smithi097.stderr:+ export PATH 2024-03-21T13:28:59.545 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T13:28:59.545 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T13:28:59.545 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T13:28:59.545 INFO:tasks.workunit.client.0.smithi097.stderr:+ unset rc 2024-03-21T13:28:59.545 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T13:28:59.545 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.545 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T13:28:59.545 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.545 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T13:28:59.546 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T13:28:59.546 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.546 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T13:28:59.546 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.546 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T13:28:59.546 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.546 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T13:28:59.546 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.546 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T13:28:59.546 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup 2024-03-21T13:28:59.546 INFO:tasks.workunit.client.0.smithi097.stderr:++ awk '{print $1}' 2024-03-21T13:28:59.546 INFO:tasks.workunit.client.0.smithi097.stderr:+ tee ceph_test_rados_api_lock.log 2024-03-21T13:28:59.546 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/locale 2024-03-21T13:28:59.547 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_api_lock --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock.xml 2024-03-21T13:28:59.547 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T13:28:59.547 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T13:28:59.547 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T13:28:59.547 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.547 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T13:28:59.547 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.547 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T13:28:59.547 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T13:28:59.547 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.547 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T13:28:59.547 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.547 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T13:28:59.547 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.547 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T13:28:59.547 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.547 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T13:28:59.548 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup 2024-03-21T13:28:59.548 INFO:tasks.workunit.client.0.smithi097.stderr:+++ umask 2024-03-21T13:28:59.548 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T13:28:59.548 INFO:tasks.workunit.client.0.smithi097.stderr:++ SHELL=/bin/bash 2024-03-21T13:28:59.548 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.548 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T13:28:59.548 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.548 INFO:tasks.workunit.client.0.smithi097.stderr:+ sed 's/^/ api_lock: /' 2024-03-21T13:28:59.548 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T13:28:59.548 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T13:28:59.548 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.548 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T13:28:59.548 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.548 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T13:28:59.548 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.548 INFO:tasks.workunit.client.0.smithi097.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.549 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.549 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T13:28:59.549 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.549 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T13:28:59.549 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.549 INFO:tasks.workunit.client.0.smithi097.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T13:28:59.549 INFO:tasks.workunit.client.0.smithi097.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T13:28:59.549 INFO:tasks.workunit.client.0.smithi097.stderr:++++ LANG=en_US.UTF-8 2024-03-21T13:28:59.549 INFO:tasks.workunit.client.0.smithi097.stderr:+++++ readlink /proc/79672/exe 2024-03-21T13:28:59.549 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.549 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T13:28:59.549 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.549 INFO:tasks.workunit.client.0.smithi097.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat.xml 2>&1 | tee ceph_test_rados_api_stat.log | sed "s/^/ api_stat: /"' 2024-03-21T13:28:59.549 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG=en_US.UTF-8 2024-03-21T13:28:59.550 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup config 2024-03-21T13:28:59.550 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.550 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.550 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' dumb = linux ']' 2024-03-21T13:28:59.550 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.550 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T13:28:59.550 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.550 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/less.sh 2024-03-21T13:28:59.550 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T13:28:59.550 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.550 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T13:28:59.550 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.550 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T13:28:59.550 INFO:tasks.workunit.client.0.smithi097.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T13:28:59.550 INFO:tasks.workunit.client.0.smithi097.stderr:++++ basename /usr/bin/bash 2024-03-21T13:28:59.550 INFO:tasks.workunit.client.0.smithi097.stderr:++ echo api_stat_pp 2024-03-21T13:28:59.551 INFO:tasks.workunit.client.0.smithi097.stdout:test api_stat_pp on pid 79856 2024-03-21T13:28:59.555 INFO:tasks.workunit.client.0.smithi097.stderr:+ ff=api_stat_pp 2024-03-21T13:28:59.555 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=79856 2024-03-21T13:28:59.556 INFO:tasks.workunit.client.0.smithi097.stderr:+ echo 'test api_stat_pp on pid 79856' 2024-03-21T13:28:59.556 INFO:tasks.workunit.client.0.smithi097.stderr:+ pids[$f]=79856 2024-03-21T13:28:59.556 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T13:28:59.556 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T13:28:59.556 INFO:tasks.workunit.client.0.smithi097.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat_pp.xml 2>&1 | tee ceph_test_rados_api_stat_pp.log | sed "s/^/ api_stat_pp: /"' 2024-03-21T13:28:59.556 INFO:tasks.workunit.client.0.smithi097.stderr:++ printf %25s api_watch_notify 2024-03-21T13:28:59.556 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T13:28:59.556 INFO:tasks.workunit.client.0.smithi097.stderr:+ r=' api_watch_notify' 2024-03-21T13:28:59.556 INFO:tasks.workunit.client.0.smithi097.stderr:+ . /etc/bashrc 2024-03-21T13:28:59.556 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T13:28:59.556 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T13:28:59.556 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T13:28:59.556 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.556 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T13:28:59.556 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.557 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T13:28:59.557 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T13:28:59.557 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.557 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -z '' ']' 2024-03-21T13:28:59.557 INFO:tasks.workunit.client.0.smithi097.stderr:++ BASHRCSOURCED=Y 2024-03-21T13:28:59.557 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.557 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/locale 2024-03-21T13:28:59.557 INFO:tasks.workunit.client.0.smithi097.stderr:++ shopt -q login_shell 2024-03-21T13:28:59.557 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T13:28:59.557 INFO:tasks.workunit.client.0.smithi097.stderr:+ . /etc/bashrc 2024-03-21T13:28:59.557 INFO:tasks.workunit.client.0.smithi097.stderr:+++ umask 2024-03-21T13:28:59.557 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -z '' ']' 2024-03-21T13:28:59.557 INFO:tasks.workunit.client.0.smithi097.stderr:++ BASHRCSOURCED=Y 2024-03-21T13:28:59.557 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.557 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.558 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T13:28:59.558 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T13:28:59.558 INFO:tasks.workunit.client.0.smithi097.stderr:++ SHELL=/bin/bash 2024-03-21T13:28:59.558 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.558 INFO:tasks.workunit.client.0.smithi097.stderr:++ shopt -q login_shell 2024-03-21T13:28:59.558 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T13:28:59.558 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T13:28:59.558 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.558 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T13:28:59.558 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.558 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T13:28:59.558 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T13:28:59.558 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.558 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T13:28:59.559 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.559 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T13:28:59.559 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.559 INFO:tasks.workunit.client.0.smithi097.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.559 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.559 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T13:28:59.559 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.559 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T13:28:59.559 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.559 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_declare='declare -f' 2024-03-21T13:28:59.559 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_opt=-f 2024-03-21T13:28:59.559 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export which_declare 2024-03-21T13:28:59.559 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export -f which 2024-03-21T13:28:59.559 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset i 2024-03-21T13:28:59.560 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset -f pathmunge 2024-03-21T13:28:59.560 INFO:tasks.workunit.client.0.smithi097.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T13:28:59.560 INFO:tasks.workunit.client.0.smithi097.stderr:+ export PATH 2024-03-21T13:28:59.560 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T13:28:59.560 INFO:tasks.workunit.client.0.smithi097.stderr:+++ umask 2024-03-21T13:28:59.560 INFO:tasks.workunit.client.0.smithi097.stderr:+ unset rc 2024-03-21T13:28:59.560 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T13:28:59.560 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T13:28:59.560 INFO:tasks.workunit.client.0.smithi097.stderr:++ SHELL=/bin/bash 2024-03-21T13:28:59.560 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.560 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T13:28:59.560 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.560 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T13:28:59.560 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T13:28:59.560 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.561 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T13:28:59.561 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.561 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T13:28:59.561 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.561 INFO:tasks.workunit.client.0.smithi097.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.561 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.561 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T13:28:59.561 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.561 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T13:28:59.561 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.561 INFO:tasks.workunit.client.0.smithi097.stderr:++ awk '{print $1}' 2024-03-21T13:28:59.561 INFO:tasks.workunit.client.0.smithi097.stderr:++ echo api_watch_notify 2024-03-21T13:28:59.561 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.561 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T13:28:59.561 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.561 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T13:28:59.562 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T13:28:59.562 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T13:28:59.562 INFO:tasks.workunit.client.0.smithi097.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T13:28:59.562 INFO:tasks.workunit.client.0.smithi097.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T13:28:59.562 INFO:tasks.workunit.client.0.smithi097.stderr:++++ LANG=en_US.UTF-8 2024-03-21T13:28:59.562 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.562 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T13:28:59.562 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.562 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG=en_US.UTF-8 2024-03-21T13:28:59.562 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup config 2024-03-21T13:28:59.562 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.562 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.562 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' dumb = linux ']' 2024-03-21T13:28:59.562 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T13:28:59.563 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.563 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T13:28:59.563 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T13:28:59.563 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.563 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/less.sh 2024-03-21T13:28:59.563 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T13:28:59.563 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T13:28:59.563 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.563 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.563 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T13:28:59.563 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T13:28:59.563 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.563 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.563 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T13:28:59.563 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T13:28:59.564 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T13:28:59.564 INFO:tasks.workunit.client.0.smithi097.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T13:28:59.564 INFO:tasks.workunit.client.0.smithi097.stderr:+++ return 2024-03-21T13:28:59.564 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.564 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T13:28:59.564 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.564 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T13:28:59.564 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export S_COLORS=auto 2024-03-21T13:28:59.564 INFO:tasks.workunit.client.0.smithi097.stderr:+++ S_COLORS=auto 2024-03-21T13:28:59.564 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.564 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T13:28:59.564 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.564 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T13:28:59.564 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.565 INFO:tasks.workunit.client.0.smithi097.stdout:test api_watch_notify on pid 79889 2024-03-21T13:28:59.571 INFO:tasks.workunit.client.0.smithi097.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T13:28:59.571 INFO:tasks.workunit.client.0.smithi097.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T13:28:59.571 INFO:tasks.workunit.client.0.smithi097.stderr:++++ LANG=en_US.UTF-8 2024-03-21T13:28:59.571 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.571 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T13:28:59.571 INFO:tasks.workunit.client.0.smithi097.stderr:+ ff=api_watch_notify 2024-03-21T13:28:59.571 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.571 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG=en_US.UTF-8 2024-03-21T13:28:59.571 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup config 2024-03-21T13:28:59.571 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.571 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=79889 2024-03-21T13:28:59.571 INFO:tasks.workunit.client.0.smithi097.stderr:+ echo 'test api_watch_notify on pid 79889' 2024-03-21T13:28:59.571 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.572 INFO:tasks.workunit.client.0.smithi097.stderr:+ pids[$f]=79889 2024-03-21T13:28:59.572 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' dumb = linux ']' 2024-03-21T13:28:59.572 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T13:28:59.572 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T13:28:59.572 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.572 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T13:28:59.572 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.572 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/less.sh 2024-03-21T13:28:59.572 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T13:28:59.572 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.572 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T13:28:59.572 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.572 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T13:28:59.572 INFO:tasks.workunit.client.0.smithi097.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T13:28:59.572 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_api_misc --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc.xml 2024-03-21T13:28:59.572 INFO:tasks.workunit.client.0.smithi097.stderr:+++++ readlink /proc/79691/exe 2024-03-21T13:28:59.572 INFO:tasks.workunit.client.0.smithi097.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify.xml 2>&1 | tee ceph_test_rados_api_watch_notify.log | sed "s/^/ api_watch_notify: /"' 2024-03-21T13:28:59.573 INFO:tasks.workunit.client.0.smithi097.stderr:+++++ readlink /proc/79719/exe 2024-03-21T13:28:59.573 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T13:28:59.573 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T13:28:59.573 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T13:28:59.573 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.573 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T13:28:59.573 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.573 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T13:28:59.573 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T13:28:59.573 INFO:tasks.workunit.client.0.smithi097.stderr:+++ return 2024-03-21T13:28:59.573 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.573 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T13:28:59.573 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.574 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T13:28:59.574 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export S_COLORS=auto 2024-03-21T13:28:59.574 INFO:tasks.workunit.client.0.smithi097.stderr:+++ S_COLORS=auto 2024-03-21T13:28:59.574 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.574 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T13:28:59.574 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.574 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T13:28:59.574 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.574 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T13:28:59.574 INFO:tasks.workunit.client.0.smithi097.stderr:+ . /etc/bashrc 2024-03-21T13:28:59.574 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -z '' ']' 2024-03-21T13:28:59.574 INFO:tasks.workunit.client.0.smithi097.stderr:++++ basename /usr/bin/bash 2024-03-21T13:28:59.574 INFO:tasks.workunit.client.0.smithi097.stderr:++ BASHRCSOURCED=Y 2024-03-21T13:28:59.574 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.574 INFO:tasks.workunit.client.0.smithi097.stderr:++ shopt -q login_shell 2024-03-21T13:28:59.575 INFO:tasks.workunit.client.0.smithi097.stderr:++ printf %25s api_watch_notify_pp 2024-03-21T13:28:59.575 INFO:tasks.workunit.client.0.smithi097.stderr:+ r=' api_watch_notify_pp' 2024-03-21T13:28:59.575 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_declare='declare -f' 2024-03-21T13:28:59.575 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_opt=-f 2024-03-21T13:28:59.575 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export which_declare 2024-03-21T13:28:59.575 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export -f which 2024-03-21T13:28:59.575 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset i 2024-03-21T13:28:59.575 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset -f pathmunge 2024-03-21T13:28:59.575 INFO:tasks.workunit.client.0.smithi097.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T13:28:59.575 INFO:tasks.workunit.client.0.smithi097.stderr:++++ basename /usr/bin/bash 2024-03-21T13:28:59.575 INFO:tasks.workunit.client.0.smithi097.stderr:+ export PATH 2024-03-21T13:28:59.575 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T13:28:59.575 INFO:tasks.workunit.client.0.smithi097.stderr:+ unset rc 2024-03-21T13:28:59.575 INFO:tasks.workunit.client.0.smithi097.stderr:+++ umask 2024-03-21T13:28:59.576 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T13:28:59.576 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T13:28:59.576 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T13:28:59.576 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.576 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T13:28:59.576 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.576 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T13:28:59.576 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_declare='declare -f' 2024-03-21T13:28:59.576 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T13:28:59.576 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_opt=-f 2024-03-21T13:28:59.576 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.576 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export which_declare 2024-03-21T13:28:59.576 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export -f which 2024-03-21T13:28:59.576 INFO:tasks.workunit.client.0.smithi097.stderr:+ sed 's/^/ api_tier_pp: /' 2024-03-21T13:28:59.576 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset i 2024-03-21T13:28:59.576 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T13:28:59.576 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset -f pathmunge 2024-03-21T13:28:59.577 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T13:28:59.577 INFO:tasks.workunit.client.0.smithi097.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T13:28:59.577 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T13:28:59.577 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.577 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T13:28:59.577 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.577 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T13:28:59.577 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T13:28:59.577 INFO:tasks.workunit.client.0.smithi097.stderr:+ export PATH 2024-03-21T13:28:59.577 INFO:tasks.workunit.client.0.smithi097.stderr:+++ return 2024-03-21T13:28:59.577 INFO:tasks.workunit.client.0.smithi097.stderr:+ tee ceph_test_rados_api_tier_pp.log 2024-03-21T13:28:59.577 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.577 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T13:28:59.577 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T13:28:59.577 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.577 INFO:tasks.workunit.client.0.smithi097.stderr:+ unset rc 2024-03-21T13:28:59.577 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T13:28:59.578 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export S_COLORS=auto 2024-03-21T13:28:59.578 INFO:tasks.workunit.client.0.smithi097.stderr:+++ S_COLORS=auto 2024-03-21T13:28:59.578 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.578 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T13:28:59.578 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.578 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T13:28:59.578 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.578 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_api_tier_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_tier_pp.xml 2024-03-21T13:28:59.578 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T13:28:59.578 INFO:tasks.workunit.client.0.smithi097.stderr:++ SHELL=/bin/bash 2024-03-21T13:28:59.578 INFO:tasks.workunit.client.0.smithi097.stderr:++ awk '{print $1}' 2024-03-21T13:28:59.578 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.578 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T13:28:59.579 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.579 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T13:28:59.579 INFO:tasks.workunit.client.0.smithi097.stderr:+ tee ceph_test_rados_api_misc.log 2024-03-21T13:28:59.579 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T13:28:59.579 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.579 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T13:28:59.579 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.579 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T13:28:59.579 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.579 INFO:tasks.workunit.client.0.smithi097.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.579 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.579 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T13:28:59.579 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.579 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T13:28:59.579 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.579 INFO:tasks.workunit.client.0.smithi097.stderr:++ echo api_watch_notify_pp 2024-03-21T13:28:59.579 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T13:28:59.580 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T13:28:59.580 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T13:28:59.580 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.580 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T13:28:59.580 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.580 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T13:28:59.580 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T13:28:59.580 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.580 INFO:tasks.workunit.client.0.smithi097.stderr:+ ff=api_watch_notify_pp 2024-03-21T13:28:59.580 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=79919 2024-03-21T13:28:59.580 INFO:tasks.workunit.client.0.smithi097.stdout:test api_watch_notify_pp on pid 79919 2024-03-21T13:28:59.587 INFO:tasks.workunit.client.0.smithi097.stderr:+ echo 'test api_watch_notify_pp on pid 79919' 2024-03-21T13:28:59.587 INFO:tasks.workunit.client.0.smithi097.stderr:+ pids[$f]=79919 2024-03-21T13:28:59.588 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T13:28:59.588 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T13:28:59.588 INFO:tasks.workunit.client.0.smithi097.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify_pp.xml 2>&1 | tee ceph_test_rados_api_watch_notify_pp.log | sed "s/^/ api_watch_notify_pp: /"' 2024-03-21T13:28:59.588 INFO:tasks.workunit.client.0.smithi097.stderr:+ sed 's/^/ api_misc: /' 2024-03-21T13:28:59.588 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T13:28:59.588 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T13:28:59.588 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T13:28:59.588 INFO:tasks.workunit.client.0.smithi097.stderr:+ . /etc/bashrc 2024-03-21T13:28:59.588 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T13:28:59.588 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.588 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T13:28:59.588 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.588 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T13:28:59.588 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T13:28:59.588 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.589 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T13:28:59.589 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.589 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -z '' ']' 2024-03-21T13:28:59.589 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T13:28:59.589 INFO:tasks.workunit.client.0.smithi097.stderr:++ BASHRCSOURCED=Y 2024-03-21T13:28:59.589 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.589 INFO:tasks.workunit.client.0.smithi097.stderr:++ shopt -q login_shell 2024-03-21T13:28:59.589 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.589 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T13:28:59.589 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.589 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T13:28:59.589 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup 2024-03-21T13:28:59.589 INFO:tasks.workunit.client.0.smithi097.stderr:+++++ readlink /proc/79725/exe 2024-03-21T13:28:59.589 INFO:tasks.workunit.client.0.smithi097.stderr:+++ umask 2024-03-21T13:28:59.589 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T13:28:59.590 INFO:tasks.workunit.client.0.smithi097.stderr:++ SHELL=/bin/bash 2024-03-21T13:28:59.590 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.590 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T13:28:59.590 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.590 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T13:28:59.590 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T13:28:59.590 INFO:tasks.workunit.client.0.smithi097.stderr:++++ basename /usr/bin/bash 2024-03-21T13:28:59.590 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.590 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T13:28:59.590 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.590 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T13:28:59.590 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.590 INFO:tasks.workunit.client.0.smithi097.stderr:++ printf %25s api_cmd 2024-03-21T13:28:59.591 INFO:tasks.workunit.client.0.smithi097.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.591 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/locale 2024-03-21T13:28:59.591 INFO:tasks.workunit.client.0.smithi097.stderr:+ r=' api_cmd' 2024-03-21T13:28:59.591 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_declare='declare -f' 2024-03-21T13:28:59.591 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_opt=-f 2024-03-21T13:28:59.591 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export which_declare 2024-03-21T13:28:59.591 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export -f which 2024-03-21T13:28:59.591 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset i 2024-03-21T13:28:59.591 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset -f pathmunge 2024-03-21T13:28:59.591 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.591 INFO:tasks.workunit.client.0.smithi097.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T13:28:59.591 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T13:28:59.591 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.591 INFO:tasks.workunit.client.0.smithi097.stderr:+ export PATH 2024-03-21T13:28:59.591 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T13:28:59.592 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T13:28:59.592 INFO:tasks.workunit.client.0.smithi097.stderr:+ unset rc 2024-03-21T13:28:59.592 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.592 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T13:28:59.592 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T13:28:59.592 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T13:28:59.592 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.592 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T13:28:59.592 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.592 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T13:28:59.592 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T13:28:59.592 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.592 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T13:28:59.592 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.592 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T13:28:59.593 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.593 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T13:28:59.593 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.593 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T13:28:59.593 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup 2024-03-21T13:28:59.593 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_api_pool --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_pool.xml 2024-03-21T13:28:59.593 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T13:28:59.593 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T13:28:59.593 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T13:28:59.593 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.593 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T13:28:59.593 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.593 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T13:28:59.594 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T13:28:59.594 INFO:tasks.workunit.client.0.smithi097.stderr:+++ return 2024-03-21T13:28:59.594 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.594 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T13:28:59.594 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.594 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T13:28:59.594 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export S_COLORS=auto 2024-03-21T13:28:59.594 INFO:tasks.workunit.client.0.smithi097.stderr:+++ S_COLORS=auto 2024-03-21T13:28:59.594 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.594 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T13:28:59.594 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.594 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T13:28:59.594 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.594 INFO:tasks.workunit.client.0.smithi097.stderr:+ tee ceph_test_rados_api_pool.log 2024-03-21T13:28:59.594 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T13:28:59.595 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T13:28:59.595 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T13:28:59.595 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.595 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T13:28:59.595 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.595 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T13:28:59.595 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T13:28:59.595 INFO:tasks.workunit.client.0.smithi097.stderr:+++ return 2024-03-21T13:28:59.595 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.595 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T13:28:59.595 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.595 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T13:28:59.595 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export S_COLORS=auto 2024-03-21T13:28:59.595 INFO:tasks.workunit.client.0.smithi097.stderr:+++ S_COLORS=auto 2024-03-21T13:28:59.595 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.596 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T13:28:59.596 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.596 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T13:28:59.596 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.596 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T13:28:59.596 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T13:28:59.596 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T13:28:59.596 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.596 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T13:28:59.596 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.596 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T13:28:59.596 INFO:tasks.workunit.client.0.smithi097.stderr:+ sed 's/^/ api_pool: /' 2024-03-21T13:28:59.596 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T13:28:59.596 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.596 INFO:tasks.workunit.client.0.smithi097.stderr:++ echo api_cmd 2024-03-21T13:28:59.596 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T13:28:59.597 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T13:28:59.597 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T13:28:59.597 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.597 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T13:28:59.597 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.597 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T13:28:59.597 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T13:28:59.597 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.597 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/locale 2024-03-21T13:28:59.597 INFO:tasks.workunit.client.0.smithi097.stderr:++ awk '{print $1}' 2024-03-21T13:28:59.597 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T13:28:59.597 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T13:28:59.597 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T13:28:59.597 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.597 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T13:28:59.598 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.598 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T13:28:59.598 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T13:28:59.598 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.598 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T13:28:59.598 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T13:28:59.598 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T13:28:59.598 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.598 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T13:28:59.598 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.598 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T13:28:59.598 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T13:28:59.598 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.599 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T13:28:59.599 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.599 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T13:28:59.599 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.599 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T13:28:59.599 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.599 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T13:28:59.599 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup 2024-03-21T13:28:59.599 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.599 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T13:28:59.599 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.599 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T13:28:59.599 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T13:28:59.599 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T13:28:59.599 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/locale 2024-03-21T13:28:59.600 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T13:28:59.600 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T13:28:59.600 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T13:28:59.600 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.600 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T13:28:59.600 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.600 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T13:28:59.600 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T13:28:59.600 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.600 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T13:28:59.600 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.600 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T13:28:59.600 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.600 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T13:28:59.600 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.601 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T13:28:59.601 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.601 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup 2024-03-21T13:28:59.601 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T13:28:59.601 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.601 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T13:28:59.601 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T13:28:59.601 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/locale 2024-03-21T13:28:59.601 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.601 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T13:28:59.601 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T13:28:59.601 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.601 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T13:28:59.601 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T13:28:59.601 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T13:28:59.602 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T13:28:59.602 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T13:28:59.602 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T13:28:59.602 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T13:28:59.602 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T13:28:59.602 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.602 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T13:28:59.602 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.602 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.602 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T13:28:59.602 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T13:28:59.602 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.602 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T13:28:59.602 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_api_misc_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc_pp.xml 2024-03-21T13:28:59.602 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T13:28:59.603 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.603 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T13:28:59.603 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T13:28:59.603 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.603 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.603 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T13:28:59.603 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T13:28:59.603 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.603 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T13:28:59.603 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.603 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T13:28:59.603 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.603 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.603 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T13:28:59.603 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T13:28:59.604 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.604 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T13:28:59.604 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup 2024-03-21T13:28:59.604 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup 2024-03-21T13:28:59.604 INFO:tasks.workunit.client.0.smithi097.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T13:28:59.604 INFO:tasks.workunit.client.0.smithi097.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T13:28:59.604 INFO:tasks.workunit.client.0.smithi097.stderr:++++ LANG=en_US.UTF-8 2024-03-21T13:28:59.604 INFO:tasks.workunit.client.0.smithi097.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T13:28:59.604 INFO:tasks.workunit.client.0.smithi097.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T13:28:59.604 INFO:tasks.workunit.client.0.smithi097.stderr:++++ LANG=en_US.UTF-8 2024-03-21T13:28:59.604 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.604 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/locale 2024-03-21T13:28:59.604 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T13:28:59.604 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.604 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.605 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T13:28:59.605 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG=en_US.UTF-8 2024-03-21T13:28:59.605 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup config 2024-03-21T13:28:59.605 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.605 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG=en_US.UTF-8 2024-03-21T13:28:59.605 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup config 2024-03-21T13:28:59.605 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.605 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.605 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.605 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' dumb = linux ']' 2024-03-21T13:28:59.605 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.605 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T13:28:59.605 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.605 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.605 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' dumb = linux ']' 2024-03-21T13:28:59.606 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/less.sh 2024-03-21T13:28:59.606 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.606 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T13:28:59.606 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.606 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T13:28:59.606 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/less.sh 2024-03-21T13:28:59.606 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.606 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T13:28:59.606 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.606 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T13:28:59.606 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T13:28:59.606 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.606 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T13:28:59.606 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.606 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T13:28:59.607 INFO:tasks.workunit.client.0.smithi097.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T13:28:59.607 INFO:tasks.workunit.client.0.smithi097.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T13:28:59.607 INFO:tasks.workunit.client.0.smithi097.stderr:+++++ readlink /proc/79746/exe 2024-03-21T13:28:59.607 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.607 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T13:28:59.607 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.607 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T13:28:59.607 INFO:tasks.workunit.client.0.smithi097.stderr:+++++ readlink /proc/79767/exe 2024-03-21T13:28:59.607 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T13:28:59.607 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T13:28:59.607 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/locale 2024-03-21T13:28:59.607 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.607 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T13:28:59.607 INFO:tasks.workunit.client.0.smithi097.stderr:+ tee ceph_test_rados_api_misc_pp.log 2024-03-21T13:28:59.607 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.608 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T13:28:59.608 INFO:tasks.workunit.client.0.smithi097.stderr:++++ basename /usr/bin/bash 2024-03-21T13:28:59.608 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T13:28:59.608 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T13:28:59.608 INFO:tasks.workunit.client.0.smithi097.stderr:+ sed 's/^/ api_misc_pp: /' 2024-03-21T13:28:59.608 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_declare='declare -f' 2024-03-21T13:28:59.608 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_opt=-f 2024-03-21T13:28:59.608 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export which_declare 2024-03-21T13:28:59.608 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export -f which 2024-03-21T13:28:59.608 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset i 2024-03-21T13:28:59.608 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset -f pathmunge 2024-03-21T13:28:59.608 INFO:tasks.workunit.client.0.smithi097.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T13:28:59.608 INFO:tasks.workunit.client.0.smithi097.stderr:+ export PATH 2024-03-21T13:28:59.608 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T13:28:59.608 INFO:tasks.workunit.client.0.smithi097.stderr:+ unset rc 2024-03-21T13:28:59.608 INFO:tasks.workunit.client.0.smithi097.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T13:28:59.608 INFO:tasks.workunit.client.0.smithi097.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T13:28:59.609 INFO:tasks.workunit.client.0.smithi097.stderr:++++ LANG=en_US.UTF-8 2024-03-21T13:28:59.609 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.609 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T13:28:59.609 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.609 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG=en_US.UTF-8 2024-03-21T13:28:59.609 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup config 2024-03-21T13:28:59.609 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.609 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.609 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' dumb = linux ']' 2024-03-21T13:28:59.609 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.609 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T13:28:59.609 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.609 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/less.sh 2024-03-21T13:28:59.609 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T13:28:59.610 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.610 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T13:28:59.610 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.610 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T13:28:59.610 INFO:tasks.workunit.client.0.smithi097.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T13:28:59.610 INFO:tasks.workunit.client.0.smithi097.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T13:28:59.610 INFO:tasks.workunit.client.0.smithi097.stderr:++++ basename /usr/bin/bash 2024-03-21T13:28:59.610 INFO:tasks.workunit.client.0.smithi097.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T13:28:59.610 INFO:tasks.workunit.client.0.smithi097.stderr:++++ LANG=en_US.UTF-8 2024-03-21T13:28:59.610 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.610 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T13:28:59.610 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.610 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG=en_US.UTF-8 2024-03-21T13:28:59.610 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup config 2024-03-21T13:28:59.610 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.611 INFO:tasks.workunit.client.0.smithi097.stdout:test api_cmd on pid 79997 2024-03-21T13:28:59.619 INFO:tasks.workunit.client.0.smithi097.stderr:+ ff=api_cmd 2024-03-21T13:28:59.619 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.619 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' dumb = linux ']' 2024-03-21T13:28:59.619 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.619 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T13:28:59.619 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.619 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/less.sh 2024-03-21T13:28:59.619 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=79997 2024-03-21T13:28:59.619 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T13:28:59.619 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.619 INFO:tasks.workunit.client.0.smithi097.stderr:+ echo 'test api_cmd on pid 79997' 2024-03-21T13:28:59.619 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T13:28:59.619 INFO:tasks.workunit.client.0.smithi097.stderr:+ pids[$f]=79997 2024-03-21T13:28:59.619 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.620 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T13:28:59.620 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T13:28:59.620 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T13:28:59.620 INFO:tasks.workunit.client.0.smithi097.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T13:28:59.620 INFO:tasks.workunit.client.0.smithi097.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd.xml 2>&1 | tee ceph_test_rados_api_cmd.log | sed "s/^/ api_cmd: /"' 2024-03-21T13:28:59.620 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_declare='declare -f' 2024-03-21T13:28:59.620 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_opt=-f 2024-03-21T13:28:59.620 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export which_declare 2024-03-21T13:28:59.620 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export -f which 2024-03-21T13:28:59.620 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset i 2024-03-21T13:28:59.620 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset -f pathmunge 2024-03-21T13:28:59.620 INFO:tasks.workunit.client.0.smithi097.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T13:28:59.620 INFO:tasks.workunit.client.0.smithi097.stderr:+ export PATH 2024-03-21T13:28:59.621 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T13:28:59.621 INFO:tasks.workunit.client.0.smithi097.stderr:+ unset rc 2024-03-21T13:28:59.621 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T13:28:59.621 INFO:tasks.workunit.client.0.smithi097.stderr:+ sed 's/^/ api_snapshots: /' 2024-03-21T13:28:59.621 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T13:28:59.621 INFO:tasks.workunit.client.0.smithi097.stderr:+ . /etc/bashrc 2024-03-21T13:28:59.621 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -z '' ']' 2024-03-21T13:28:59.621 INFO:tasks.workunit.client.0.smithi097.stderr:++ BASHRCSOURCED=Y 2024-03-21T13:28:59.621 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.621 INFO:tasks.workunit.client.0.smithi097.stderr:++ shopt -q login_shell 2024-03-21T13:28:59.621 INFO:tasks.workunit.client.0.smithi097.stderr:+++++ readlink /proc/79889/exe 2024-03-21T13:28:59.621 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_api_snapshots_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots_pp.xml 2024-03-21T13:28:59.621 INFO:tasks.workunit.client.0.smithi097.stderr:+ tee ceph_test_rados_api_snapshots.log 2024-03-21T13:28:59.621 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_api_snapshots --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots.xml 2024-03-21T13:28:59.621 INFO:tasks.workunit.client.0.smithi097.stderr:+ tee ceph_test_rados_api_snapshots_pp.log 2024-03-21T13:28:59.621 INFO:tasks.workunit.client.0.smithi097.stderr:+++++ readlink /proc/79819/exe 2024-03-21T13:28:59.621 INFO:tasks.workunit.client.0.smithi097.stderr:++++ basename /usr/bin/bash 2024-03-21T13:28:59.622 INFO:tasks.workunit.client.0.smithi097.stderr:++ printf %25s api_cmd_pp 2024-03-21T13:28:59.622 INFO:tasks.workunit.client.0.smithi097.stderr:++++ basename /usr/bin/bash 2024-03-21T13:28:59.622 INFO:tasks.workunit.client.0.smithi097.stderr:+ r=' api_cmd_pp' 2024-03-21T13:28:59.622 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_declare='declare -f' 2024-03-21T13:28:59.622 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_opt=-f 2024-03-21T13:28:59.622 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export which_declare 2024-03-21T13:28:59.622 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export -f which 2024-03-21T13:28:59.622 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset i 2024-03-21T13:28:59.622 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset -f pathmunge 2024-03-21T13:28:59.622 INFO:tasks.workunit.client.0.smithi097.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T13:28:59.622 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_declare='declare -f' 2024-03-21T13:28:59.622 INFO:tasks.workunit.client.0.smithi097.stderr:+ export PATH 2024-03-21T13:28:59.622 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_opt=-f 2024-03-21T13:28:59.622 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T13:28:59.623 INFO:tasks.workunit.client.0.smithi097.stderr:+ unset rc 2024-03-21T13:28:59.623 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export which_declare 2024-03-21T13:28:59.623 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export -f which 2024-03-21T13:28:59.623 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset i 2024-03-21T13:28:59.623 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset -f pathmunge 2024-03-21T13:28:59.623 INFO:tasks.workunit.client.0.smithi097.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T13:28:59.623 INFO:tasks.workunit.client.0.smithi097.stderr:+ export PATH 2024-03-21T13:28:59.623 INFO:tasks.workunit.client.0.smithi097.stderr:+ sed 's/^/ api_snapshots_pp: /' 2024-03-21T13:28:59.623 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T13:28:59.623 INFO:tasks.workunit.client.0.smithi097.stderr:+ unset rc 2024-03-21T13:28:59.623 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.623 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T13:28:59.623 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.623 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T13:28:59.624 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T13:28:59.624 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T13:28:59.624 INFO:tasks.workunit.client.0.smithi097.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T13:28:59.624 INFO:tasks.workunit.client.0.smithi097.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T13:28:59.624 INFO:tasks.workunit.client.0.smithi097.stderr:++++ LANG=en_US.UTF-8 2024-03-21T13:28:59.624 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.624 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T13:28:59.624 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.624 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG=en_US.UTF-8 2024-03-21T13:28:59.624 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup config 2024-03-21T13:28:59.624 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.624 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.624 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' dumb = linux ']' 2024-03-21T13:28:59.624 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.624 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T13:28:59.624 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.625 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/less.sh 2024-03-21T13:28:59.625 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T13:28:59.625 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.625 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T13:28:59.625 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.625 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T13:28:59.625 INFO:tasks.workunit.client.0.smithi097.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T13:28:59.625 INFO:tasks.workunit.client.0.smithi097.stderr:+ tee ceph_test_rados_api_stat.log 2024-03-21T13:28:59.625 INFO:tasks.workunit.client.0.smithi097.stderr:+ sed 's/^/ api_stat: /' 2024-03-21T13:28:59.625 INFO:tasks.workunit.client.0.smithi097.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T13:28:59.625 INFO:tasks.workunit.client.0.smithi097.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T13:28:59.625 INFO:tasks.workunit.client.0.smithi097.stderr:++++ LANG=en_US.UTF-8 2024-03-21T13:28:59.625 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.625 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T13:28:59.625 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.625 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG=en_US.UTF-8 2024-03-21T13:28:59.626 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup config 2024-03-21T13:28:59.626 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.626 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.626 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' dumb = linux ']' 2024-03-21T13:28:59.626 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.626 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T13:28:59.626 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.626 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/less.sh 2024-03-21T13:28:59.626 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T13:28:59.626 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.626 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T13:28:59.626 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.626 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T13:28:59.626 INFO:tasks.workunit.client.0.smithi097.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T13:28:59.626 INFO:tasks.workunit.client.0.smithi097.stderr:+++++ readlink /proc/79919/exe 2024-03-21T13:28:59.627 INFO:tasks.workunit.client.0.smithi097.stderr:++++ basename /usr/bin/bash 2024-03-21T13:28:59.627 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_declare='declare -f' 2024-03-21T13:28:59.627 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_opt=-f 2024-03-21T13:28:59.627 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export which_declare 2024-03-21T13:28:59.627 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export -f which 2024-03-21T13:28:59.627 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset i 2024-03-21T13:28:59.627 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset -f pathmunge 2024-03-21T13:28:59.627 INFO:tasks.workunit.client.0.smithi097.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T13:28:59.627 INFO:tasks.workunit.client.0.smithi097.stderr:+ sed 's/^/ api_watch_notify: /' 2024-03-21T13:28:59.627 INFO:tasks.workunit.client.0.smithi097.stderr:+ export PATH 2024-03-21T13:28:59.627 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T13:28:59.627 INFO:tasks.workunit.client.0.smithi097.stderr:+ unset rc 2024-03-21T13:28:59.627 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_api_stat --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat.xml 2024-03-21T13:28:59.627 INFO:tasks.workunit.client.0.smithi097.stderr:+ sed 's/^/ api_watch_notify_pp: /' 2024-03-21T13:28:59.627 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_api_watch_notify --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify.xml 2024-03-21T13:28:59.628 INFO:tasks.workunit.client.0.smithi097.stderr:+ tee ceph_test_rados_api_watch_notify_pp.log 2024-03-21T13:28:59.628 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_api_watch_notify_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify_pp.xml 2024-03-21T13:28:59.628 INFO:tasks.workunit.client.0.smithi097.stderr:+ tee ceph_test_rados_api_watch_notify.log 2024-03-21T13:28:59.628 INFO:tasks.workunit.client.0.smithi097.stderr:+++ umask 2024-03-21T13:28:59.628 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T13:28:59.628 INFO:tasks.workunit.client.0.smithi097.stderr:++ SHELL=/bin/bash 2024-03-21T13:28:59.628 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.628 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T13:28:59.628 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.628 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T13:28:59.628 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T13:28:59.628 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.628 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T13:28:59.628 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.628 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T13:28:59.629 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.629 INFO:tasks.workunit.client.0.smithi097.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.629 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.629 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T13:28:59.629 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.629 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T13:28:59.629 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.629 INFO:tasks.workunit.client.0.smithi097.stderr:++ echo api_cmd_pp 2024-03-21T13:28:59.629 INFO:tasks.workunit.client.0.smithi097.stderr:+++++ readlink /proc/79856/exe 2024-03-21T13:28:59.629 INFO:tasks.workunit.client.0.smithi097.stderr:++ awk '{print $1}' 2024-03-21T13:28:59.629 INFO:tasks.workunit.client.0.smithi097.stderr:++++ basename /usr/bin/bash 2024-03-21T13:28:59.629 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_declare='declare -f' 2024-03-21T13:28:59.629 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_opt=-f 2024-03-21T13:28:59.629 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export which_declare 2024-03-21T13:28:59.629 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export -f which 2024-03-21T13:28:59.630 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset i 2024-03-21T13:28:59.630 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset -f pathmunge 2024-03-21T13:28:59.630 INFO:tasks.workunit.client.0.smithi097.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T13:28:59.630 INFO:tasks.workunit.client.0.smithi097.stderr:+ export PATH 2024-03-21T13:28:59.630 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T13:28:59.630 INFO:tasks.workunit.client.0.smithi097.stderr:+ unset rc 2024-03-21T13:28:59.630 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T13:28:59.630 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T13:28:59.630 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T13:28:59.630 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.630 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T13:28:59.630 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.630 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T13:28:59.630 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T13:28:59.630 INFO:tasks.workunit.client.0.smithi097.stderr:+++ return 2024-03-21T13:28:59.630 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.631 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T13:28:59.631 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.631 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T13:28:59.631 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export S_COLORS=auto 2024-03-21T13:28:59.631 INFO:tasks.workunit.client.0.smithi097.stderr:+++ S_COLORS=auto 2024-03-21T13:28:59.631 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.631 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T13:28:59.631 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.631 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T13:28:59.631 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.631 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_api_stat_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat_pp.xml 2024-03-21T13:28:59.631 INFO:tasks.workunit.client.0.smithi097.stdout:test api_cmd_pp on pid 80043 2024-03-21T13:28:59.634 INFO:tasks.workunit.client.0.smithi097.stderr:+ ff=api_cmd_pp 2024-03-21T13:28:59.634 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=80043 2024-03-21T13:28:59.634 INFO:tasks.workunit.client.0.smithi097.stderr:+ echo 'test api_cmd_pp on pid 80043' 2024-03-21T13:28:59.634 INFO:tasks.workunit.client.0.smithi097.stderr:+ pids[$f]=80043 2024-03-21T13:28:59.634 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T13:28:59.634 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T13:28:59.634 INFO:tasks.workunit.client.0.smithi097.stderr:+ tee ceph_test_rados_api_stat_pp.log 2024-03-21T13:28:59.635 INFO:tasks.workunit.client.0.smithi097.stderr:++ printf %25s api_service 2024-03-21T13:28:59.635 INFO:tasks.workunit.client.0.smithi097.stderr:+ r=' api_service' 2024-03-21T13:28:59.635 INFO:tasks.workunit.client.0.smithi097.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd_pp.xml 2>&1 | tee ceph_test_rados_api_cmd_pp.log | sed "s/^/ api_cmd_pp: /"' 2024-03-21T13:28:59.635 INFO:tasks.workunit.client.0.smithi097.stderr:++ echo api_service 2024-03-21T13:28:59.635 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T13:28:59.635 INFO:tasks.workunit.client.0.smithi097.stderr:+ . /etc/bashrc 2024-03-21T13:28:59.635 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -z '' ']' 2024-03-21T13:28:59.635 INFO:tasks.workunit.client.0.smithi097.stderr:+ sed 's/^/ api_stat_pp: /' 2024-03-21T13:28:59.635 INFO:tasks.workunit.client.0.smithi097.stderr:++ BASHRCSOURCED=Y 2024-03-21T13:28:59.635 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.635 INFO:tasks.workunit.client.0.smithi097.stderr:++ shopt -q login_shell 2024-03-21T13:28:59.635 INFO:tasks.workunit.client.0.smithi097.stderr:+++ umask 2024-03-21T13:28:59.635 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T13:28:59.635 INFO:tasks.workunit.client.0.smithi097.stderr:++ SHELL=/bin/bash 2024-03-21T13:28:59.636 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.636 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T13:28:59.636 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.636 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T13:28:59.636 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T13:28:59.636 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.636 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T13:28:59.636 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.636 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T13:28:59.636 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.636 INFO:tasks.workunit.client.0.smithi097.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.636 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.636 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T13:28:59.636 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.637 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T13:28:59.637 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.637 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T13:28:59.637 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T13:28:59.637 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T13:28:59.637 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.637 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T13:28:59.637 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.637 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T13:28:59.637 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T13:28:59.637 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.637 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T13:28:59.637 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T13:28:59.637 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T13:28:59.638 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.638 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T13:28:59.638 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.638 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T13:28:59.638 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T13:28:59.638 INFO:tasks.workunit.client.0.smithi097.stderr:+++ return 2024-03-21T13:28:59.638 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.638 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T13:28:59.638 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.638 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T13:28:59.638 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export S_COLORS=auto 2024-03-21T13:28:59.638 INFO:tasks.workunit.client.0.smithi097.stderr:+++ S_COLORS=auto 2024-03-21T13:28:59.638 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.638 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T13:28:59.639 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.639 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T13:28:59.639 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.639 INFO:tasks.workunit.client.0.smithi097.stderr:++ awk '{print $1}' 2024-03-21T13:28:59.639 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T13:28:59.639 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T13:28:59.639 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T13:28:59.639 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.639 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T13:28:59.639 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.639 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T13:28:59.639 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T13:28:59.639 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.640 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T13:28:59.640 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.640 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T13:28:59.640 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.640 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T13:28:59.640 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.640 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T13:28:59.640 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup 2024-03-21T13:28:59.640 INFO:tasks.workunit.client.0.smithi097.stderr:+ ff=api_service 2024-03-21T13:28:59.640 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=80102 2024-03-21T13:28:59.640 INFO:tasks.workunit.client.0.smithi097.stdout:test api_service on pid 80102 2024-03-21T13:28:59.645 INFO:tasks.workunit.client.0.smithi097.stderr:+ echo 'test api_service on pid 80102' 2024-03-21T13:28:59.645 INFO:tasks.workunit.client.0.smithi097.stderr:+ pids[$f]=80102 2024-03-21T13:28:59.646 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T13:28:59.646 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T13:28:59.646 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T13:28:59.646 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T13:28:59.646 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T13:28:59.646 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.646 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T13:28:59.646 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.646 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T13:28:59.646 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T13:28:59.646 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.646 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T13:28:59.646 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T13:28:59.646 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T13:28:59.647 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.647 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T13:28:59.647 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.647 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T13:28:59.647 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T13:28:59.647 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.647 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T13:28:59.647 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.647 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T13:28:59.647 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.647 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T13:28:59.647 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.647 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T13:28:59.647 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup 2024-03-21T13:28:59.648 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/locale 2024-03-21T13:28:59.648 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/locale 2024-03-21T13:28:59.648 INFO:tasks.workunit.client.0.smithi097.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service.xml 2>&1 | tee ceph_test_rados_api_service.log | sed "s/^/ api_service: /"' 2024-03-21T13:28:59.648 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T13:28:59.648 INFO:tasks.workunit.client.0.smithi097.stderr:+ . /etc/bashrc 2024-03-21T13:28:59.648 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -z '' ']' 2024-03-21T13:28:59.648 INFO:tasks.workunit.client.0.smithi097.stderr:++ BASHRCSOURCED=Y 2024-03-21T13:28:59.648 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.648 INFO:tasks.workunit.client.0.smithi097.stderr:++ shopt -q login_shell 2024-03-21T13:28:59.648 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.648 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T13:28:59.648 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.648 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T13:28:59.648 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T13:28:59.648 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.649 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T13:28:59.649 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.649 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T13:28:59.649 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T13:28:59.649 INFO:tasks.workunit.client.0.smithi097.stderr:++ printf %25s api_service_pp 2024-03-21T13:28:59.649 INFO:tasks.workunit.client.0.smithi097.stderr:+ r=' api_service_pp' 2024-03-21T13:28:59.649 INFO:tasks.workunit.client.0.smithi097.stderr:+++ umask 2024-03-21T13:28:59.649 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T13:28:59.649 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T13:28:59.649 INFO:tasks.workunit.client.0.smithi097.stderr:++ SHELL=/bin/bash 2024-03-21T13:28:59.649 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.649 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T13:28:59.649 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.649 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T13:28:59.650 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T13:28:59.650 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.650 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T13:28:59.650 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.650 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T13:28:59.650 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.650 INFO:tasks.workunit.client.0.smithi097.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.650 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.650 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T13:28:59.650 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.650 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T13:28:59.650 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.650 INFO:tasks.workunit.client.0.smithi097.stderr:++ awk '{print $1}' 2024-03-21T13:28:59.650 INFO:tasks.workunit.client.0.smithi097.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T13:28:59.651 INFO:tasks.workunit.client.0.smithi097.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T13:28:59.651 INFO:tasks.workunit.client.0.smithi097.stderr:++++ LANG=en_US.UTF-8 2024-03-21T13:28:59.651 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.651 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T13:28:59.651 INFO:tasks.workunit.client.0.smithi097.stderr:++ echo api_service_pp 2024-03-21T13:28:59.651 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.651 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG=en_US.UTF-8 2024-03-21T13:28:59.651 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup config 2024-03-21T13:28:59.651 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.651 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.651 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' dumb = linux ']' 2024-03-21T13:28:59.651 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.651 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T13:28:59.651 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.651 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/less.sh 2024-03-21T13:28:59.652 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T13:28:59.652 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.652 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T13:28:59.652 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.652 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T13:28:59.652 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T13:28:59.652 INFO:tasks.workunit.client.0.smithi097.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T13:28:59.652 INFO:tasks.workunit.client.0.smithi097.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T13:28:59.652 INFO:tasks.workunit.client.0.smithi097.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T13:28:59.652 INFO:tasks.workunit.client.0.smithi097.stderr:++++ LANG=en_US.UTF-8 2024-03-21T13:28:59.652 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.652 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T13:28:59.652 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.652 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG=en_US.UTF-8 2024-03-21T13:28:59.652 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup config 2024-03-21T13:28:59.652 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.653 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.653 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' dumb = linux ']' 2024-03-21T13:28:59.653 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.653 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T13:28:59.653 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.653 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/less.sh 2024-03-21T13:28:59.653 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T13:28:59.653 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.653 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T13:28:59.653 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.653 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T13:28:59.653 INFO:tasks.workunit.client.0.smithi097.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T13:28:59.654 INFO:tasks.workunit.client.0.smithi097.stdout:test api_service_pp on pid 80146 2024-03-21T13:28:59.656 INFO:tasks.workunit.client.0.smithi097.stderr:+ ff=api_service_pp 2024-03-21T13:28:59.656 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=80146 2024-03-21T13:28:59.656 INFO:tasks.workunit.client.0.smithi097.stderr:+ echo 'test api_service_pp on pid 80146' 2024-03-21T13:28:59.656 INFO:tasks.workunit.client.0.smithi097.stderr:+ pids[$f]=80146 2024-03-21T13:28:59.656 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T13:28:59.656 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T13:28:59.656 INFO:tasks.workunit.client.0.smithi097.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service_pp.xml 2>&1 | tee ceph_test_rados_api_service_pp.log | sed "s/^/ api_service_pp: /"' 2024-03-21T13:28:59.657 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T13:28:59.657 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T13:28:59.657 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T13:28:59.657 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.657 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T13:28:59.657 INFO:tasks.workunit.client.0.smithi097.stderr:+++++ readlink /proc/80043/exe 2024-03-21T13:28:59.657 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.657 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T13:28:59.657 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T13:28:59.657 INFO:tasks.workunit.client.0.smithi097.stderr:+++ return 2024-03-21T13:28:59.657 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.657 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T13:28:59.657 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.657 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T13:28:59.657 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export S_COLORS=auto 2024-03-21T13:28:59.658 INFO:tasks.workunit.client.0.smithi097.stderr:+++ S_COLORS=auto 2024-03-21T13:28:59.658 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.658 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T13:28:59.658 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.658 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T13:28:59.658 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.658 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T13:28:59.658 INFO:tasks.workunit.client.0.smithi097.stderr:+ . /etc/bashrc 2024-03-21T13:28:59.658 INFO:tasks.workunit.client.0.smithi097.stderr:++ printf %25s api_c_write_operations 2024-03-21T13:28:59.658 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -z '' ']' 2024-03-21T13:28:59.658 INFO:tasks.workunit.client.0.smithi097.stderr:++ BASHRCSOURCED=Y 2024-03-21T13:28:59.658 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.658 INFO:tasks.workunit.client.0.smithi097.stderr:++ shopt -q login_shell 2024-03-21T13:28:59.658 INFO:tasks.workunit.client.0.smithi097.stderr:+++ umask 2024-03-21T13:28:59.658 INFO:tasks.workunit.client.0.smithi097.stderr:++++ basename /usr/bin/bash 2024-03-21T13:28:59.659 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T13:28:59.659 INFO:tasks.workunit.client.0.smithi097.stderr:++ SHELL=/bin/bash 2024-03-21T13:28:59.659 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.659 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T13:28:59.659 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.659 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T13:28:59.659 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T13:28:59.659 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.659 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T13:28:59.659 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.659 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T13:28:59.659 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.659 INFO:tasks.workunit.client.0.smithi097.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.659 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.659 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T13:28:59.660 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.660 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T13:28:59.660 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.660 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_declare='declare -f' 2024-03-21T13:28:59.660 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_opt=-f 2024-03-21T13:28:59.660 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export which_declare 2024-03-21T13:28:59.660 INFO:tasks.workunit.client.0.smithi097.stderr:+ r=' api_c_write_operations' 2024-03-21T13:28:59.660 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export -f which 2024-03-21T13:28:59.660 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset i 2024-03-21T13:28:59.660 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset -f pathmunge 2024-03-21T13:28:59.660 INFO:tasks.workunit.client.0.smithi097.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T13:28:59.660 INFO:tasks.workunit.client.0.smithi097.stderr:+ export PATH 2024-03-21T13:28:59.660 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T13:28:59.660 INFO:tasks.workunit.client.0.smithi097.stderr:+ unset rc 2024-03-21T13:28:59.660 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_api_cmd_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd_pp.xml 2024-03-21T13:28:59.660 INFO:tasks.workunit.client.0.smithi097.stderr:+ tee ceph_test_rados_api_cmd_pp.log 2024-03-21T13:28:59.661 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T13:28:59.661 INFO:tasks.workunit.client.0.smithi097.stderr:+++++ readlink /proc/79997/exe 2024-03-21T13:28:59.661 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T13:28:59.661 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T13:28:59.661 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.661 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T13:28:59.661 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.661 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T13:28:59.661 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T13:28:59.661 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.661 INFO:tasks.workunit.client.0.smithi097.stderr:+ sed 's/^/ api_cmd_pp: /' 2024-03-21T13:28:59.661 INFO:tasks.workunit.client.0.smithi097.stderr:++++ basename /usr/bin/bash 2024-03-21T13:28:59.661 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_declare='declare -f' 2024-03-21T13:28:59.661 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_opt=-f 2024-03-21T13:28:59.661 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export which_declare 2024-03-21T13:28:59.662 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export -f which 2024-03-21T13:28:59.662 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset i 2024-03-21T13:28:59.662 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset -f pathmunge 2024-03-21T13:28:59.662 INFO:tasks.workunit.client.0.smithi097.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T13:28:59.662 INFO:tasks.workunit.client.0.smithi097.stderr:+ export PATH 2024-03-21T13:28:59.662 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T13:28:59.662 INFO:tasks.workunit.client.0.smithi097.stderr:+ unset rc 2024-03-21T13:28:59.662 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_api_cmd --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd.xml 2024-03-21T13:28:59.662 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T13:28:59.662 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T13:28:59.662 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T13:28:59.662 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.662 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T13:28:59.662 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.662 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T13:28:59.663 INFO:tasks.workunit.client.0.smithi097.stderr:++ echo api_c_write_operations 2024-03-21T13:28:59.663 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T13:28:59.663 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.663 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T13:28:59.663 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.663 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T13:28:59.663 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.663 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T13:28:59.663 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.663 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T13:28:59.663 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup 2024-03-21T13:28:59.663 INFO:tasks.workunit.client.0.smithi097.stderr:++ awk '{print $1}' 2024-03-21T13:28:59.663 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T13:28:59.663 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T13:28:59.664 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T13:28:59.664 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.664 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T13:28:59.664 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.664 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T13:28:59.664 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T13:28:59.664 INFO:tasks.workunit.client.0.smithi097.stderr:+++ return 2024-03-21T13:28:59.664 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.664 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T13:28:59.664 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.664 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T13:28:59.664 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export S_COLORS=auto 2024-03-21T13:28:59.664 INFO:tasks.workunit.client.0.smithi097.stderr:+++ S_COLORS=auto 2024-03-21T13:28:59.664 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.664 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T13:28:59.664 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.665 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T13:28:59.665 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.665 INFO:tasks.workunit.client.0.smithi097.stderr:+ sed 's/^/ api_cmd: /' 2024-03-21T13:28:59.665 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/locale 2024-03-21T13:28:59.665 INFO:tasks.workunit.client.0.smithi097.stdout:test api_c_write_operations on pid 80205 2024-03-21T13:28:59.667 INFO:tasks.workunit.client.0.smithi097.stderr:+ ff=api_c_write_operations 2024-03-21T13:28:59.667 INFO:tasks.workunit.client.0.smithi097.stderr:+ tee ceph_test_rados_api_cmd.log 2024-03-21T13:28:59.667 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=80205 2024-03-21T13:28:59.667 INFO:tasks.workunit.client.0.smithi097.stderr:+ echo 'test api_c_write_operations on pid 80205' 2024-03-21T13:28:59.667 INFO:tasks.workunit.client.0.smithi097.stderr:+ pids[$f]=80205 2024-03-21T13:28:59.667 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T13:28:59.667 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T13:28:59.667 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.667 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T13:28:59.667 INFO:tasks.workunit.client.0.smithi097.stderr:++ printf %25s api_c_read_operations 2024-03-21T13:28:59.667 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.668 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T13:28:59.668 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T13:28:59.668 INFO:tasks.workunit.client.0.smithi097.stderr:+ r=' api_c_read_operations' 2024-03-21T13:28:59.668 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T13:28:59.668 INFO:tasks.workunit.client.0.smithi097.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_write_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_write_operations.xml 2>&1 | tee ceph_test_rados_api_c_write_operations.log | sed "s/^/ api_c_write_operations: /"' 2024-03-21T13:28:59.668 INFO:tasks.workunit.client.0.smithi097.stderr:++ echo api_c_read_operations 2024-03-21T13:28:59.668 INFO:tasks.workunit.client.0.smithi097.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T13:28:59.668 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T13:28:59.668 INFO:tasks.workunit.client.0.smithi097.stderr:+ . /etc/bashrc 2024-03-21T13:28:59.668 INFO:tasks.workunit.client.0.smithi097.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T13:28:59.668 INFO:tasks.workunit.client.0.smithi097.stderr:++++ LANG=en_US.UTF-8 2024-03-21T13:28:59.668 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.668 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T13:28:59.668 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.669 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG=en_US.UTF-8 2024-03-21T13:28:59.669 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup config 2024-03-21T13:28:59.669 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -z '' ']' 2024-03-21T13:28:59.669 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.669 INFO:tasks.workunit.client.0.smithi097.stderr:++ BASHRCSOURCED=Y 2024-03-21T13:28:59.669 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.669 INFO:tasks.workunit.client.0.smithi097.stderr:++ shopt -q login_shell 2024-03-21T13:28:59.669 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.669 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' dumb = linux ']' 2024-03-21T13:28:59.669 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.669 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T13:28:59.669 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.669 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/less.sh 2024-03-21T13:28:59.669 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T13:28:59.669 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.670 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T13:28:59.670 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.670 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T13:28:59.670 INFO:tasks.workunit.client.0.smithi097.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T13:28:59.670 INFO:tasks.workunit.client.0.smithi097.stderr:+++ umask 2024-03-21T13:28:59.670 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T13:28:59.670 INFO:tasks.workunit.client.0.smithi097.stderr:++ SHELL=/bin/bash 2024-03-21T13:28:59.670 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.670 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T13:28:59.670 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.670 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T13:28:59.670 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T13:28:59.670 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.670 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T13:28:59.671 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.671 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T13:28:59.671 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.671 INFO:tasks.workunit.client.0.smithi097.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.671 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.671 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T13:28:59.671 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.671 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T13:28:59.671 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.671 INFO:tasks.workunit.client.0.smithi097.stderr:+++++ readlink /proc/80102/exe 2024-03-21T13:28:59.671 INFO:tasks.workunit.client.0.smithi097.stderr:++++ basename /usr/bin/bash 2024-03-21T13:28:59.671 INFO:tasks.workunit.client.0.smithi097.stderr:++ awk '{print $1}' 2024-03-21T13:28:59.671 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_declare='declare -f' 2024-03-21T13:28:59.671 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_opt=-f 2024-03-21T13:28:59.671 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export which_declare 2024-03-21T13:28:59.672 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export -f which 2024-03-21T13:28:59.672 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T13:28:59.672 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset i 2024-03-21T13:28:59.672 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T13:28:59.672 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset -f pathmunge 2024-03-21T13:28:59.672 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T13:28:59.672 INFO:tasks.workunit.client.0.smithi097.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T13:28:59.672 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.672 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T13:28:59.672 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.672 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T13:28:59.672 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T13:28:59.672 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.672 INFO:tasks.workunit.client.0.smithi097.stderr:+ export PATH 2024-03-21T13:28:59.672 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T13:28:59.673 INFO:tasks.workunit.client.0.smithi097.stderr:+ unset rc 2024-03-21T13:28:59.673 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_api_service --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service.xml 2024-03-21T13:28:59.673 INFO:tasks.workunit.client.0.smithi097.stderr:+ tee ceph_test_rados_api_service.log 2024-03-21T13:28:59.673 INFO:tasks.workunit.client.0.smithi097.stderr:+ sed 's/^/ api_service: /' 2024-03-21T13:28:59.673 INFO:tasks.workunit.client.0.smithi097.stderr:+ ff=api_c_read_operations 2024-03-21T13:28:59.673 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=80242 2024-03-21T13:28:59.673 INFO:tasks.workunit.client.0.smithi097.stdout:test api_c_read_operations on pid 80242 2024-03-21T13:28:59.675 INFO:tasks.workunit.client.0.smithi097.stderr:+ echo 'test api_c_read_operations on pid 80242' 2024-03-21T13:28:59.675 INFO:tasks.workunit.client.0.smithi097.stderr:+ pids[$f]=80242 2024-03-21T13:28:59.675 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T13:28:59.675 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T13:28:59.676 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T13:28:59.676 INFO:tasks.workunit.client.0.smithi097.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_read_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_read_operations.xml 2>&1 | tee ceph_test_rados_api_c_read_operations.log | sed "s/^/ api_c_read_operations: /"' 2024-03-21T13:28:59.676 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T13:28:59.676 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T13:28:59.676 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.676 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T13:28:59.676 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.676 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T13:28:59.676 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T13:28:59.676 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.676 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T13:28:59.676 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.676 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T13:28:59.676 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.677 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T13:28:59.677 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.677 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T13:28:59.677 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup 2024-03-21T13:28:59.677 INFO:tasks.workunit.client.0.smithi097.stderr:++ printf %25s api_cls_remote_reads 2024-03-21T13:28:59.677 INFO:tasks.workunit.client.0.smithi097.stderr:+ r=' api_cls_remote_reads' 2024-03-21T13:28:59.677 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/locale 2024-03-21T13:28:59.677 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T13:28:59.677 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T13:28:59.677 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T13:28:59.677 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T13:28:59.677 INFO:tasks.workunit.client.0.smithi097.stderr:+ . /etc/bashrc 2024-03-21T13:28:59.677 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.677 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T13:28:59.677 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.678 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T13:28:59.678 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T13:28:59.678 INFO:tasks.workunit.client.0.smithi097.stderr:+++ return 2024-03-21T13:28:59.678 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.678 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T13:28:59.678 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.678 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T13:28:59.678 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -z '' ']' 2024-03-21T13:28:59.678 INFO:tasks.workunit.client.0.smithi097.stderr:++ BASHRCSOURCED=Y 2024-03-21T13:28:59.678 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.678 INFO:tasks.workunit.client.0.smithi097.stderr:++ shopt -q login_shell 2024-03-21T13:28:59.678 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export S_COLORS=auto 2024-03-21T13:28:59.678 INFO:tasks.workunit.client.0.smithi097.stderr:+++ S_COLORS=auto 2024-03-21T13:28:59.678 INFO:tasks.workunit.client.0.smithi097.stderr:++ echo api_cls_remote_reads 2024-03-21T13:28:59.678 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.679 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T13:28:59.679 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.697 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T13:28:59.697 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.697 INFO:tasks.workunit.client.0.smithi097.stderr:++ awk '{print $1}' 2024-03-21T13:28:59.698 INFO:tasks.workunit.client.0.smithi097.stdout:test api_cls_remote_reads on pid 80253 2024-03-21T13:28:59.700 INFO:tasks.workunit.client.0.smithi097.stderr:+ ff=api_cls_remote_reads 2024-03-21T13:28:59.700 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=80253 2024-03-21T13:28:59.701 INFO:tasks.workunit.client.0.smithi097.stderr:+ echo 'test api_cls_remote_reads on pid 80253' 2024-03-21T13:28:59.701 INFO:tasks.workunit.client.0.smithi097.stderr:+ pids[$f]=80253 2024-03-21T13:28:59.701 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T13:28:59.701 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T13:28:59.701 INFO:tasks.workunit.client.0.smithi097.stderr:+++ umask 2024-03-21T13:28:59.701 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T13:28:59.701 INFO:tasks.workunit.client.0.smithi097.stderr:++ SHELL=/bin/bash 2024-03-21T13:28:59.701 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T13:28:59.701 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.701 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T13:28:59.701 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T13:28:59.701 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.701 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T13:28:59.701 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T13:28:59.701 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.702 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T13:28:59.702 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.702 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T13:28:59.702 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T13:28:59.702 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T13:28:59.702 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.702 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.702 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T13:28:59.702 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.702 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T13:28:59.702 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.702 INFO:tasks.workunit.client.0.smithi097.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.702 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.702 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T13:28:59.702 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.703 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T13:28:59.703 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.703 INFO:tasks.workunit.client.0.smithi097.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cls_remote_reads --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cls_remote_reads.xml 2>&1 | tee ceph_test_rados_api_cls_remote_reads.log | sed "s/^/ api_cls_remote_reads: /"' 2024-03-21T13:28:59.703 INFO:tasks.workunit.client.0.smithi097.stderr:++ printf %25s list_parallel 2024-03-21T13:28:59.703 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.703 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T13:28:59.703 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.703 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T13:28:59.703 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T13:28:59.703 INFO:tasks.workunit.client.0.smithi097.stderr:+ r=' list_parallel' 2024-03-21T13:28:59.703 INFO:tasks.workunit.client.0.smithi097.stderr:++ awk '{print $1}' 2024-03-21T13:28:59.703 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T13:28:59.703 INFO:tasks.workunit.client.0.smithi097.stderr:+ . /etc/bashrc 2024-03-21T13:28:59.703 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -z '' ']' 2024-03-21T13:28:59.703 INFO:tasks.workunit.client.0.smithi097.stderr:++ BASHRCSOURCED=Y 2024-03-21T13:28:59.704 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.704 INFO:tasks.workunit.client.0.smithi097.stderr:++ shopt -q login_shell 2024-03-21T13:28:59.704 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T13:28:59.704 INFO:tasks.workunit.client.0.smithi097.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T13:28:59.704 INFO:tasks.workunit.client.0.smithi097.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T13:28:59.704 INFO:tasks.workunit.client.0.smithi097.stderr:++++ LANG=en_US.UTF-8 2024-03-21T13:28:59.704 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.704 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T13:28:59.704 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.704 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG=en_US.UTF-8 2024-03-21T13:28:59.704 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup config 2024-03-21T13:28:59.704 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.704 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.704 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' dumb = linux ']' 2024-03-21T13:28:59.704 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.705 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T13:28:59.705 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.705 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/less.sh 2024-03-21T13:28:59.705 INFO:tasks.workunit.client.0.smithi097.stderr:++ echo list_parallel 2024-03-21T13:28:59.705 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T13:28:59.705 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.705 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T13:28:59.705 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.705 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T13:28:59.705 INFO:tasks.workunit.client.0.smithi097.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T13:28:59.705 INFO:tasks.workunit.client.0.smithi097.stderr:+++ umask 2024-03-21T13:28:59.705 INFO:tasks.workunit.client.0.smithi097.stderr:+ ff=list_parallel 2024-03-21T13:28:59.705 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=80272 2024-03-21T13:28:59.706 INFO:tasks.workunit.client.0.smithi097.stdout:test list_parallel on pid 80272 2024-03-21T13:28:59.709 INFO:tasks.workunit.client.0.smithi097.stderr:+ echo 'test list_parallel on pid 80272' 2024-03-21T13:28:59.709 INFO:tasks.workunit.client.0.smithi097.stderr:+ pids[$f]=80272 2024-03-21T13:28:59.709 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T13:28:59.709 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T13:28:59.709 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T13:28:59.709 INFO:tasks.workunit.client.0.smithi097.stderr:++ SHELL=/bin/bash 2024-03-21T13:28:59.709 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.709 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T13:28:59.709 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.709 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T13:28:59.709 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T13:28:59.709 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.710 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T13:28:59.710 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.710 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T13:28:59.710 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.710 INFO:tasks.workunit.client.0.smithi097.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.710 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.710 INFO:tasks.workunit.client.0.smithi097.stderr:+++++ readlink /proc/80146/exe 2024-03-21T13:28:59.710 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T13:28:59.710 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.710 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T13:28:59.710 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.710 INFO:tasks.workunit.client.0.smithi097.stderr:+ bash -o pipefail -exc 'ceph_test_rados_list_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/list_parallel.xml 2>&1 | tee ceph_test_rados_list_parallel.log | sed "s/^/ list_parallel: /"' 2024-03-21T13:28:59.710 INFO:tasks.workunit.client.0.smithi097.stderr:++++ basename /usr/bin/bash 2024-03-21T13:28:59.710 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_declare='declare -f' 2024-03-21T13:28:59.711 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_opt=-f 2024-03-21T13:28:59.711 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export which_declare 2024-03-21T13:28:59.711 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export -f which 2024-03-21T13:28:59.711 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset i 2024-03-21T13:28:59.711 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset -f pathmunge 2024-03-21T13:28:59.711 INFO:tasks.workunit.client.0.smithi097.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T13:28:59.711 INFO:tasks.workunit.client.0.smithi097.stderr:+ export PATH 2024-03-21T13:28:59.711 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T13:28:59.711 INFO:tasks.workunit.client.0.smithi097.stderr:+ unset rc 2024-03-21T13:28:59.711 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T13:28:59.711 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T13:28:59.711 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T13:28:59.711 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.711 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T13:28:59.712 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.712 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T13:28:59.712 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T13:28:59.712 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.712 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T13:28:59.712 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.712 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T13:28:59.712 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.712 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T13:28:59.712 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.712 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T13:28:59.712 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup 2024-03-21T13:28:59.712 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T13:28:59.712 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T13:28:59.712 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T13:28:59.713 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.713 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T13:28:59.713 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.713 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T13:28:59.713 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T13:28:59.713 INFO:tasks.workunit.client.0.smithi097.stderr:+++ return 2024-03-21T13:28:59.713 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T13:28:59.713 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.713 INFO:tasks.workunit.client.0.smithi097.stderr:+ . /etc/bashrc 2024-03-21T13:28:59.713 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T13:28:59.713 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.713 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T13:28:59.713 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export S_COLORS=auto 2024-03-21T13:28:59.713 INFO:tasks.workunit.client.0.smithi097.stderr:+++ S_COLORS=auto 2024-03-21T13:28:59.713 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.714 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T13:28:59.714 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.714 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T13:28:59.714 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.714 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -z '' ']' 2024-03-21T13:28:59.714 INFO:tasks.workunit.client.0.smithi097.stderr:++ BASHRCSOURCED=Y 2024-03-21T13:28:59.714 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.714 INFO:tasks.workunit.client.0.smithi097.stderr:++ shopt -q login_shell 2024-03-21T13:28:59.714 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T13:28:59.714 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_api_service_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service_pp.xml 2024-03-21T13:28:59.714 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T13:28:59.714 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T13:28:59.714 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.715 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T13:28:59.715 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/locale 2024-03-21T13:28:59.715 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.715 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T13:28:59.715 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T13:28:59.715 INFO:tasks.workunit.client.0.smithi097.stderr:+++ return 2024-03-21T13:28:59.715 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.715 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T13:28:59.715 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.715 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T13:28:59.715 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export S_COLORS=auto 2024-03-21T13:28:59.715 INFO:tasks.workunit.client.0.smithi097.stderr:+++ S_COLORS=auto 2024-03-21T13:28:59.715 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.715 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T13:28:59.715 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.716 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T13:28:59.716 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.716 INFO:tasks.workunit.client.0.smithi097.stderr:+ sed 's/^/ api_service_pp: /' 2024-03-21T13:28:59.716 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.716 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T13:28:59.716 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.716 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T13:28:59.716 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T13:28:59.716 INFO:tasks.workunit.client.0.smithi097.stderr:+++ umask 2024-03-21T13:28:59.716 INFO:tasks.workunit.client.0.smithi097.stderr:+ tee ceph_test_rados_api_service_pp.log 2024-03-21T13:28:59.716 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T13:28:59.716 INFO:tasks.workunit.client.0.smithi097.stderr:++ SHELL=/bin/bash 2024-03-21T13:28:59.716 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.716 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T13:28:59.716 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.716 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T13:28:59.717 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T13:28:59.717 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.717 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T13:28:59.717 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.717 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T13:28:59.717 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.717 INFO:tasks.workunit.client.0.smithi097.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.717 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.717 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T13:28:59.717 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.717 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T13:28:59.717 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.717 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T13:28:59.717 INFO:tasks.workunit.client.0.smithi097.stderr:++ printf %25s open_pools_parallel 2024-03-21T13:28:59.717 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T13:28:59.718 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T13:28:59.718 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T13:28:59.718 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.718 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T13:28:59.718 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.718 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T13:28:59.718 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T13:28:59.718 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.718 INFO:tasks.workunit.client.0.smithi097.stderr:+ r=' open_pools_parallel' 2024-03-21T13:28:59.718 INFO:tasks.workunit.client.0.smithi097.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T13:28:59.718 INFO:tasks.workunit.client.0.smithi097.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T13:28:59.718 INFO:tasks.workunit.client.0.smithi097.stderr:++++ LANG=en_US.UTF-8 2024-03-21T13:28:59.718 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.718 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T13:28:59.718 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.719 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG=en_US.UTF-8 2024-03-21T13:28:59.719 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup config 2024-03-21T13:28:59.719 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.719 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.719 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T13:28:59.719 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' dumb = linux ']' 2024-03-21T13:28:59.719 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T13:28:59.719 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.719 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T13:28:59.719 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T13:28:59.719 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.719 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/less.sh 2024-03-21T13:28:59.719 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.719 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T13:28:59.719 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.720 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T13:28:59.720 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T13:28:59.720 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.720 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T13:28:59.720 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T13:28:59.720 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.720 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T13:28:59.720 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.720 INFO:tasks.workunit.client.0.smithi097.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T13:28:59.720 INFO:tasks.workunit.client.0.smithi097.stderr:++ awk '{print $1}' 2024-03-21T13:28:59.720 INFO:tasks.workunit.client.0.smithi097.stderr:+++++ readlink /proc/80205/exe 2024-03-21T13:28:59.720 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T13:28:59.720 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T13:28:59.720 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T13:28:59.721 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.721 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T13:28:59.721 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.721 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T13:28:59.721 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T13:28:59.721 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.721 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T13:28:59.721 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.721 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T13:28:59.721 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.721 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T13:28:59.721 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.721 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T13:28:59.721 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup 2024-03-21T13:28:59.721 INFO:tasks.workunit.client.0.smithi097.stderr:++++ basename /usr/bin/bash 2024-03-21T13:28:59.722 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T13:28:59.722 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T13:28:59.722 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T13:28:59.722 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.722 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T13:28:59.722 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/locale 2024-03-21T13:28:59.722 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.722 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T13:28:59.722 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T13:28:59.722 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.722 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T13:28:59.722 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.722 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T13:28:59.722 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.723 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T13:28:59.723 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.723 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T13:28:59.723 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup 2024-03-21T13:28:59.723 INFO:tasks.workunit.client.0.smithi097.stderr:++ echo open_pools_parallel 2024-03-21T13:28:59.723 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_declare='declare -f' 2024-03-21T13:28:59.723 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_opt=-f 2024-03-21T13:28:59.723 INFO:tasks.workunit.client.0.smithi097.stdout:test open_pools_parallel on pid 80310 2024-03-21T13:28:59.726 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export which_declare 2024-03-21T13:28:59.726 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export -f which 2024-03-21T13:28:59.726 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset i 2024-03-21T13:28:59.726 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset -f pathmunge 2024-03-21T13:28:59.726 INFO:tasks.workunit.client.0.smithi097.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T13:28:59.726 INFO:tasks.workunit.client.0.smithi097.stderr:+ export PATH 2024-03-21T13:28:59.726 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T13:28:59.726 INFO:tasks.workunit.client.0.smithi097.stderr:+ unset rc 2024-03-21T13:28:59.726 INFO:tasks.workunit.client.0.smithi097.stderr:+ ff=open_pools_parallel 2024-03-21T13:28:59.726 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=80310 2024-03-21T13:28:59.726 INFO:tasks.workunit.client.0.smithi097.stderr:+ echo 'test open_pools_parallel on pid 80310' 2024-03-21T13:28:59.726 INFO:tasks.workunit.client.0.smithi097.stderr:+ pids[$f]=80310 2024-03-21T13:28:59.727 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T13:28:59.727 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T13:28:59.727 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.727 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T13:28:59.727 INFO:tasks.workunit.client.0.smithi097.stderr:+ sed 's/^/ api_c_write_operations: /' 2024-03-21T13:28:59.727 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.727 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_api_c_write_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_write_operations.xml 2024-03-21T13:28:59.727 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T13:28:59.727 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T13:28:59.727 INFO:tasks.workunit.client.0.smithi097.stderr:++ printf %25s delete_pools_parallel 2024-03-21T13:28:59.727 INFO:tasks.workunit.client.0.smithi097.stderr:+ r=' delete_pools_parallel' 2024-03-21T13:28:59.727 INFO:tasks.workunit.client.0.smithi097.stderr:+ bash -o pipefail -exc 'ceph_test_rados_open_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/open_pools_parallel.xml 2>&1 | tee ceph_test_rados_open_pools_parallel.log | sed "s/^/ open_pools_parallel: /"' 2024-03-21T13:28:59.727 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T13:28:59.727 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T13:28:59.727 INFO:tasks.workunit.client.0.smithi097.stderr:+ . /etc/bashrc 2024-03-21T13:28:59.728 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -z '' ']' 2024-03-21T13:28:59.728 INFO:tasks.workunit.client.0.smithi097.stderr:++ BASHRCSOURCED=Y 2024-03-21T13:28:59.728 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.728 INFO:tasks.workunit.client.0.smithi097.stderr:++ shopt -q login_shell 2024-03-21T13:28:59.728 INFO:tasks.workunit.client.0.smithi097.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T13:28:59.728 INFO:tasks.workunit.client.0.smithi097.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T13:28:59.728 INFO:tasks.workunit.client.0.smithi097.stderr:++++ LANG=en_US.UTF-8 2024-03-21T13:28:59.728 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.728 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T13:28:59.728 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.728 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG=en_US.UTF-8 2024-03-21T13:28:59.728 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup config 2024-03-21T13:28:59.728 INFO:tasks.workunit.client.0.smithi097.stderr:+++ umask 2024-03-21T13:28:59.728 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.729 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.729 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' dumb = linux ']' 2024-03-21T13:28:59.729 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.729 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T13:28:59.729 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.729 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/less.sh 2024-03-21T13:28:59.729 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T13:28:59.729 INFO:tasks.workunit.client.0.smithi097.stderr:++ SHELL=/bin/bash 2024-03-21T13:28:59.729 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T13:28:59.729 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.729 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T13:28:59.729 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.729 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T13:28:59.729 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.729 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T13:28:59.730 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.730 INFO:tasks.workunit.client.0.smithi097.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T13:28:59.730 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T13:28:59.730 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T13:28:59.730 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.730 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T13:28:59.730 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.730 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T13:28:59.730 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.730 INFO:tasks.workunit.client.0.smithi097.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.730 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.730 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T13:28:59.730 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.730 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T13:28:59.730 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.731 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/locale 2024-03-21T13:28:59.731 INFO:tasks.workunit.client.0.smithi097.stderr:++ awk '{print $1}' 2024-03-21T13:28:59.731 INFO:tasks.workunit.client.0.smithi097.stderr:++ echo delete_pools_parallel 2024-03-21T13:28:59.731 INFO:tasks.workunit.client.0.smithi097.stdout:test delete_pools_parallel on pid 80337 2024-03-21T13:28:59.733 INFO:tasks.workunit.client.0.smithi097.stderr:+ ff=delete_pools_parallel 2024-03-21T13:28:59.733 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.733 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T13:28:59.733 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.733 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T13:28:59.733 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=80337 2024-03-21T13:28:59.734 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T13:28:59.734 INFO:tasks.workunit.client.0.smithi097.stderr:+ echo 'test delete_pools_parallel on pid 80337' 2024-03-21T13:28:59.734 INFO:tasks.workunit.client.0.smithi097.stderr:+ pids[$f]=80337 2024-03-21T13:28:59.734 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-21T13:28:59.734 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T13:28:59.734 INFO:tasks.workunit.client.0.smithi097.stderr:+ bash -o pipefail -exc 'ceph_test_rados_delete_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/delete_pools_parallel.xml 2>&1 | tee ceph_test_rados_delete_pools_parallel.log | sed "s/^/ delete_pools_parallel: /"' 2024-03-21T13:28:59.734 INFO:tasks.workunit.client.0.smithi097.stderr:+ tee ceph_test_rados_api_c_write_operations.log 2024-03-21T13:28:59.734 INFO:tasks.workunit.client.0.smithi097.stderr:++ printf %25s cls 2024-03-21T13:28:59.734 INFO:tasks.workunit.client.0.smithi097.stderr:+ r=' cls' 2024-03-21T13:28:59.734 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T13:28:59.734 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T13:28:59.734 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T13:28:59.734 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.734 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T13:28:59.734 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.735 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T13:28:59.735 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T13:28:59.735 INFO:tasks.workunit.client.0.smithi097.stderr:+++ return 2024-03-21T13:28:59.735 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.735 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T13:28:59.735 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.735 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T13:28:59.735 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export S_COLORS=auto 2024-03-21T13:28:59.735 INFO:tasks.workunit.client.0.smithi097.stderr:+++ S_COLORS=auto 2024-03-21T13:28:59.735 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.735 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T13:28:59.735 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.735 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T13:28:59.735 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.735 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T13:28:59.736 INFO:tasks.workunit.client.0.smithi097.stderr:+ . /etc/bashrc 2024-03-21T13:28:59.736 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -z '' ']' 2024-03-21T13:28:59.736 INFO:tasks.workunit.client.0.smithi097.stderr:++ BASHRCSOURCED=Y 2024-03-21T13:28:59.736 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.736 INFO:tasks.workunit.client.0.smithi097.stderr:++ shopt -q login_shell 2024-03-21T13:28:59.736 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T13:28:59.736 INFO:tasks.workunit.client.0.smithi097.stderr:++ awk '{print $1}' 2024-03-21T13:28:59.736 INFO:tasks.workunit.client.0.smithi097.stderr:++ echo cls 2024-03-21T13:28:59.736 INFO:tasks.workunit.client.0.smithi097.stderr:+++ umask 2024-03-21T13:28:59.736 INFO:tasks.workunit.client.0.smithi097.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T13:28:59.736 INFO:tasks.workunit.client.0.smithi097.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T13:28:59.736 INFO:tasks.workunit.client.0.smithi097.stderr:++++ LANG=en_US.UTF-8 2024-03-21T13:28:59.736 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.736 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T13:28:59.736 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.736 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG=en_US.UTF-8 2024-03-21T13:28:59.737 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T13:28:59.737 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup config 2024-03-21T13:28:59.737 INFO:tasks.workunit.client.0.smithi097.stderr:++ SHELL=/bin/bash 2024-03-21T13:28:59.737 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.737 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.737 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T13:28:59.737 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.737 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T13:28:59.737 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.737 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' dumb = linux ']' 2024-03-21T13:28:59.737 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.737 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T13:28:59.737 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.737 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/less.sh 2024-03-21T13:28:59.737 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T13:28:59.738 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T13:28:59.738 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.738 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.738 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T13:28:59.738 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T13:28:59.738 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.738 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.738 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T13:28:59.738 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T13:28:59.738 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.738 INFO:tasks.workunit.client.0.smithi097.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.738 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.738 INFO:tasks.workunit.client.0.smithi097.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T13:28:59.738 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T13:28:59.738 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.739 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T13:28:59.739 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.739 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T13:28:59.739 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T13:28:59.739 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T13:28:59.739 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.739 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T13:28:59.739 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.739 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T13:28:59.739 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T13:28:59.739 INFO:tasks.workunit.client.0.smithi097.stderr:+++ return 2024-03-21T13:28:59.739 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.739 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T13:28:59.739 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.739 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T13:28:59.740 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export S_COLORS=auto 2024-03-21T13:28:59.740 INFO:tasks.workunit.client.0.smithi097.stderr:+++ S_COLORS=auto 2024-03-21T13:28:59.740 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.740 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T13:28:59.740 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.740 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T13:28:59.740 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.740 INFO:tasks.workunit.client.0.smithi097.stderr:+ ff=cls 2024-03-21T13:28:59.740 INFO:tasks.workunit.client.0.smithi097.stdout:test cls on pid 80353 2024-03-21T13:28:59.742 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=80353 2024-03-21T13:28:59.742 INFO:tasks.workunit.client.0.smithi097.stderr:+ echo 'test cls on pid 80353' 2024-03-21T13:28:59.742 INFO:tasks.workunit.client.0.smithi097.stderr:+ pids[$f]=80353 2024-03-21T13:28:59.742 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-21T13:28:59.742 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T13:28:59.742 INFO:tasks.workunit.client.0.smithi097.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_cls 2>&1 | tee ceph_test_neorados_cls.log | sed "s/^/ cls: /"' 2024-03-21T13:28:59.742 INFO:tasks.workunit.client.0.smithi097.stderr:++ printf %25s cmd 2024-03-21T13:28:59.742 INFO:tasks.workunit.client.0.smithi097.stderr:+ r=' cmd' 2024-03-21T13:28:59.743 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T13:28:59.743 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T13:28:59.743 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T13:28:59.743 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.743 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T13:28:59.743 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.743 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T13:28:59.743 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T13:28:59.743 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.743 INFO:tasks.workunit.client.0.smithi097.stderr:+++++ readlink /proc/80253/exe 2024-03-21T13:28:59.743 INFO:tasks.workunit.client.0.smithi097.stderr:++++ basename /usr/bin/bash 2024-03-21T13:28:59.743 INFO:tasks.workunit.client.0.smithi097.stderr:++ echo cmd 2024-03-21T13:28:59.743 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T13:28:59.743 INFO:tasks.workunit.client.0.smithi097.stderr:+ . /etc/bashrc 2024-03-21T13:28:59.743 INFO:tasks.workunit.client.0.smithi097.stderr:++ awk '{print $1}' 2024-03-21T13:28:59.744 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -z '' ']' 2024-03-21T13:28:59.744 INFO:tasks.workunit.client.0.smithi097.stderr:++ BASHRCSOURCED=Y 2024-03-21T13:28:59.744 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.744 INFO:tasks.workunit.client.0.smithi097.stderr:++ shopt -q login_shell 2024-03-21T13:28:59.744 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_declare='declare -f' 2024-03-21T13:28:59.744 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_opt=-f 2024-03-21T13:28:59.744 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export which_declare 2024-03-21T13:28:59.744 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export -f which 2024-03-21T13:28:59.744 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset i 2024-03-21T13:28:59.744 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset -f pathmunge 2024-03-21T13:28:59.744 INFO:tasks.workunit.client.0.smithi097.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T13:28:59.744 INFO:tasks.workunit.client.0.smithi097.stderr:+ export PATH 2024-03-21T13:28:59.744 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T13:28:59.744 INFO:tasks.workunit.client.0.smithi097.stderr:+ unset rc 2024-03-21T13:28:59.745 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_api_cls_remote_reads --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cls_remote_reads.xml 2024-03-21T13:28:59.745 INFO:tasks.workunit.client.0.smithi097.stdout:test cmd on pid 80374 2024-03-21T13:28:59.749 INFO:tasks.workunit.client.0.smithi097.stderr:+ ff=cmd 2024-03-21T13:28:59.749 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=80374 2024-03-21T13:28:59.749 INFO:tasks.workunit.client.0.smithi097.stderr:+ echo 'test cmd on pid 80374' 2024-03-21T13:28:59.749 INFO:tasks.workunit.client.0.smithi097.stderr:+ pids[$f]=80374 2024-03-21T13:28:59.749 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-21T13:28:59.749 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T13:28:59.749 INFO:tasks.workunit.client.0.smithi097.stderr:+ tee ceph_test_rados_api_cls_remote_reads.log 2024-03-21T13:28:59.749 INFO:tasks.workunit.client.0.smithi097.stderr:+ sed 's/^/ api_cls_remote_reads: /' 2024-03-21T13:28:59.749 INFO:tasks.workunit.client.0.smithi097.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_cmd 2>&1 | tee ceph_test_neorados_cmd.log | sed "s/^/ cmd: /"' 2024-03-21T13:28:59.749 INFO:tasks.workunit.client.0.smithi097.stderr:+++ umask 2024-03-21T13:28:59.749 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T13:28:59.750 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T13:28:59.750 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T13:28:59.750 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.750 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T13:28:59.750 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.750 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T13:28:59.750 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T13:28:59.750 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.750 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T13:28:59.750 INFO:tasks.workunit.client.0.smithi097.stderr:+ . /etc/bashrc 2024-03-21T13:28:59.750 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -z '' ']' 2024-03-21T13:28:59.750 INFO:tasks.workunit.client.0.smithi097.stderr:+++++ readlink /proc/80242/exe 2024-03-21T13:28:59.750 INFO:tasks.workunit.client.0.smithi097.stderr:++ BASHRCSOURCED=Y 2024-03-21T13:28:59.750 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.751 INFO:tasks.workunit.client.0.smithi097.stderr:++ shopt -q login_shell 2024-03-21T13:28:59.751 INFO:tasks.workunit.client.0.smithi097.stderr:+++ umask 2024-03-21T13:28:59.751 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T13:28:59.751 INFO:tasks.workunit.client.0.smithi097.stderr:++ SHELL=/bin/bash 2024-03-21T13:28:59.751 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.751 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T13:28:59.751 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.751 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T13:28:59.751 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T13:28:59.751 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.751 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T13:28:59.751 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T13:28:59.751 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.751 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T13:28:59.751 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T13:28:59.752 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.752 INFO:tasks.workunit.client.0.smithi097.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.752 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T13:28:59.752 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.752 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T13:28:59.752 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.752 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T13:28:59.752 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.752 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T13:28:59.752 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.752 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.752 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T13:28:59.752 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T13:28:59.752 INFO:tasks.workunit.client.0.smithi097.stderr:+++ return 2024-03-21T13:28:59.753 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.753 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T13:28:59.753 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.753 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T13:28:59.753 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export S_COLORS=auto 2024-03-21T13:28:59.753 INFO:tasks.workunit.client.0.smithi097.stderr:+++ S_COLORS=auto 2024-03-21T13:28:59.753 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.753 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T13:28:59.753 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.753 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T13:28:59.753 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.753 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T13:28:59.753 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T13:28:59.753 INFO:tasks.workunit.client.0.smithi097.stderr:++ printf %25s handler_error 2024-03-21T13:28:59.753 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T13:28:59.754 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.754 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T13:28:59.754 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.754 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T13:28:59.754 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T13:28:59.754 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.754 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T13:28:59.754 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.754 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T13:28:59.754 INFO:tasks.workunit.client.0.smithi097.stderr:+ r=' handler_error' 2024-03-21T13:28:59.754 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.755 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T13:28:59.755 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.755 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T13:28:59.755 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup 2024-03-21T13:28:59.755 INFO:tasks.workunit.client.0.smithi097.stderr:++ echo handler_error 2024-03-21T13:28:59.755 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T13:28:59.755 INFO:tasks.workunit.client.0.smithi097.stderr:++ SHELL=/bin/bash 2024-03-21T13:28:59.755 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.755 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T13:28:59.755 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.755 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T13:28:59.755 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T13:28:59.755 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.756 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T13:28:59.756 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.756 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T13:28:59.756 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T13:28:59.756 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.756 INFO:tasks.workunit.client.0.smithi097.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.756 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T13:28:59.756 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.756 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T13:28:59.756 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T13:28:59.756 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.756 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T13:28:59.756 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.756 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T13:28:59.756 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.757 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.757 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T13:28:59.757 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T13:28:59.757 INFO:tasks.workunit.client.0.smithi097.stderr:+++ return 2024-03-21T13:28:59.757 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.757 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T13:28:59.757 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.757 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T13:28:59.757 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export S_COLORS=auto 2024-03-21T13:28:59.757 INFO:tasks.workunit.client.0.smithi097.stderr:+++ S_COLORS=auto 2024-03-21T13:28:59.757 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.757 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T13:28:59.757 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.757 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T13:28:59.757 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.758 INFO:tasks.workunit.client.0.smithi097.stderr:++ awk '{print $1}' 2024-03-21T13:28:59.758 INFO:tasks.workunit.client.0.smithi097.stderr:++++ basename /usr/bin/bash 2024-03-21T13:28:59.758 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T13:28:59.758 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T13:28:59.758 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T13:28:59.758 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.758 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T13:28:59.758 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.758 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T13:28:59.758 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T13:28:59.758 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.759 INFO:tasks.workunit.client.0.smithi097.stdout:test handler_error on pid 80402 2024-03-21T13:28:59.762 INFO:tasks.workunit.client.0.smithi097.stderr:+ ff=handler_error 2024-03-21T13:28:59.762 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=80402 2024-03-21T13:28:59.762 INFO:tasks.workunit.client.0.smithi097.stderr:+ echo 'test handler_error on pid 80402' 2024-03-21T13:28:59.762 INFO:tasks.workunit.client.0.smithi097.stderr:+ pids[$f]=80402 2024-03-21T13:28:59.762 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-21T13:28:59.762 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T13:28:59.763 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_declare='declare -f' 2024-03-21T13:28:59.763 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_opt=-f 2024-03-21T13:28:59.763 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export which_declare 2024-03-21T13:28:59.763 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export -f which 2024-03-21T13:28:59.763 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset i 2024-03-21T13:28:59.763 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset -f pathmunge 2024-03-21T13:28:59.763 INFO:tasks.workunit.client.0.smithi097.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T13:28:59.763 INFO:tasks.workunit.client.0.smithi097.stderr:+ export PATH 2024-03-21T13:28:59.763 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T13:28:59.763 INFO:tasks.workunit.client.0.smithi097.stderr:+ unset rc 2024-03-21T13:28:59.763 INFO:tasks.workunit.client.0.smithi097.stderr:++ printf %25s io 2024-03-21T13:28:59.763 INFO:tasks.workunit.client.0.smithi097.stderr:+ r=' io' 2024-03-21T13:28:59.763 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/locale 2024-03-21T13:28:59.763 INFO:tasks.workunit.client.0.smithi097.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_handler_error 2>&1 | tee ceph_test_neorados_handler_error.log | sed "s/^/ handler_error: /"' 2024-03-21T13:28:59.763 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.763 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T13:28:59.764 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.764 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T13:28:59.764 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T13:28:59.764 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T13:28:59.764 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_api_c_read_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_read_operations.xml 2024-03-21T13:28:59.764 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T13:28:59.764 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T13:28:59.764 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T13:28:59.764 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.764 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T13:28:59.764 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.764 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T13:28:59.764 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T13:28:59.764 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.764 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T13:28:59.764 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.765 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T13:28:59.765 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.765 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T13:28:59.765 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.765 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T13:28:59.765 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup 2024-03-21T13:28:59.765 INFO:tasks.workunit.client.0.smithi097.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T13:28:59.765 INFO:tasks.workunit.client.0.smithi097.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T13:28:59.765 INFO:tasks.workunit.client.0.smithi097.stderr:++++ LANG=en_US.UTF-8 2024-03-21T13:28:59.765 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.765 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T13:28:59.765 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.765 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG=en_US.UTF-8 2024-03-21T13:28:59.765 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup config 2024-03-21T13:28:59.765 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.766 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.766 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' dumb = linux ']' 2024-03-21T13:28:59.766 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.766 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T13:28:59.766 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.766 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/less.sh 2024-03-21T13:28:59.766 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T13:28:59.766 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.766 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T13:28:59.766 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.766 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T13:28:59.766 INFO:tasks.workunit.client.0.smithi097.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T13:28:59.766 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T13:28:59.766 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T13:28:59.766 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T13:28:59.767 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.767 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T13:28:59.767 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.767 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T13:28:59.767 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T13:28:59.767 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.767 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T13:28:59.767 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.767 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T13:28:59.767 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T13:28:59.767 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.767 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T13:28:59.767 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T13:28:59.767 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.767 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T13:28:59.767 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T13:28:59.767 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.768 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup 2024-03-21T13:28:59.768 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T13:28:59.768 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.768 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T13:28:59.768 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T13:28:59.768 INFO:tasks.workunit.client.0.smithi097.stderr:+++ return 2024-03-21T13:28:59.768 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.768 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T13:28:59.768 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.768 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T13:28:59.768 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export S_COLORS=auto 2024-03-21T13:28:59.768 INFO:tasks.workunit.client.0.smithi097.stderr:+++ S_COLORS=auto 2024-03-21T13:28:59.768 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.768 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T13:28:59.768 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.768 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T13:28:59.769 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.769 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/locale 2024-03-21T13:28:59.769 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T13:28:59.769 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T13:28:59.769 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T13:28:59.769 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.769 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T13:28:59.769 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.769 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T13:28:59.769 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/locale 2024-03-21T13:28:59.769 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T13:28:59.769 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.769 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T13:28:59.769 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T13:28:59.769 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T13:28:59.769 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.769 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T13:28:59.770 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.770 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T13:28:59.770 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T13:28:59.770 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.770 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.770 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T13:28:59.770 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.770 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T13:28:59.770 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T13:28:59.770 INFO:tasks.workunit.client.0.smithi097.stderr:+ tee ceph_test_rados_api_c_read_operations.log 2024-03-21T13:28:59.770 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T13:28:59.770 INFO:tasks.workunit.client.0.smithi097.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T13:28:59.770 INFO:tasks.workunit.client.0.smithi097.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T13:28:59.770 INFO:tasks.workunit.client.0.smithi097.stderr:++++ LANG=en_US.UTF-8 2024-03-21T13:28:59.770 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.770 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T13:28:59.771 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.771 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG=en_US.UTF-8 2024-03-21T13:28:59.771 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup config 2024-03-21T13:28:59.771 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.771 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.771 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' dumb = linux ']' 2024-03-21T13:28:59.771 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.771 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T13:28:59.771 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.771 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/less.sh 2024-03-21T13:28:59.771 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T13:28:59.771 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.771 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T13:28:59.771 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.771 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T13:28:59.771 INFO:tasks.workunit.client.0.smithi097.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T13:28:59.771 INFO:tasks.workunit.client.0.smithi097.stderr:+ sed 's/^/ api_c_read_operations: /' 2024-03-21T13:28:59.772 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T13:28:59.772 INFO:tasks.workunit.client.0.smithi097.stderr:+ . /etc/bashrc 2024-03-21T13:28:59.772 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -z '' ']' 2024-03-21T13:28:59.772 INFO:tasks.workunit.client.0.smithi097.stderr:++ BASHRCSOURCED=Y 2024-03-21T13:28:59.772 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.772 INFO:tasks.workunit.client.0.smithi097.stderr:++ shopt -q login_shell 2024-03-21T13:28:59.772 INFO:tasks.workunit.client.0.smithi097.stderr:++ echo io 2024-03-21T13:28:59.772 INFO:tasks.workunit.client.0.smithi097.stderr:++ awk '{print $1}' 2024-03-21T13:28:59.772 INFO:tasks.workunit.client.0.smithi097.stderr:+++ umask 2024-03-21T13:28:59.772 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T13:28:59.772 INFO:tasks.workunit.client.0.smithi097.stderr:++ SHELL=/bin/bash 2024-03-21T13:28:59.772 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.772 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T13:28:59.772 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.772 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T13:28:59.772 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T13:28:59.772 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.772 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T13:28:59.773 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.773 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T13:28:59.773 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.773 INFO:tasks.workunit.client.0.smithi097.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.773 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.773 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T13:28:59.773 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.773 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T13:28:59.773 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.773 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.773 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T13:28:59.773 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.773 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T13:28:59.773 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T13:28:59.773 INFO:tasks.workunit.client.0.smithi097.stdout:test io on pid 80441 2024-03-21T13:28:59.777 INFO:tasks.workunit.client.0.smithi097.stderr:+ ff=io 2024-03-21T13:28:59.777 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=80441 2024-03-21T13:28:59.777 INFO:tasks.workunit.client.0.smithi097.stderr:+ echo 'test io on pid 80441' 2024-03-21T13:28:59.777 INFO:tasks.workunit.client.0.smithi097.stderr:+ pids[$f]=80441 2024-03-21T13:28:59.777 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-21T13:28:59.777 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T13:28:59.778 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T13:28:59.778 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T13:28:59.778 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T13:28:59.778 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T13:28:59.778 INFO:tasks.workunit.client.0.smithi097.stderr:++ printf %25s ec_io 2024-03-21T13:28:59.778 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.778 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T13:28:59.778 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.778 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T13:28:59.778 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T13:28:59.778 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.778 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T13:28:59.778 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.778 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T13:28:59.778 INFO:tasks.workunit.client.0.smithi097.stderr:+ r=' ec_io' 2024-03-21T13:28:59.779 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.779 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T13:28:59.779 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.779 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T13:28:59.779 INFO:tasks.workunit.client.0.smithi097.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_io 2>&1 | tee ceph_test_neorados_io.log | sed "s/^/ io: /"' 2024-03-21T13:28:59.779 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup 2024-03-21T13:28:59.779 INFO:tasks.workunit.client.0.smithi097.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T13:28:59.779 INFO:tasks.workunit.client.0.smithi097.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T13:28:59.779 INFO:tasks.workunit.client.0.smithi097.stderr:++++ LANG=en_US.UTF-8 2024-03-21T13:28:59.779 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.779 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T13:28:59.779 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.779 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG=en_US.UTF-8 2024-03-21T13:28:59.779 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup config 2024-03-21T13:28:59.779 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.779 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/locale 2024-03-21T13:28:59.779 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.780 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' dumb = linux ']' 2024-03-21T13:28:59.780 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.780 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T13:28:59.780 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.780 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/less.sh 2024-03-21T13:28:59.780 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T13:28:59.780 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.780 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T13:28:59.780 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.780 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T13:28:59.780 INFO:tasks.workunit.client.0.smithi097.stderr:+++++ readlink /proc/80272/exe 2024-03-21T13:28:59.780 INFO:tasks.workunit.client.0.smithi097.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T13:28:59.780 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T13:28:59.780 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.780 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T13:28:59.780 INFO:tasks.workunit.client.0.smithi097.stderr:++++ basename /usr/bin/bash 2024-03-21T13:28:59.781 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.781 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T13:28:59.781 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T13:28:59.781 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T13:28:59.781 INFO:tasks.workunit.client.0.smithi097.stderr:+++++ readlink /proc/80310/exe 2024-03-21T13:28:59.781 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_declare='declare -f' 2024-03-21T13:28:59.781 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_opt=-f 2024-03-21T13:28:59.781 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export which_declare 2024-03-21T13:28:59.781 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export -f which 2024-03-21T13:28:59.781 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset i 2024-03-21T13:28:59.781 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset -f pathmunge 2024-03-21T13:28:59.781 INFO:tasks.workunit.client.0.smithi097.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T13:28:59.781 INFO:tasks.workunit.client.0.smithi097.stderr:+ export PATH 2024-03-21T13:28:59.781 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T13:28:59.781 INFO:tasks.workunit.client.0.smithi097.stderr:+ unset rc 2024-03-21T13:28:59.781 INFO:tasks.workunit.client.0.smithi097.stderr:+ . /etc/bashrc 2024-03-21T13:28:59.781 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -z '' ']' 2024-03-21T13:28:59.782 INFO:tasks.workunit.client.0.smithi097.stderr:++ BASHRCSOURCED=Y 2024-03-21T13:28:59.782 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.782 INFO:tasks.workunit.client.0.smithi097.stderr:++ shopt -q login_shell 2024-03-21T13:28:59.782 INFO:tasks.workunit.client.0.smithi097.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T13:28:59.782 INFO:tasks.workunit.client.0.smithi097.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T13:28:59.782 INFO:tasks.workunit.client.0.smithi097.stderr:++++ LANG=en_US.UTF-8 2024-03-21T13:28:59.782 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.782 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T13:28:59.782 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.782 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG=en_US.UTF-8 2024-03-21T13:28:59.782 INFO:tasks.workunit.client.0.smithi097.stderr:+ sed 's/^/ list_parallel: /' 2024-03-21T13:28:59.782 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup config 2024-03-21T13:28:59.782 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.782 INFO:tasks.workunit.client.0.smithi097.stderr:++++ basename /usr/bin/bash 2024-03-21T13:28:59.782 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.782 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' dumb = linux ']' 2024-03-21T13:28:59.782 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_list_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/list_parallel.xml 2024-03-21T13:28:59.783 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.783 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T13:28:59.783 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.783 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/less.sh 2024-03-21T13:28:59.783 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T13:28:59.783 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.783 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T13:28:59.783 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.783 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T13:28:59.783 INFO:tasks.workunit.client.0.smithi097.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T13:28:59.783 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_declare='declare -f' 2024-03-21T13:28:59.783 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_opt=-f 2024-03-21T13:28:59.783 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export which_declare 2024-03-21T13:28:59.783 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export -f which 2024-03-21T13:28:59.783 INFO:tasks.workunit.client.0.smithi097.stderr:+++ umask 2024-03-21T13:28:59.783 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset i 2024-03-21T13:28:59.783 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset -f pathmunge 2024-03-21T13:28:59.784 INFO:tasks.workunit.client.0.smithi097.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T13:28:59.784 INFO:tasks.workunit.client.0.smithi097.stderr:+ export PATH 2024-03-21T13:28:59.784 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T13:28:59.784 INFO:tasks.workunit.client.0.smithi097.stderr:+ unset rc 2024-03-21T13:28:59.784 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T13:28:59.784 INFO:tasks.workunit.client.0.smithi097.stderr:++ SHELL=/bin/bash 2024-03-21T13:28:59.784 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T13:28:59.784 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.784 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T13:28:59.784 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T13:28:59.784 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.784 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T13:28:59.784 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T13:28:59.784 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.784 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T13:28:59.784 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.784 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T13:28:59.785 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T13:28:59.785 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.785 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T13:28:59.785 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.785 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T13:28:59.785 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T13:28:59.785 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.785 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.785 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T13:28:59.785 INFO:tasks.workunit.client.0.smithi097.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.785 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.785 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.785 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T13:28:59.785 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T13:28:59.785 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.785 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T13:28:59.785 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.786 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.786 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T13:28:59.786 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.786 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T13:28:59.786 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup 2024-03-21T13:28:59.786 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_open_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/open_pools_parallel.xml 2024-03-21T13:28:59.786 INFO:tasks.workunit.client.0.smithi097.stderr:+ tee ceph_test_rados_open_pools_parallel.log 2024-03-21T13:28:59.786 INFO:tasks.workunit.client.0.smithi097.stderr:+ tee ceph_test_rados_list_parallel.log 2024-03-21T13:28:59.786 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/locale 2024-03-21T13:28:59.786 INFO:tasks.workunit.client.0.smithi097.stderr:+++++ readlink /proc/80337/exe 2024-03-21T13:28:59.786 INFO:tasks.workunit.client.0.smithi097.stderr:+++++ readlink /proc/80353/exe 2024-03-21T13:28:59.786 INFO:tasks.workunit.client.0.smithi097.stderr:++ awk '{print $1}' 2024-03-21T13:28:59.786 INFO:tasks.workunit.client.0.smithi097.stderr:++++ basename /usr/bin/bash 2024-03-21T13:28:59.786 INFO:tasks.workunit.client.0.smithi097.stderr:+ sed 's/^/ open_pools_parallel: /' 2024-03-21T13:28:59.787 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_declare='declare -f' 2024-03-21T13:28:59.787 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_opt=-f 2024-03-21T13:28:59.787 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export which_declare 2024-03-21T13:28:59.787 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export -f which 2024-03-21T13:28:59.787 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset i 2024-03-21T13:28:59.787 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset -f pathmunge 2024-03-21T13:28:59.787 INFO:tasks.workunit.client.0.smithi097.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T13:28:59.787 INFO:tasks.workunit.client.0.smithi097.stderr:+ export PATH 2024-03-21T13:28:59.787 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T13:28:59.787 INFO:tasks.workunit.client.0.smithi097.stderr:+ unset rc 2024-03-21T13:28:59.787 INFO:tasks.workunit.client.0.smithi097.stderr:++ echo ec_io 2024-03-21T13:28:59.787 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T13:28:59.787 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T13:28:59.787 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T13:28:59.787 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.787 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T13:28:59.787 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.788 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T13:28:59.788 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T13:28:59.788 INFO:tasks.workunit.client.0.smithi097.stderr:+++ return 2024-03-21T13:28:59.788 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.788 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T13:28:59.788 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.788 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T13:28:59.788 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export S_COLORS=auto 2024-03-21T13:28:59.788 INFO:tasks.workunit.client.0.smithi097.stderr:+++ S_COLORS=auto 2024-03-21T13:28:59.788 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.788 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T13:28:59.788 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.788 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T13:28:59.788 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.788 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.788 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T13:28:59.788 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.789 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T13:28:59.789 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T13:28:59.789 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T13:28:59.789 INFO:tasks.workunit.client.0.smithi097.stderr:+ sed 's/^/ delete_pools_parallel: /' 2024-03-21T13:28:59.789 INFO:tasks.workunit.client.0.smithi097.stderr:++++ basename /usr/bin/bash 2024-03-21T13:28:59.789 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_declare='declare -f' 2024-03-21T13:28:59.789 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_opt=-f 2024-03-21T13:28:59.789 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export which_declare 2024-03-21T13:28:59.789 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export -f which 2024-03-21T13:28:59.789 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset i 2024-03-21T13:28:59.789 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset -f pathmunge 2024-03-21T13:28:59.789 INFO:tasks.workunit.client.0.smithi097.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T13:28:59.789 INFO:tasks.workunit.client.0.smithi097.stderr:+ export PATH 2024-03-21T13:28:59.789 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T13:28:59.789 INFO:tasks.workunit.client.0.smithi097.stderr:+ unset rc 2024-03-21T13:28:59.789 INFO:tasks.workunit.client.0.smithi097.stderr:+ tee ceph_test_rados_delete_pools_parallel.log 2024-03-21T13:28:59.790 INFO:tasks.workunit.client.0.smithi097.stdout:test ec_io on pid 80509 2024-03-21T13:28:59.791 INFO:tasks.workunit.client.0.smithi097.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T13:28:59.791 INFO:tasks.workunit.client.0.smithi097.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T13:28:59.791 INFO:tasks.workunit.client.0.smithi097.stderr:++++ LANG=en_US.UTF-8 2024-03-21T13:28:59.791 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.791 INFO:tasks.workunit.client.0.smithi097.stderr:+ ff=ec_io 2024-03-21T13:28:59.792 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T13:28:59.792 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.792 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG=en_US.UTF-8 2024-03-21T13:28:59.792 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup config 2024-03-21T13:28:59.792 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.792 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=80509 2024-03-21T13:28:59.792 INFO:tasks.workunit.client.0.smithi097.stderr:+ echo 'test ec_io on pid 80509' 2024-03-21T13:28:59.792 INFO:tasks.workunit.client.0.smithi097.stderr:+ pids[$f]=80509 2024-03-21T13:28:59.792 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-21T13:28:59.792 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.792 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T13:28:59.792 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' dumb = linux ']' 2024-03-21T13:28:59.792 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.792 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T13:28:59.792 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.792 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/less.sh 2024-03-21T13:28:59.792 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T13:28:59.793 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.793 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T13:28:59.793 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T13:28:59.793 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.793 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T13:28:59.793 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T13:28:59.793 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_neorados_cls 2024-03-21T13:28:59.793 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T13:28:59.793 INFO:tasks.workunit.client.0.smithi097.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T13:28:59.793 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.793 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T13:28:59.793 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.793 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T13:28:59.793 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T13:28:59.793 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.793 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_rados_delete_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/delete_pools_parallel.xml 2024-03-21T13:28:59.793 INFO:tasks.workunit.client.0.smithi097.stderr:+ tee ceph_test_neorados_cls.log 2024-03-21T13:28:59.794 INFO:tasks.workunit.client.0.smithi097.stderr:+ sed 's/^/ cls: /' 2024-03-21T13:28:59.794 INFO:tasks.workunit.client.0.smithi097.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_ec_io 2>&1 | tee ceph_test_neorados_ec_io.log | sed "s/^/ ec_io: /"' 2024-03-21T13:28:59.794 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T13:28:59.794 INFO:tasks.workunit.client.0.smithi097.stderr:+ . /etc/bashrc 2024-03-21T13:28:59.794 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -z '' ']' 2024-03-21T13:28:59.794 INFO:tasks.workunit.client.0.smithi097.stderr:++ BASHRCSOURCED=Y 2024-03-21T13:28:59.794 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.794 INFO:tasks.workunit.client.0.smithi097.stderr:++ printf %25s list 2024-03-21T13:28:59.794 INFO:tasks.workunit.client.0.smithi097.stderr:++ shopt -q login_shell 2024-03-21T13:28:59.794 INFO:tasks.workunit.client.0.smithi097.stderr:+ r=' list' 2024-03-21T13:28:59.794 INFO:tasks.workunit.client.0.smithi097.stderr:++ awk '{print $1}' 2024-03-21T13:28:59.794 INFO:tasks.workunit.client.0.smithi097.stderr:++ echo list 2024-03-21T13:28:59.794 INFO:tasks.workunit.client.0.smithi097.stderr:+ ff=list 2024-03-21T13:28:59.795 INFO:tasks.workunit.client.0.smithi097.stdout:test list on pid 80545 2024-03-21T13:28:59.798 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=80545 2024-03-21T13:28:59.798 INFO:tasks.workunit.client.0.smithi097.stderr:+ echo 'test list on pid 80545' 2024-03-21T13:28:59.798 INFO:tasks.workunit.client.0.smithi097.stderr:+ pids[$f]=80545 2024-03-21T13:28:59.798 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-21T13:28:59.798 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T13:28:59.799 INFO:tasks.workunit.client.0.smithi097.stderr:++ printf %25s ec_list 2024-03-21T13:28:59.799 INFO:tasks.workunit.client.0.smithi097.stderr:+ r=' ec_list' 2024-03-21T13:28:59.799 INFO:tasks.workunit.client.0.smithi097.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_list 2>&1 | tee ceph_test_neorados_list.log | sed "s/^/ list: /"' 2024-03-21T13:28:59.799 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T13:28:59.799 INFO:tasks.workunit.client.0.smithi097.stderr:+ . /etc/bashrc 2024-03-21T13:28:59.799 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -z '' ']' 2024-03-21T13:28:59.799 INFO:tasks.workunit.client.0.smithi097.stderr:++ BASHRCSOURCED=Y 2024-03-21T13:28:59.799 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.799 INFO:tasks.workunit.client.0.smithi097.stderr:++ shopt -q login_shell 2024-03-21T13:28:59.799 INFO:tasks.workunit.client.0.smithi097.stderr:+++++ readlink /proc/80374/exe 2024-03-21T13:28:59.799 INFO:tasks.workunit.client.0.smithi097.stderr:+++ umask 2024-03-21T13:28:59.799 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T13:28:59.799 INFO:tasks.workunit.client.0.smithi097.stderr:++ SHELL=/bin/bash 2024-03-21T13:28:59.799 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.799 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T13:28:59.799 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.799 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T13:28:59.800 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T13:28:59.800 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.800 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T13:28:59.800 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.800 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T13:28:59.800 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.800 INFO:tasks.workunit.client.0.smithi097.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.800 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.800 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T13:28:59.800 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.800 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T13:28:59.800 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.800 INFO:tasks.workunit.client.0.smithi097.stderr:+++ umask 2024-03-21T13:28:59.800 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T13:28:59.800 INFO:tasks.workunit.client.0.smithi097.stderr:++ SHELL=/bin/bash 2024-03-21T13:28:59.800 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.800 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T13:28:59.801 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.801 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T13:28:59.801 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T13:28:59.801 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.801 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T13:28:59.801 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.801 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T13:28:59.801 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.801 INFO:tasks.workunit.client.0.smithi097.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.801 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.801 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T13:28:59.801 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.801 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T13:28:59.801 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.801 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T13:28:59.801 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T13:28:59.801 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T13:28:59.802 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.802 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T13:28:59.802 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.802 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T13:28:59.802 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T13:28:59.802 INFO:tasks.workunit.client.0.smithi097.stderr:+++ return 2024-03-21T13:28:59.802 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.802 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T13:28:59.802 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.802 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T13:28:59.802 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export S_COLORS=auto 2024-03-21T13:28:59.802 INFO:tasks.workunit.client.0.smithi097.stderr:+++ S_COLORS=auto 2024-03-21T13:28:59.802 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.802 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T13:28:59.802 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.802 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T13:28:59.802 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.803 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T13:28:59.803 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T13:28:59.803 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T13:28:59.803 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.803 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T13:28:59.803 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.803 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T13:28:59.803 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T13:28:59.803 INFO:tasks.workunit.client.0.smithi097.stderr:+++ return 2024-03-21T13:28:59.803 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.803 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T13:28:59.803 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.803 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T13:28:59.803 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export S_COLORS=auto 2024-03-21T13:28:59.803 INFO:tasks.workunit.client.0.smithi097.stderr:+++ S_COLORS=auto 2024-03-21T13:28:59.803 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.804 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T13:28:59.804 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.804 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T13:28:59.804 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.804 INFO:tasks.workunit.client.0.smithi097.stderr:++++ basename /usr/bin/bash 2024-03-21T13:28:59.804 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T13:28:59.804 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T13:28:59.804 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T13:28:59.804 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.804 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T13:28:59.804 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.804 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T13:28:59.804 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T13:28:59.804 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.804 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T13:28:59.804 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.804 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T13:28:59.805 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.805 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T13:28:59.805 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.805 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T13:28:59.805 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup 2024-03-21T13:28:59.805 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_declare='declare -f' 2024-03-21T13:28:59.805 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_opt=-f 2024-03-21T13:28:59.805 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export which_declare 2024-03-21T13:28:59.805 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export -f which 2024-03-21T13:28:59.805 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset i 2024-03-21T13:28:59.805 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset -f pathmunge 2024-03-21T13:28:59.805 INFO:tasks.workunit.client.0.smithi097.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T13:28:59.805 INFO:tasks.workunit.client.0.smithi097.stderr:+ export PATH 2024-03-21T13:28:59.805 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T13:28:59.805 INFO:tasks.workunit.client.0.smithi097.stderr:+ unset rc 2024-03-21T13:28:59.805 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T13:28:59.805 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T13:28:59.805 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T13:28:59.806 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.806 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T13:28:59.806 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.806 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/locale 2024-03-21T13:28:59.806 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T13:28:59.806 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T13:28:59.806 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.806 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.806 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T13:28:59.806 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.806 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T13:28:59.806 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T13:28:59.806 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T13:28:59.806 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T13:28:59.806 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T13:28:59.807 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.807 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T13:28:59.807 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.807 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T13:28:59.807 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T13:28:59.807 INFO:tasks.workunit.client.0.smithi097.stderr:+++ return 2024-03-21T13:28:59.807 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.807 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T13:28:59.807 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.807 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T13:28:59.807 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export S_COLORS=auto 2024-03-21T13:28:59.807 INFO:tasks.workunit.client.0.smithi097.stderr:+++ S_COLORS=auto 2024-03-21T13:28:59.807 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.807 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T13:28:59.807 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.807 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T13:28:59.807 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.808 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_neorados_cmd 2024-03-21T13:28:59.808 INFO:tasks.workunit.client.0.smithi097.stderr:+ tee ceph_test_neorados_cmd.log 2024-03-21T13:28:59.808 INFO:tasks.workunit.client.0.smithi097.stderr:+ sed 's/^/ cmd: /' 2024-03-21T13:28:59.808 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T13:28:59.808 INFO:tasks.workunit.client.0.smithi097.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T13:28:59.808 INFO:tasks.workunit.client.0.smithi097.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T13:28:59.808 INFO:tasks.workunit.client.0.smithi097.stderr:++++ LANG=en_US.UTF-8 2024-03-21T13:28:59.808 INFO:tasks.workunit.client.0.smithi097.stderr:++ echo ec_list 2024-03-21T13:28:59.808 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.808 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T13:28:59.808 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.808 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG=en_US.UTF-8 2024-03-21T13:28:59.808 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup config 2024-03-21T13:28:59.808 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.808 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.808 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' dumb = linux ']' 2024-03-21T13:28:59.808 INFO:tasks.workunit.client.0.smithi097.stderr:++ awk '{print $1}' 2024-03-21T13:28:59.809 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.809 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T13:28:59.809 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.809 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/less.sh 2024-03-21T13:28:59.809 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T13:28:59.809 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.809 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T13:28:59.809 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.809 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T13:28:59.809 INFO:tasks.workunit.client.0.smithi097.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T13:28:59.809 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T13:28:59.809 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T13:28:59.809 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T13:28:59.809 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.809 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T13:28:59.809 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.810 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T13:28:59.810 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T13:28:59.810 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.810 INFO:tasks.workunit.client.0.smithi097.stderr:+ ff=ec_list 2024-03-21T13:28:59.810 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=80592 2024-03-21T13:28:59.810 INFO:tasks.workunit.client.0.smithi097.stderr:+ echo 'test ec_list on pid 80592' 2024-03-21T13:28:59.810 INFO:tasks.workunit.client.0.smithi097.stdout:test ec_list on pid 80592 2024-03-21T13:28:59.815 INFO:tasks.workunit.client.0.smithi097.stderr:+ pids[$f]=80592 2024-03-21T13:28:59.815 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-21T13:28:59.815 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T13:28:59.815 INFO:tasks.workunit.client.0.smithi097.stderr:++ printf %25s misc 2024-03-21T13:28:59.815 INFO:tasks.workunit.client.0.smithi097.stderr:+ r=' misc' 2024-03-21T13:28:59.815 INFO:tasks.workunit.client.0.smithi097.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_ec_list 2>&1 | tee ceph_test_neorados_ec_list.log | sed "s/^/ ec_list: /"' 2024-03-21T13:28:59.815 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T13:28:59.815 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T13:28:59.815 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T13:28:59.816 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.816 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T13:28:59.816 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.816 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T13:28:59.816 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T13:28:59.816 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.816 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T13:28:59.816 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.816 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T13:28:59.816 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.816 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T13:28:59.816 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.816 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T13:28:59.816 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup 2024-03-21T13:28:59.816 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T13:28:59.816 INFO:tasks.workunit.client.0.smithi097.stderr:+ . /etc/bashrc 2024-03-21T13:28:59.817 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -z '' ']' 2024-03-21T13:28:59.817 INFO:tasks.workunit.client.0.smithi097.stderr:++ BASHRCSOURCED=Y 2024-03-21T13:28:59.817 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.817 INFO:tasks.workunit.client.0.smithi097.stderr:++ shopt -q login_shell 2024-03-21T13:28:59.817 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T13:28:59.817 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T13:28:59.817 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T13:28:59.817 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.817 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T13:28:59.817 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.817 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T13:28:59.817 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T13:28:59.817 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.817 INFO:tasks.workunit.client.0.smithi097.stderr:+++++ readlink /proc/80441/exe 2024-03-21T13:28:59.817 INFO:tasks.workunit.client.0.smithi097.stderr:++++ basename /usr/bin/bash 2024-03-21T13:28:59.817 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_declare='declare -f' 2024-03-21T13:28:59.817 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_opt=-f 2024-03-21T13:28:59.818 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export which_declare 2024-03-21T13:28:59.818 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export -f which 2024-03-21T13:28:59.818 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset i 2024-03-21T13:28:59.818 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset -f pathmunge 2024-03-21T13:28:59.818 INFO:tasks.workunit.client.0.smithi097.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T13:28:59.818 INFO:tasks.workunit.client.0.smithi097.stderr:+ export PATH 2024-03-21T13:28:59.818 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T13:28:59.818 INFO:tasks.workunit.client.0.smithi097.stderr:+ unset rc 2024-03-21T13:28:59.818 INFO:tasks.workunit.client.0.smithi097.stderr:++ echo misc 2024-03-21T13:28:59.818 INFO:tasks.workunit.client.0.smithi097.stderr:+ tee ceph_test_neorados_io.log 2024-03-21T13:28:59.818 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T13:28:59.818 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T13:28:59.818 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T13:28:59.818 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.818 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T13:28:59.818 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.819 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T13:28:59.819 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T13:28:59.819 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.819 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T13:28:59.819 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.819 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T13:28:59.819 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.819 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T13:28:59.819 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.819 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T13:28:59.819 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup 2024-03-21T13:28:59.819 INFO:tasks.workunit.client.0.smithi097.stderr:++ awk '{print $1}' 2024-03-21T13:28:59.819 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/locale 2024-03-21T13:28:59.819 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T13:28:59.819 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T13:28:59.819 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T13:28:59.819 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.820 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T13:28:59.820 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.820 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T13:28:59.820 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T13:28:59.820 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.820 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T13:28:59.820 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.820 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T13:28:59.820 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.820 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T13:28:59.820 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.820 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T13:28:59.820 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup 2024-03-21T13:28:59.820 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_neorados_io 2024-03-21T13:28:59.820 INFO:tasks.workunit.client.0.smithi097.stderr:+ sed 's/^/ io: /' 2024-03-21T13:28:59.820 INFO:tasks.workunit.client.0.smithi097.stderr:+++ umask 2024-03-21T13:28:59.820 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T13:28:59.821 INFO:tasks.workunit.client.0.smithi097.stderr:++ SHELL=/bin/bash 2024-03-21T13:28:59.821 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.821 INFO:tasks.workunit.client.0.smithi097.stderr:+ ff=misc 2024-03-21T13:28:59.821 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T13:28:59.821 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.821 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T13:28:59.821 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=80660 2024-03-21T13:28:59.821 INFO:tasks.workunit.client.0.smithi097.stdout:test misc on pid 80660 2024-03-21T13:28:59.825 INFO:tasks.workunit.client.0.smithi097.stderr:+ echo 'test misc on pid 80660' 2024-03-21T13:28:59.826 INFO:tasks.workunit.client.0.smithi097.stderr:+ pids[$f]=80660 2024-03-21T13:28:59.826 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-21T13:28:59.826 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T13:28:59.826 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T13:28:59.826 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.826 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T13:28:59.826 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.826 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T13:28:59.826 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.826 INFO:tasks.workunit.client.0.smithi097.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.826 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.826 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T13:28:59.826 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.826 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T13:28:59.827 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.827 INFO:tasks.workunit.client.0.smithi097.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_misc 2>&1 | tee ceph_test_neorados_misc.log | sed "s/^/ misc: /"' 2024-03-21T13:28:59.827 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.827 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T13:28:59.827 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.827 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T13:28:59.827 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T13:28:59.827 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T13:28:59.827 INFO:tasks.workunit.client.0.smithi097.stderr:+ . /etc/bashrc 2024-03-21T13:28:59.827 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T13:28:59.827 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -z '' ']' 2024-03-21T13:28:59.827 INFO:tasks.workunit.client.0.smithi097.stderr:++ BASHRCSOURCED=Y 2024-03-21T13:28:59.827 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.827 INFO:tasks.workunit.client.0.smithi097.stderr:++ shopt -q login_shell 2024-03-21T13:28:59.827 INFO:tasks.workunit.client.0.smithi097.stderr:++ printf %25s pool 2024-03-21T13:28:59.827 INFO:tasks.workunit.client.0.smithi097.stderr:+ r=' pool' 2024-03-21T13:28:59.827 INFO:tasks.workunit.client.0.smithi097.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T13:28:59.828 INFO:tasks.workunit.client.0.smithi097.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T13:28:59.828 INFO:tasks.workunit.client.0.smithi097.stderr:++++ LANG=en_US.UTF-8 2024-03-21T13:28:59.828 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.828 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T13:28:59.828 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.828 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG=en_US.UTF-8 2024-03-21T13:28:59.828 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup config 2024-03-21T13:28:59.828 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/locale 2024-03-21T13:28:59.828 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.828 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.828 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' dumb = linux ']' 2024-03-21T13:28:59.828 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.828 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T13:28:59.828 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.828 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/less.sh 2024-03-21T13:28:59.828 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T13:28:59.828 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.829 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T13:28:59.829 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.829 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T13:28:59.829 INFO:tasks.workunit.client.0.smithi097.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T13:28:59.829 INFO:tasks.workunit.client.0.smithi097.stderr:+++ umask 2024-03-21T13:28:59.829 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T13:28:59.829 INFO:tasks.workunit.client.0.smithi097.stderr:++ SHELL=/bin/bash 2024-03-21T13:28:59.829 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.829 INFO:tasks.workunit.client.0.smithi097.stderr:+++++ readlink /proc/80509/exe 2024-03-21T13:28:59.829 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T13:28:59.829 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.829 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T13:28:59.829 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T13:28:59.829 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.829 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T13:28:59.830 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.830 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T13:28:59.830 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.830 INFO:tasks.workunit.client.0.smithi097.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.830 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.830 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T13:28:59.830 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.830 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T13:28:59.830 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.830 INFO:tasks.workunit.client.0.smithi097.stderr:++++ basename /usr/bin/bash 2024-03-21T13:28:59.830 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_declare='declare -f' 2024-03-21T13:28:59.830 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_opt=-f 2024-03-21T13:28:59.830 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export which_declare 2024-03-21T13:28:59.830 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export -f which 2024-03-21T13:28:59.830 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset i 2024-03-21T13:28:59.830 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset -f pathmunge 2024-03-21T13:28:59.831 INFO:tasks.workunit.client.0.smithi097.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T13:28:59.831 INFO:tasks.workunit.client.0.smithi097.stderr:+ export PATH 2024-03-21T13:28:59.831 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T13:28:59.831 INFO:tasks.workunit.client.0.smithi097.stderr:+ unset rc 2024-03-21T13:28:59.831 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/locale 2024-03-21T13:28:59.831 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_neorados_ec_io 2024-03-21T13:28:59.831 INFO:tasks.workunit.client.0.smithi097.stderr:++ echo pool 2024-03-21T13:28:59.831 INFO:tasks.workunit.client.0.smithi097.stderr:++ awk '{print $1}' 2024-03-21T13:28:59.831 INFO:tasks.workunit.client.0.smithi097.stderr:+ tee ceph_test_neorados_ec_io.log 2024-03-21T13:28:59.831 INFO:tasks.workunit.client.0.smithi097.stdout:test pool on pid 80724 2024-03-21T13:28:59.835 INFO:tasks.workunit.client.0.smithi097.stderr:+ ff=pool 2024-03-21T13:28:59.835 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=80724 2024-03-21T13:28:59.835 INFO:tasks.workunit.client.0.smithi097.stderr:+ echo 'test pool on pid 80724' 2024-03-21T13:28:59.835 INFO:tasks.workunit.client.0.smithi097.stderr:+ pids[$f]=80724 2024-03-21T13:28:59.835 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-21T13:28:59.835 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T13:28:59.835 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T13:28:59.836 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T13:28:59.836 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T13:28:59.836 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.836 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T13:28:59.836 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.836 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T13:28:59.836 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T13:28:59.836 INFO:tasks.workunit.client.0.smithi097.stderr:+++ return 2024-03-21T13:28:59.836 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.836 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T13:28:59.836 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.836 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.836 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T13:28:59.836 INFO:tasks.workunit.client.0.smithi097.stderr:+ sed 's/^/ ec_io: /' 2024-03-21T13:28:59.836 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T13:28:59.836 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.837 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T13:28:59.837 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T13:28:59.837 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export S_COLORS=auto 2024-03-21T13:28:59.837 INFO:tasks.workunit.client.0.smithi097.stderr:+++ S_COLORS=auto 2024-03-21T13:28:59.837 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.837 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T13:28:59.837 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.837 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T13:28:59.837 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.837 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T13:28:59.837 INFO:tasks.workunit.client.0.smithi097.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T13:28:59.837 INFO:tasks.workunit.client.0.smithi097.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T13:28:59.837 INFO:tasks.workunit.client.0.smithi097.stderr:++++ LANG=en_US.UTF-8 2024-03-21T13:28:59.837 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.838 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T13:28:59.838 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.838 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG=en_US.UTF-8 2024-03-21T13:28:59.838 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup config 2024-03-21T13:28:59.838 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.838 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.838 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' dumb = linux ']' 2024-03-21T13:28:59.838 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.838 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T13:28:59.838 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.838 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/less.sh 2024-03-21T13:28:59.838 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T13:28:59.838 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.838 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T13:28:59.838 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.838 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T13:28:59.839 INFO:tasks.workunit.client.0.smithi097.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T13:28:59.839 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.839 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T13:28:59.839 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.839 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T13:28:59.839 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T13:28:59.839 INFO:tasks.workunit.client.0.smithi097.stderr:++ printf %25s read_operations 2024-03-21T13:28:59.839 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T13:28:59.839 INFO:tasks.workunit.client.0.smithi097.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_pool 2>&1 | tee ceph_test_neorados_pool.log | sed "s/^/ pool: /"' 2024-03-21T13:28:59.839 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T13:28:59.839 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T13:28:59.839 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.839 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T13:28:59.839 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.839 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T13:28:59.839 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T13:28:59.840 INFO:tasks.workunit.client.0.smithi097.stderr:+++ return 2024-03-21T13:28:59.840 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.840 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T13:28:59.840 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.840 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T13:28:59.840 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export S_COLORS=auto 2024-03-21T13:28:59.840 INFO:tasks.workunit.client.0.smithi097.stderr:+++ S_COLORS=auto 2024-03-21T13:28:59.840 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.840 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T13:28:59.840 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.840 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T13:28:59.840 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.840 INFO:tasks.workunit.client.0.smithi097.stderr:+ r=' read_operations' 2024-03-21T13:28:59.840 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T13:28:59.840 INFO:tasks.workunit.client.0.smithi097.stderr:+ . /etc/bashrc 2024-03-21T13:28:59.840 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -z '' ']' 2024-03-21T13:28:59.841 INFO:tasks.workunit.client.0.smithi097.stderr:++ BASHRCSOURCED=Y 2024-03-21T13:28:59.841 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.841 INFO:tasks.workunit.client.0.smithi097.stderr:++ shopt -q login_shell 2024-03-21T13:28:59.841 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T13:28:59.841 INFO:tasks.workunit.client.0.smithi097.stderr:++ echo read_operations 2024-03-21T13:28:59.841 INFO:tasks.workunit.client.0.smithi097.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T13:28:59.841 INFO:tasks.workunit.client.0.smithi097.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T13:28:59.841 INFO:tasks.workunit.client.0.smithi097.stderr:++++ LANG=en_US.UTF-8 2024-03-21T13:28:59.841 INFO:tasks.workunit.client.0.smithi097.stderr:++ awk '{print $1}' 2024-03-21T13:28:59.841 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.841 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T13:28:59.841 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.841 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG=en_US.UTF-8 2024-03-21T13:28:59.841 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup config 2024-03-21T13:28:59.841 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.841 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.842 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' dumb = linux ']' 2024-03-21T13:28:59.842 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.842 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T13:28:59.842 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.842 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/less.sh 2024-03-21T13:28:59.842 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T13:28:59.842 INFO:tasks.workunit.client.0.smithi097.stderr:+++++ readlink /proc/80402/exe 2024-03-21T13:28:59.842 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T13:28:59.842 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T13:28:59.842 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.842 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T13:28:59.842 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.842 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T13:28:59.842 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T13:28:59.842 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.842 INFO:tasks.workunit.client.0.smithi097.stderr:++++ basename /usr/bin/bash 2024-03-21T13:28:59.843 INFO:tasks.workunit.client.0.smithi097.stderr:+++ umask 2024-03-21T13:28:59.843 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T13:28:59.843 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T13:28:59.843 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.843 INFO:tasks.workunit.client.0.smithi097.stderr:++ SHELL=/bin/bash 2024-03-21T13:28:59.843 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T13:28:59.843 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.843 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T13:28:59.843 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.843 INFO:tasks.workunit.client.0.smithi097.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T13:28:59.843 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T13:28:59.843 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.843 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T13:28:59.843 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T13:28:59.843 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_declare='declare -f' 2024-03-21T13:28:59.844 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_opt=-f 2024-03-21T13:28:59.844 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.844 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T13:28:59.844 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.844 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T13:28:59.844 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export which_declare 2024-03-21T13:28:59.844 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.844 INFO:tasks.workunit.client.0.smithi097.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.844 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export -f which 2024-03-21T13:28:59.844 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.844 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T13:28:59.844 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.844 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset i 2024-03-21T13:28:59.844 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T13:28:59.844 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset -f pathmunge 2024-03-21T13:28:59.844 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.844 INFO:tasks.workunit.client.0.smithi097.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T13:28:59.845 INFO:tasks.workunit.client.0.smithi097.stderr:+ export PATH 2024-03-21T13:28:59.845 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T13:28:59.845 INFO:tasks.workunit.client.0.smithi097.stderr:+ unset rc 2024-03-21T13:28:59.845 INFO:tasks.workunit.client.0.smithi097.stdout:test read_operations on pid 80760 2024-03-21T13:28:59.847 INFO:tasks.workunit.client.0.smithi097.stderr:+ ff=read_operations 2024-03-21T13:28:59.847 INFO:tasks.workunit.client.0.smithi097.stderr:+ tee ceph_test_neorados_handler_error.log 2024-03-21T13:28:59.847 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=80760 2024-03-21T13:28:59.848 INFO:tasks.workunit.client.0.smithi097.stderr:+ echo 'test read_operations on pid 80760' 2024-03-21T13:28:59.848 INFO:tasks.workunit.client.0.smithi097.stderr:+ pids[$f]=80760 2024-03-21T13:28:59.848 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-21T13:28:59.848 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T13:28:59.848 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_neorados_handler_error 2024-03-21T13:28:59.848 INFO:tasks.workunit.client.0.smithi097.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_read_operations 2>&1 | tee ceph_test_neorados_read_operations.log | sed "s/^/ read_operations: /"' 2024-03-21T13:28:59.848 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T13:28:59.848 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T13:28:59.848 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T13:28:59.848 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.848 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T13:28:59.848 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.848 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T13:28:59.848 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T13:28:59.848 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.848 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T13:28:59.849 INFO:tasks.workunit.client.0.smithi097.stderr:+ . /etc/bashrc 2024-03-21T13:28:59.849 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -z '' ']' 2024-03-21T13:28:59.849 INFO:tasks.workunit.client.0.smithi097.stderr:++ BASHRCSOURCED=Y 2024-03-21T13:28:59.849 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.849 INFO:tasks.workunit.client.0.smithi097.stderr:++ shopt -q login_shell 2024-03-21T13:28:59.849 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T13:28:59.849 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T13:28:59.849 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T13:28:59.849 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.849 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T13:28:59.849 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.849 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T13:28:59.849 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T13:28:59.849 INFO:tasks.workunit.client.0.smithi097.stderr:+++ return 2024-03-21T13:28:59.849 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.850 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T13:28:59.850 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.850 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T13:28:59.850 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export S_COLORS=auto 2024-03-21T13:28:59.850 INFO:tasks.workunit.client.0.smithi097.stderr:+++ S_COLORS=auto 2024-03-21T13:28:59.850 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.850 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T13:28:59.850 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.850 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T13:28:59.850 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.850 INFO:tasks.workunit.client.0.smithi097.stderr:+ sed 's/^/ handler_error: /' 2024-03-21T13:28:59.850 INFO:tasks.workunit.client.0.smithi097.stderr:++ printf %25s snapshots 2024-03-21T13:28:59.850 INFO:tasks.workunit.client.0.smithi097.stderr:+ r=' snapshots' 2024-03-21T13:28:59.851 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:28:59 smithi097 ceph-mon[30040]: pgmap v66: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:59.851 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:28:59 smithi097 ceph-mon[39416]: pgmap v66: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:59.853 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T13:28:59.853 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T13:28:59.853 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T13:28:59.853 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.853 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T13:28:59.853 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.853 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T13:28:59.853 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T13:28:59.853 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.853 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T13:28:59.853 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.853 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T13:28:59.853 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.853 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T13:28:59.854 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.854 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T13:28:59.854 INFO:tasks.workunit.client.0.smithi097.stderr:+++++ readlink /proc/80545/exe 2024-03-21T13:28:59.854 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup 2024-03-21T13:28:59.854 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T13:28:59.854 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T13:28:59.854 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T13:28:59.854 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.854 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T13:28:59.854 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.854 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T13:28:59.854 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T13:28:59.854 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.854 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T13:28:59.854 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.854 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T13:28:59.855 INFO:tasks.workunit.client.0.smithi097.stderr:++++ basename /usr/bin/bash 2024-03-21T13:28:59.855 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.855 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T13:28:59.855 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.855 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T13:28:59.855 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup 2024-03-21T13:28:59.855 INFO:tasks.workunit.client.0.smithi097.stderr:++ echo snapshots 2024-03-21T13:28:59.855 INFO:tasks.workunit.client.0.smithi097.stderr:++ awk '{print $1}' 2024-03-21T13:28:59.855 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_declare='declare -f' 2024-03-21T13:28:59.855 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_opt=-f 2024-03-21T13:28:59.855 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export which_declare 2024-03-21T13:28:59.855 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export -f which 2024-03-21T13:28:59.855 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset i 2024-03-21T13:28:59.855 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset -f pathmunge 2024-03-21T13:28:59.855 INFO:tasks.workunit.client.0.smithi097.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T13:28:59.856 INFO:tasks.workunit.client.0.smithi097.stderr:+ export PATH 2024-03-21T13:28:59.856 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T13:28:59.856 INFO:tasks.workunit.client.0.smithi097.stderr:+ unset rc 2024-03-21T13:28:59.856 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/locale 2024-03-21T13:28:59.856 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.856 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T13:28:59.856 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.856 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T13:28:59.856 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T13:28:59.856 INFO:tasks.workunit.client.0.smithi097.stderr:+++ umask 2024-03-21T13:28:59.856 INFO:tasks.workunit.client.0.smithi097.stderr:+ ff=snapshots 2024-03-21T13:28:59.856 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=80801 2024-03-21T13:28:59.856 INFO:tasks.workunit.client.0.smithi097.stderr:+ echo 'test snapshots on pid 80801' 2024-03-21T13:28:59.856 INFO:tasks.workunit.client.0.smithi097.stdout:test snapshots on pid 80801 2024-03-21T13:28:59.863 INFO:tasks.workunit.client.0.smithi097.stderr:+ pids[$f]=80801 2024-03-21T13:28:59.863 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-21T13:28:59.863 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T13:28:59.863 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T13:28:59.863 INFO:tasks.workunit.client.0.smithi097.stderr:++ SHELL=/bin/bash 2024-03-21T13:28:59.863 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.863 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T13:28:59.863 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.863 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T13:28:59.863 INFO:tasks.workunit.client.0.smithi097.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_snapshots 2>&1 | tee ceph_test_neorados_snapshots.log | sed "s/^/ snapshots: /"' 2024-03-21T13:28:59.863 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T13:28:59.863 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.864 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T13:28:59.864 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.864 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T13:28:59.864 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.864 INFO:tasks.workunit.client.0.smithi097.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.864 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.864 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T13:28:59.864 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.864 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T13:28:59.864 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.864 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T13:28:59.864 INFO:tasks.workunit.client.0.smithi097.stderr:+ . /etc/bashrc 2024-03-21T13:28:59.864 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -z '' ']' 2024-03-21T13:28:59.864 INFO:tasks.workunit.client.0.smithi097.stderr:++ BASHRCSOURCED=Y 2024-03-21T13:28:59.864 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.864 INFO:tasks.workunit.client.0.smithi097.stderr:++ shopt -q login_shell 2024-03-21T13:28:59.864 INFO:tasks.workunit.client.0.smithi097.stderr:+++ umask 2024-03-21T13:28:59.865 INFO:tasks.workunit.client.0.smithi097.stderr:++ printf %25s watch_notify 2024-03-21T13:28:59.865 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_neorados_list 2024-03-21T13:28:59.865 INFO:tasks.workunit.client.0.smithi097.stderr:+ r=' watch_notify' 2024-03-21T13:28:59.865 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T13:28:59.865 INFO:tasks.workunit.client.0.smithi097.stderr:++ SHELL=/bin/bash 2024-03-21T13:28:59.865 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.865 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T13:28:59.865 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.865 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T13:28:59.865 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T13:28:59.865 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.865 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T13:28:59.865 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.865 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T13:28:59.865 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T13:28:59.865 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/locale 2024-03-21T13:28:59.865 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T13:28:59.866 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T13:28:59.866 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.866 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T13:28:59.866 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.866 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T13:28:59.866 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T13:28:59.866 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.866 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.866 INFO:tasks.workunit.client.0.smithi097.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.866 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.866 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T13:28:59.866 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.866 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T13:28:59.866 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.867 INFO:tasks.workunit.client.0.smithi097.stderr:++ echo watch_notify 2024-03-21T13:28:59.867 INFO:tasks.workunit.client.0.smithi097.stderr:+ tee ceph_test_neorados_list.log 2024-03-21T13:28:59.867 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.867 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T13:28:59.867 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.867 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T13:28:59.867 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T13:28:59.867 INFO:tasks.workunit.client.0.smithi097.stderr:+ sed 's/^/ list: /' 2024-03-21T13:28:59.867 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T13:28:59.867 INFO:tasks.workunit.client.0.smithi097.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T13:28:59.867 INFO:tasks.workunit.client.0.smithi097.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T13:28:59.867 INFO:tasks.workunit.client.0.smithi097.stderr:++++ LANG=en_US.UTF-8 2024-03-21T13:28:59.867 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.867 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T13:28:59.867 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.867 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG=en_US.UTF-8 2024-03-21T13:28:59.868 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup config 2024-03-21T13:28:59.868 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.868 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.868 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' dumb = linux ']' 2024-03-21T13:28:59.868 INFO:tasks.workunit.client.0.smithi097.stderr:++ awk '{print $1}' 2024-03-21T13:28:59.868 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.868 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T13:28:59.868 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.868 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/less.sh 2024-03-21T13:28:59.868 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T13:28:59.868 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.868 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T13:28:59.868 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.868 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T13:28:59.868 INFO:tasks.workunit.client.0.smithi097.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T13:28:59.868 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T13:28:59.868 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T13:28:59.869 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T13:28:59.869 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.869 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T13:28:59.869 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.869 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T13:28:59.869 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T13:28:59.869 INFO:tasks.workunit.client.0.smithi097.stderr:+++ return 2024-03-21T13:28:59.869 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.869 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T13:28:59.869 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.869 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T13:28:59.869 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export S_COLORS=auto 2024-03-21T13:28:59.869 INFO:tasks.workunit.client.0.smithi097.stderr:+++ S_COLORS=auto 2024-03-21T13:28:59.869 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.869 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T13:28:59.870 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.870 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T13:28:59.870 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.870 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T13:28:59.870 INFO:tasks.workunit.client.0.smithi097.stdout:test watch_notify on pid 80832 2024-03-21T13:28:59.875 INFO:tasks.workunit.client.0.smithi097.stderr:+ ff=watch_notify 2024-03-21T13:28:59.875 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=80832 2024-03-21T13:28:59.875 INFO:tasks.workunit.client.0.smithi097.stderr:+ echo 'test watch_notify on pid 80832' 2024-03-21T13:28:59.875 INFO:tasks.workunit.client.0.smithi097.stderr:+ pids[$f]=80832 2024-03-21T13:28:59.875 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-21T13:28:59.875 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T13:28:59.875 INFO:tasks.workunit.client.0.smithi097.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_watch_notify 2>&1 | tee ceph_test_neorados_watch_notify.log | sed "s/^/ watch_notify: /"' 2024-03-21T13:28:59.876 INFO:tasks.workunit.client.0.smithi097.stderr:++ printf %25s write_operations 2024-03-21T13:28:59.876 INFO:tasks.workunit.client.0.smithi097.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T13:28:59.876 INFO:tasks.workunit.client.0.smithi097.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T13:28:59.876 INFO:tasks.workunit.client.0.smithi097.stderr:++++ LANG=en_US.UTF-8 2024-03-21T13:28:59.876 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.876 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T13:28:59.876 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.876 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG=en_US.UTF-8 2024-03-21T13:28:59.876 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup config 2024-03-21T13:28:59.876 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.876 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.876 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T13:28:59.876 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' dumb = linux ']' 2024-03-21T13:28:59.876 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.876 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T13:28:59.876 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T13:28:59.877 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.877 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T13:28:59.877 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/less.sh 2024-03-21T13:28:59.877 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.877 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T13:28:59.877 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T13:28:59.877 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.877 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T13:28:59.877 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.877 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T13:28:59.877 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.877 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T13:28:59.877 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T13:28:59.877 INFO:tasks.workunit.client.0.smithi097.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T13:28:59.877 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.877 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T13:28:59.878 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.878 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T13:28:59.878 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.878 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T13:28:59.878 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.878 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T13:28:59.878 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup 2024-03-21T13:28:59.878 INFO:tasks.workunit.client.0.smithi097.stderr:+ r=' write_operations' 2024-03-21T13:28:59.878 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/locale 2024-03-21T13:28:59.878 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T13:28:59.878 INFO:tasks.workunit.client.0.smithi097.stderr:+ . /etc/bashrc 2024-03-21T13:28:59.878 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -z '' ']' 2024-03-21T13:28:59.878 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T13:28:59.878 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T13:28:59.879 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T13:28:59.879 INFO:tasks.workunit.client.0.smithi097.stderr:++ BASHRCSOURCED=Y 2024-03-21T13:28:59.879 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.879 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T13:28:59.879 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.879 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T13:28:59.879 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T13:28:59.879 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.879 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.879 INFO:tasks.workunit.client.0.smithi097.stderr:++ shopt -q login_shell 2024-03-21T13:28:59.879 INFO:tasks.workunit.client.0.smithi097.stderr:+++++ readlink /proc/80592/exe 2024-03-21T13:28:59.879 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.879 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T13:28:59.879 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.880 INFO:tasks.workunit.client.0.smithi097.stderr:++ echo write_operations 2024-03-21T13:28:59.880 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T13:28:59.880 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T13:28:59.880 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T13:28:59.880 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T13:28:59.880 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T13:28:59.880 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.880 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T13:28:59.880 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.880 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T13:28:59.880 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T13:28:59.880 INFO:tasks.workunit.client.0.smithi097.stderr:+++ return 2024-03-21T13:28:59.880 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.880 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T13:28:59.880 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.880 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T13:28:59.881 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export S_COLORS=auto 2024-03-21T13:28:59.881 INFO:tasks.workunit.client.0.smithi097.stderr:+++ S_COLORS=auto 2024-03-21T13:28:59.881 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.881 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T13:28:59.881 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.881 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T13:28:59.881 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.881 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T13:28:59.881 INFO:tasks.workunit.client.0.smithi097.stderr:+++ umask 2024-03-21T13:28:59.881 INFO:tasks.workunit.client.0.smithi097.stderr:++++ basename /usr/bin/bash 2024-03-21T13:28:59.881 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T13:28:59.881 INFO:tasks.workunit.client.0.smithi097.stderr:++ SHELL=/bin/bash 2024-03-21T13:28:59.881 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.881 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T13:28:59.881 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.881 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T13:28:59.882 INFO:tasks.workunit.client.0.smithi097.stderr:++ awk '{print $1}' 2024-03-21T13:28:59.882 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T13:28:59.882 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.882 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T13:28:59.882 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.882 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T13:28:59.882 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.882 INFO:tasks.workunit.client.0.smithi097.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.882 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.882 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T13:28:59.882 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.882 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T13:28:59.882 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.882 INFO:tasks.workunit.client.0.smithi097.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T13:28:59.882 INFO:tasks.workunit.client.0.smithi097.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T13:28:59.882 INFO:tasks.workunit.client.0.smithi097.stderr:++++ LANG=en_US.UTF-8 2024-03-21T13:28:59.883 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.883 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T13:28:59.883 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.883 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG=en_US.UTF-8 2024-03-21T13:28:59.883 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup config 2024-03-21T13:28:59.883 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.883 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.883 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' dumb = linux ']' 2024-03-21T13:28:59.883 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.883 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T13:28:59.883 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.883 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/less.sh 2024-03-21T13:28:59.883 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T13:28:59.883 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.883 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T13:28:59.883 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.884 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T13:28:59.884 INFO:tasks.workunit.client.0.smithi097.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T13:28:59.884 INFO:tasks.workunit.client.0.smithi097.stdout:test write_operations on pid 80859 2024-03-21T13:28:59.886 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:28:59 smithi144 ceph-mon[33633]: pgmap v66: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:59.894 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_declare='declare -f' 2024-03-21T13:28:59.894 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_opt=-f 2024-03-21T13:28:59.894 INFO:tasks.workunit.client.0.smithi097.stderr:+ ff=write_operations 2024-03-21T13:28:59.894 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export which_declare 2024-03-21T13:28:59.894 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export -f which 2024-03-21T13:28:59.894 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset i 2024-03-21T13:28:59.894 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset -f pathmunge 2024-03-21T13:28:59.894 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=80859 2024-03-21T13:28:59.895 INFO:tasks.workunit.client.0.smithi097.stderr:+ echo 'test write_operations on pid 80859' 2024-03-21T13:28:59.895 INFO:tasks.workunit.client.0.smithi097.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T13:28:59.895 INFO:tasks.workunit.client.0.smithi097.stderr:+ pids[$f]=80859 2024-03-21T13:28:59.895 INFO:tasks.workunit.client.0.smithi097.stderr:+ ret=0 2024-03-21T13:28:59.895 INFO:tasks.workunit.client.0.smithi097.stderr:+++++ readlink /proc/80660/exe 2024-03-21T13:28:59.895 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T13:28:59.895 INFO:tasks.workunit.client.0.smithi097.stderr:+ export PATH 2024-03-21T13:28:59.895 INFO:tasks.workunit.client.0.smithi097.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:28:59.895 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=79719 2024-03-21T13:28:59.895 INFO:tasks.workunit.client.0.smithi097.stderr:+ wait 79719 2024-03-21T13:28:59.895 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T13:28:59.895 INFO:tasks.workunit.client.0.smithi097.stderr:+ unset rc 2024-03-21T13:28:59.895 INFO:tasks.workunit.client.0.smithi097.stderr:+++++ readlink /proc/80724/exe 2024-03-21T13:28:59.895 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T13:28:59.895 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T13:28:59.895 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T13:28:59.896 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.896 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T13:28:59.896 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.896 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T13:28:59.896 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T13:28:59.896 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.896 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_neorados_ec_list 2024-03-21T13:28:59.896 INFO:tasks.workunit.client.0.smithi097.stderr:++++ basename /usr/bin/bash 2024-03-21T13:28:59.896 INFO:tasks.workunit.client.0.smithi097.stderr:++++ basename /usr/bin/bash 2024-03-21T13:28:59.896 INFO:tasks.workunit.client.0.smithi097.stderr:+ sed 's/^/ ec_list: /' 2024-03-21T13:28:59.896 INFO:tasks.workunit.client.0.smithi097.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_write_operations 2>&1 | tee ceph_test_neorados_write_operations.log | sed "s/^/ write_operations: /"' 2024-03-21T13:28:59.896 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_declare='declare -f' 2024-03-21T13:28:59.896 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_opt=-f 2024-03-21T13:28:59.896 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export which_declare 2024-03-21T13:28:59.896 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export -f which 2024-03-21T13:28:59.896 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset i 2024-03-21T13:28:59.897 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset -f pathmunge 2024-03-21T13:28:59.897 INFO:tasks.workunit.client.0.smithi097.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T13:28:59.897 INFO:tasks.workunit.client.0.smithi097.stderr:+ export PATH 2024-03-21T13:28:59.897 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_declare='declare -f' 2024-03-21T13:28:59.897 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_opt=-f 2024-03-21T13:28:59.897 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T13:28:59.897 INFO:tasks.workunit.client.0.smithi097.stderr:+ unset rc 2024-03-21T13:28:59.897 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export which_declare 2024-03-21T13:28:59.897 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export -f which 2024-03-21T13:28:59.897 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset i 2024-03-21T13:28:59.897 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset -f pathmunge 2024-03-21T13:28:59.897 INFO:tasks.workunit.client.0.smithi097.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T13:28:59.897 INFO:tasks.workunit.client.0.smithi097.stderr:+ export PATH 2024-03-21T13:28:59.897 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T13:28:59.897 INFO:tasks.workunit.client.0.smithi097.stderr:+ unset rc 2024-03-21T13:28:59.897 INFO:tasks.workunit.client.0.smithi097.stderr:+ tee ceph_test_neorados_ec_list.log 2024-03-21T13:28:59.898 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T13:28:59.898 INFO:tasks.workunit.client.0.smithi097.stderr:+ sed 's/^/ pool: /' 2024-03-21T13:28:59.898 INFO:tasks.workunit.client.0.smithi097.stderr:+ sed 's/^/ misc: /' 2024-03-21T13:28:59.898 INFO:tasks.workunit.client.0.smithi097.stderr:+ tee ceph_test_neorados_misc.log 2024-03-21T13:28:59.898 INFO:tasks.workunit.client.0.smithi097.stderr:+ . /etc/bashrc 2024-03-21T13:28:59.898 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -z '' ']' 2024-03-21T13:28:59.898 INFO:tasks.workunit.client.0.smithi097.stderr:++ BASHRCSOURCED=Y 2024-03-21T13:28:59.898 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.898 INFO:tasks.workunit.client.0.smithi097.stderr:++ shopt -q login_shell 2024-03-21T13:28:59.898 INFO:tasks.workunit.client.0.smithi097.stderr:+ tee ceph_test_neorados_pool.log 2024-03-21T13:28:59.898 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_neorados_misc 2024-03-21T13:28:59.898 INFO:tasks.workunit.client.0.smithi097.stderr:+++ umask 2024-03-21T13:28:59.898 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T13:28:59.898 INFO:tasks.workunit.client.0.smithi097.stderr:++ SHELL=/bin/bash 2024-03-21T13:28:59.898 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.899 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T13:28:59.899 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.899 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T13:28:59.899 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T13:28:59.899 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.899 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T13:28:59.899 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.899 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T13:28:59.899 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.899 INFO:tasks.workunit.client.0.smithi097.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T13:28:59.899 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.899 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T13:28:59.899 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.899 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T13:28:59.899 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.900 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T13:28:59.900 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T13:28:59.900 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T13:28:59.900 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.900 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T13:28:59.900 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.900 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T13:28:59.900 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T13:28:59.900 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.900 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T13:28:59.900 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.900 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T13:28:59.900 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.900 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T13:28:59.900 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.900 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T13:28:59.901 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup 2024-03-21T13:28:59.901 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/locale 2024-03-21T13:28:59.901 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T13:28:59.901 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_neorados_pool 2024-03-21T13:28:59.901 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T13:28:59.901 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T13:28:59.901 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.901 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T13:28:59.901 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.901 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T13:28:59.901 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T13:28:59.901 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.901 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T13:28:59.901 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.901 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T13:28:59.901 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.902 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T13:28:59.902 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.902 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T13:28:59.902 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup 2024-03-21T13:28:59.902 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/locale 2024-03-21T13:28:59.902 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.902 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T13:28:59.902 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.902 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T13:28:59.902 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T13:28:59.902 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T13:28:59.902 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T13:28:59.902 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T13:28:59.902 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.902 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T13:28:59.902 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.903 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T13:28:59.903 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T13:28:59.903 INFO:tasks.workunit.client.0.smithi097.stderr:+++ return 2024-03-21T13:28:59.903 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.903 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T13:28:59.903 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.903 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T13:28:59.903 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T13:28:59.903 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export S_COLORS=auto 2024-03-21T13:28:59.903 INFO:tasks.workunit.client.0.smithi097.stderr:+++ S_COLORS=auto 2024-03-21T13:28:59.903 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.903 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T13:28:59.903 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.903 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T13:28:59.903 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.903 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.903 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T13:28:59.904 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.904 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T13:28:59.904 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T13:28:59.904 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T13:28:59.904 INFO:tasks.workunit.client.0.smithi097.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T13:28:59.904 INFO:tasks.workunit.client.0.smithi097.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T13:28:59.904 INFO:tasks.workunit.client.0.smithi097.stderr:++++ LANG=en_US.UTF-8 2024-03-21T13:28:59.904 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.904 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T13:28:59.904 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.904 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG=en_US.UTF-8 2024-03-21T13:28:59.904 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup config 2024-03-21T13:28:59.904 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.904 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.904 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' dumb = linux ']' 2024-03-21T13:28:59.904 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.905 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T13:28:59.905 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.905 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/less.sh 2024-03-21T13:28:59.905 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T13:28:59.905 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.905 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T13:28:59.905 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.905 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T13:28:59.905 INFO:tasks.workunit.client.0.smithi097.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T13:28:59.905 INFO:tasks.workunit.client.0.smithi097.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T13:28:59.905 INFO:tasks.workunit.client.0.smithi097.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T13:28:59.905 INFO:tasks.workunit.client.0.smithi097.stderr:++++ LANG=en_US.UTF-8 2024-03-21T13:28:59.905 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.905 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T13:28:59.905 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.905 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG=en_US.UTF-8 2024-03-21T13:28:59.905 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup config 2024-03-21T13:28:59.906 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.906 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.906 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' dumb = linux ']' 2024-03-21T13:28:59.906 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.906 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T13:28:59.906 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.906 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/less.sh 2024-03-21T13:28:59.906 INFO:tasks.workunit.client.0.smithi097.stderr:+++++ readlink /proc/80760/exe 2024-03-21T13:28:59.906 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T13:28:59.906 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.906 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T13:28:59.906 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.906 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T13:28:59.906 INFO:tasks.workunit.client.0.smithi097.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T13:28:59.906 INFO:tasks.workunit.client.0.smithi097.stderr:+++++ readlink /proc/80801/exe 2024-03-21T13:28:59.907 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T13:28:59.907 INFO:tasks.workunit.client.0.smithi097.stderr:++++ basename /usr/bin/bash 2024-03-21T13:28:59.907 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T13:28:59.907 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T13:28:59.907 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.907 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T13:28:59.907 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.907 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T13:28:59.907 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T13:28:59.907 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.907 INFO:tasks.workunit.client.0.smithi097.stderr:++++ basename /usr/bin/bash 2024-03-21T13:28:59.907 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_declare='declare -f' 2024-03-21T13:28:59.907 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_opt=-f 2024-03-21T13:28:59.907 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export which_declare 2024-03-21T13:28:59.907 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export -f which 2024-03-21T13:28:59.907 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset i 2024-03-21T13:28:59.907 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset -f pathmunge 2024-03-21T13:28:59.908 INFO:tasks.workunit.client.0.smithi097.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T13:28:59.908 INFO:tasks.workunit.client.0.smithi097.stderr:+ export PATH 2024-03-21T13:28:59.908 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T13:28:59.908 INFO:tasks.workunit.client.0.smithi097.stderr:+ unset rc 2024-03-21T13:28:59.908 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_neorados_read_operations 2024-03-21T13:28:59.908 INFO:tasks.workunit.client.0.smithi097.stderr:+ sed 's/^/ read_operations: /' 2024-03-21T13:28:59.908 INFO:tasks.workunit.client.0.smithi097.stderr:+ tee ceph_test_neorados_read_operations.log 2024-03-21T13:28:59.908 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T13:28:59.908 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T13:28:59.908 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T13:28:59.908 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.908 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T13:28:59.908 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.908 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T13:28:59.908 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T13:28:59.908 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.909 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T13:28:59.909 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.909 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T13:28:59.909 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.909 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T13:28:59.909 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.909 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T13:28:59.909 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup 2024-03-21T13:28:59.909 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_declare='declare -f' 2024-03-21T13:28:59.909 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_opt=-f 2024-03-21T13:28:59.909 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export which_declare 2024-03-21T13:28:59.909 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export -f which 2024-03-21T13:28:59.909 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/locale 2024-03-21T13:28:59.909 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset i 2024-03-21T13:28:59.909 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset -f pathmunge 2024-03-21T13:28:59.910 INFO:tasks.workunit.client.0.smithi097.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T13:28:59.910 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T13:28:59.910 INFO:tasks.workunit.client.0.smithi097.stderr:+ export PATH 2024-03-21T13:28:59.910 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T13:28:59.910 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T13:28:59.910 INFO:tasks.workunit.client.0.smithi097.stderr:+ unset rc 2024-03-21T13:28:59.910 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T13:28:59.910 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.910 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T13:28:59.910 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.910 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T13:28:59.910 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T13:28:59.910 INFO:tasks.workunit.client.0.smithi097.stderr:+++ return 2024-03-21T13:28:59.910 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.910 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T13:28:59.911 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.911 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T13:28:59.911 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export S_COLORS=auto 2024-03-21T13:28:59.911 INFO:tasks.workunit.client.0.smithi097.stderr:+++ S_COLORS=auto 2024-03-21T13:28:59.911 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.911 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T13:28:59.911 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.911 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T13:28:59.911 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_neorados_snapshots 2024-03-21T13:28:59.911 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.911 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.911 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T13:28:59.911 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.911 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T13:28:59.911 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T13:28:59.912 INFO:tasks.workunit.client.0.smithi097.stderr:+ sed 's/^/ snapshots: /' 2024-03-21T13:28:59.912 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T13:28:59.912 INFO:tasks.workunit.client.0.smithi097.stderr:+ tee ceph_test_neorados_snapshots.log 2024-03-21T13:28:59.912 INFO:tasks.workunit.client.0.smithi097.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T13:28:59.912 INFO:tasks.workunit.client.0.smithi097.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T13:28:59.912 INFO:tasks.workunit.client.0.smithi097.stderr:++++ LANG=en_US.UTF-8 2024-03-21T13:28:59.912 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.912 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T13:28:59.912 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.912 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG=en_US.UTF-8 2024-03-21T13:28:59.912 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup config 2024-03-21T13:28:59.912 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.912 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.912 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' dumb = linux ']' 2024-03-21T13:28:59.912 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.912 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T13:28:59.913 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.913 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/less.sh 2024-03-21T13:28:59.913 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T13:28:59.913 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.913 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T13:28:59.913 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.913 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T13:28:59.913 INFO:tasks.workunit.client.0.smithi097.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T13:28:59.913 INFO:tasks.workunit.client.0.smithi097.stderr:+++++ readlink /proc/80859/exe 2024-03-21T13:28:59.913 INFO:tasks.workunit.client.0.smithi097.stderr:++++ basename /usr/bin/bash 2024-03-21T13:28:59.913 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_declare='declare -f' 2024-03-21T13:28:59.913 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_opt=-f 2024-03-21T13:28:59.913 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export which_declare 2024-03-21T13:28:59.913 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export -f which 2024-03-21T13:28:59.913 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset i 2024-03-21T13:28:59.913 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset -f pathmunge 2024-03-21T13:28:59.913 INFO:tasks.workunit.client.0.smithi097.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T13:28:59.914 INFO:tasks.workunit.client.0.smithi097.stderr:+ export PATH 2024-03-21T13:28:59.914 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T13:28:59.914 INFO:tasks.workunit.client.0.smithi097.stderr:+ unset rc 2024-03-21T13:28:59.914 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_neorados_write_operations 2024-03-21T13:28:59.914 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T13:28:59.914 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T13:28:59.914 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T13:28:59.914 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.914 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T13:28:59.914 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.914 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T13:28:59.914 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T13:28:59.914 INFO:tasks.workunit.client.0.smithi097.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T13:28:59.914 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T13:28:59.914 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T13:28:59.914 INFO:tasks.workunit.client.0.smithi097.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T13:28:59.915 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.915 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T13:28:59.915 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.915 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T13:28:59.915 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T13:28:59.915 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.915 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T13:28:59.915 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.915 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T13:28:59.915 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.915 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T13:28:59.915 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.915 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T13:28:59.915 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup 2024-03-21T13:28:59.915 INFO:tasks.workunit.client.0.smithi097.stderr:+ tee ceph_test_neorados_write_operations.log 2024-03-21T13:28:59.915 INFO:tasks.workunit.client.0.smithi097.stderr:+ sed 's/^/ write_operations: /' 2024-03-21T13:28:59.916 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/locale 2024-03-21T13:28:59.916 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.916 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T13:28:59.916 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.916 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T13:28:59.916 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T13:28:59.916 INFO:tasks.workunit.client.0.smithi097.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T13:28:59.916 INFO:tasks.workunit.client.0.smithi097.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T13:28:59.916 INFO:tasks.workunit.client.0.smithi097.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T13:28:59.916 INFO:tasks.workunit.client.0.smithi097.stderr:++++ LANG=en_US.UTF-8 2024-03-21T13:28:59.916 INFO:tasks.workunit.client.0.smithi097.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T13:28:59.916 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T13:28:59.916 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.916 INFO:tasks.workunit.client.0.smithi097.stderr:+++ LANG=en_US.UTF-8 2024-03-21T13:28:59.916 INFO:tasks.workunit.client.0.smithi097.stderr:+++ unset LANG_backup config 2024-03-21T13:28:59.916 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n '' ']' 2024-03-21T13:28:59.917 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T13:28:59.917 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' dumb = linux ']' 2024-03-21T13:28:59.917 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.917 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T13:28:59.917 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.917 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/less.sh 2024-03-21T13:28:59.917 INFO:tasks.workunit.client.0.smithi097.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T13:28:59.917 INFO:tasks.workunit.client.0.smithi097.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T13:28:59.917 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T13:28:59.917 INFO:tasks.workunit.client.0.smithi097.stderr:++ '[' '' ']' 2024-03-21T13:28:59.917 INFO:tasks.workunit.client.0.smithi097.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T13:28:59.917 INFO:tasks.workunit.client.0.smithi097.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T13:28:59.917 INFO:tasks.workunit.client.0.smithi097.stderr:+++++ readlink /proc/80832/exe 2024-03-21T13:28:59.917 INFO:tasks.workunit.client.0.smithi097.stderr:++++ basename /usr/bin/bash 2024-03-21T13:28:59.917 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_declare='declare -f' 2024-03-21T13:28:59.917 INFO:tasks.workunit.client.0.smithi097.stderr:+++ which_opt=-f 2024-03-21T13:28:59.918 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export which_declare 2024-03-21T13:28:59.918 INFO:tasks.workunit.client.0.smithi097.stderr:+++ export -f which 2024-03-21T13:28:59.918 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset i 2024-03-21T13:28:59.918 INFO:tasks.workunit.client.0.smithi097.stderr:++ unset -f pathmunge 2024-03-21T13:28:59.918 INFO:tasks.workunit.client.0.smithi097.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T13:28:59.918 INFO:tasks.workunit.client.0.smithi097.stderr:+ export PATH 2024-03-21T13:28:59.918 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T13:28:59.918 INFO:tasks.workunit.client.0.smithi097.stderr:+ unset rc 2024-03-21T13:28:59.918 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph_test_neorados_watch_notify 2024-03-21T13:28:59.918 INFO:tasks.workunit.client.0.smithi097.stderr:+ tee ceph_test_neorados_watch_notify.log 2024-03-21T13:28:59.918 INFO:tasks.workunit.client.0.smithi097.stderr:+ sed 's/^/ watch_notify: /' 2024-03-21T13:29:00.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:00 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3780032783' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-03-21T13:29:00.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:00 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3605606612' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi097-80861-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:00.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:00 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3780032783' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-03-21T13:29:00.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:00 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3605606612' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi097-80861-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:00.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:00 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3780032783' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-03-21T13:29:00.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:00 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3605606612' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi097-80861-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:02.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:29:01 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:29:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:29:03.871 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:29:03 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:29:08.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:08 smithi097 ceph-mon[30040]: pgmap v67: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:29:08.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:08 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4203963449' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi097-79675-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:08.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:08 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3213135837' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi097-80712-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:08.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:08 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4046544770' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi097-79618-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:08.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:08 smithi097 ceph-mon[30040]: pgmap v69: 260 pgs: 128 unknown, 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:29:08.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:08 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:29:08.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:08 smithi097 ceph-mon[30040]: pgmap v70: 260 pgs: 66 creating+peering, 132 active+clean, 62 unknown; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:29:08.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:08 smithi097 ceph-mon[30040]: osdmap e72: 8 total, 8 up, 8 in 2024-03-21T13:29:08.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:08 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi097-80861-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:08.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:08 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi097-79675-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:08.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:08 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi097-80712-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:08.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:08 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi097-79618-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:08.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:08 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2306887523' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi097-79664-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:08.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:08 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/532200677' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi097-79636-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:08.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:08 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2555609167' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi097-80028-1"}]: dispatch 2024-03-21T13:29:08.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:08 smithi097 ceph-mon[39416]: pgmap v67: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:29:08.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:08 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4203963449' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi097-79675-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:08.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:08 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3213135837' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi097-80712-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:08.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:08 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4046544770' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi097-79618-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:08.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:08 smithi097 ceph-mon[39416]: pgmap v69: 260 pgs: 128 unknown, 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:29:08.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:08 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:29:08.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:08 smithi097 ceph-mon[39416]: pgmap v70: 260 pgs: 66 creating+peering, 132 active+clean, 62 unknown; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:29:08.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:08 smithi097 ceph-mon[39416]: osdmap e72: 8 total, 8 up, 8 in 2024-03-21T13:29:08.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:08 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi097-80861-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:08.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:08 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi097-79675-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:08.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:08 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi097-80712-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:08.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:08 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi097-79618-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:08.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:08 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2306887523' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi097-79664-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:08.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:08 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/532200677' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi097-79636-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:08.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:08 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2555609167' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi097-80028-1"}]: dispatch 2024-03-21T13:29:08.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:08 smithi144 ceph-mon[33633]: pgmap v67: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:29:08.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:08 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4203963449' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi097-79675-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:08.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:08 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3213135837' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi097-80712-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:08.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:08 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4046544770' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi097-79618-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:08.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:08 smithi144 ceph-mon[33633]: pgmap v69: 260 pgs: 128 unknown, 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:29:08.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:08 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:29:08.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:08 smithi144 ceph-mon[33633]: pgmap v70: 260 pgs: 66 creating+peering, 132 active+clean, 62 unknown; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:29:08.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:08 smithi144 ceph-mon[33633]: osdmap e72: 8 total, 8 up, 8 in 2024-03-21T13:29:08.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:08 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi097-80861-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:08.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:08 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi097-79675-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:08.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:08 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi097-80712-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:08.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:08 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi097-79618-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:08.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:08 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2306887523' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi097-79664-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:08.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:08 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/532200677' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi097-79636-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:08.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:08 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2555609167' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi097-80028-1"}]: dispatch 2024-03-21T13:29:09.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3394100719' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi097-79906-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/580556212' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi097-80014-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1171752757' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi097-80039-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2508049812' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi097-80000-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4280506793' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80370-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3422212324' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi097-80011-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3548820191' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80187-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/854518831' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi097-79992-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi097-79910-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4223951013' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi097-79828-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4071170553' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80165-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi097-79710-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[39416]: pgmap v72: 964 pgs: 66 creating+peering, 132 active+clean, 766 unknown; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-21T13:29:09.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:29:09.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[39416]: osdmap e73: 8 total, 8 up, 8 in 2024-03-21T13:29:09.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[39416]: Health check failed: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:29:09.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi097-79906-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi097-79992-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3394100719' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi097-79906-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/580556212' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi097-80014-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1171752757' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi097-80039-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2508049812' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi097-80000-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4280506793' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80370-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3422212324' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi097-80011-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3548820191' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80187-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/854518831' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi097-79992-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi097-79910-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4223951013' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi097-79828-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.557 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4071170553' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80165-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.557 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi097-79710-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.557 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[30040]: pgmap v72: 964 pgs: 66 creating+peering, 132 active+clean, 766 unknown; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-21T13:29:09.557 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:29:09.557 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[30040]: osdmap e73: 8 total, 8 up, 8 in 2024-03-21T13:29:09.557 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[30040]: Health check failed: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:29:09.557 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi097-79906-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.557 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi097-79992-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.557 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2555609167' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi097-80028-1"}]: dispatch 2024-03-21T13:29:09.557 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi097-80014-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.557 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi097-80039-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.557 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi097-79828-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.557 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi097-80000-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.557 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80370-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.558 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80165-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.558 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi097-80011-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.558 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi097-79710-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.558 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80187-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.558 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi097-79910-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.558 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi097-80861-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:29:09.558 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi097-79675-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:09.558 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi097-80712-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:29:09.558 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi097-79618-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:09.558 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2306887523' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi097-79664-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:09.558 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/532200677' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi097-79636-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:09.558 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3605606612' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi097-80861-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi097-80861-1"}]: dispatch 2024-03-21T13:29:09.558 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3213135837' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi097-80712-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi097-80712-1"}]: dispatch 2024-03-21T13:29:09.558 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[30040]: osdmap e74: 8 total, 8 up, 8 in 2024-03-21T13:29:09.559 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/453786208' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-03-21T13:29:09.559 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi097-80861-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi097-80861-1"}]: dispatch 2024-03-21T13:29:09.559 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1124128008' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.559 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2555609167' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi097-80028-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:09.559 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi097-80712-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi097-80712-1"}]: dispatch 2024-03-21T13:29:09.559 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3130993506' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi097-79740-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.559 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2920807904' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.559 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2881128302' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi097-80404-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.559 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.559 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2555609167' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi097-80028-1"}]: dispatch 2024-03-21T13:29:09.559 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi097-80014-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.559 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi097-80039-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.559 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi097-79828-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.560 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi097-80000-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.560 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80370-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.560 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80165-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.560 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi097-80011-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.560 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi097-79710-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.560 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80187-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.560 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi097-79910-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.560 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi097-80861-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:29:09.560 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi097-79675-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:09.560 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi097-80712-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:29:09.560 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi097-79618-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:09.560 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2306887523' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi097-79664-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:09.560 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/532200677' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi097-79636-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:09.560 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3605606612' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi097-80861-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi097-80861-1"}]: dispatch 2024-03-21T13:29:09.561 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3213135837' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi097-80712-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi097-80712-1"}]: dispatch 2024-03-21T13:29:09.561 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[39416]: osdmap e74: 8 total, 8 up, 8 in 2024-03-21T13:29:09.561 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/453786208' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-03-21T13:29:09.561 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi097-80861-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi097-80861-1"}]: dispatch 2024-03-21T13:29:09.561 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1124128008' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.561 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2555609167' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi097-80028-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:09.561 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi097-80712-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi097-80712-1"}]: dispatch 2024-03-21T13:29:09.561 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3130993506' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi097-79740-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.561 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2920807904' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.561 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2881128302' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi097-80404-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.561 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:09 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:09 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3394100719' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi097-79906-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:09 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/580556212' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi097-80014-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:09 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1171752757' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi097-80039-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:09 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2508049812' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi097-80000-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:09 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4280506793' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80370-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:09 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3422212324' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi097-80011-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:09 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3548820191' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80187-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:09 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/854518831' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi097-79992-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:09 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi097-79910-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:09 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4223951013' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi097-79828-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:09 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4071170553' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80165-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:09 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi097-79710-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:09 smithi144 ceph-mon[33633]: pgmap v72: 964 pgs: 66 creating+peering, 132 active+clean, 766 unknown; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-21T13:29:09.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:09 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:29:09.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:09 smithi144 ceph-mon[33633]: osdmap e73: 8 total, 8 up, 8 in 2024-03-21T13:29:09.625 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:09 smithi144 ceph-mon[33633]: Health check failed: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:29:09.626 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:09 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi097-79906-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.626 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:09 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi097-79992-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.626 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:09 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2555609167' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi097-80028-1"}]: dispatch 2024-03-21T13:29:09.626 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:09 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi097-80014-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.626 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:09 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi097-80039-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.626 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:09 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi097-79828-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.626 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:09 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi097-80000-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.627 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:09 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80370-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.627 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:09 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80165-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.627 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:09 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi097-80011-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.627 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:09 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi097-79710-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.627 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:09 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80187-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.627 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:09 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi097-79910-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.627 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:09 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi097-80861-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:29:09.627 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:09 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi097-79675-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:09.627 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:09 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi097-80712-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:29:09.627 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:09 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi097-79618-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:09.627 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:09 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2306887523' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi097-79664-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:09.627 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:09 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/532200677' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi097-79636-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:09.627 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:09 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3605606612' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi097-80861-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi097-80861-1"}]: dispatch 2024-03-21T13:29:09.627 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:09 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3213135837' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi097-80712-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi097-80712-1"}]: dispatch 2024-03-21T13:29:09.628 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:09 smithi144 ceph-mon[33633]: osdmap e74: 8 total, 8 up, 8 in 2024-03-21T13:29:09.628 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:09 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/453786208' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-03-21T13:29:09.628 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:09 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi097-80861-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi097-80861-1"}]: dispatch 2024-03-21T13:29:09.628 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:09 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1124128008' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.628 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:09 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2555609167' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi097-80028-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:09.628 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:09 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi097-80712-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi097-80712-1"}]: dispatch 2024-03-21T13:29:09.628 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:09 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3130993506' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi097-79740-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.628 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:09 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2920807904' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.628 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:09 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2881128302' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi097-80404-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.628 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:09 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:09.643 INFO:tasks.workunit.client.0.smithi097.stdout: api_asio: [==========] Running 12 tests from 1 test suite. 2024-03-21T13:29:09.643 INFO:tasks.workunit.client.0.smithi097.stdout: api_asio: [----------] Global test environment set-up. 2024-03-21T13:29:09.644 INFO:tasks.workunit.client.0.smithi097.stdout: api_asio: [----------] 12 tests from AsioRados 2024-03-21T13:29:09.644 INFO:tasks.workunit.client.0.smithi097.stdout: api_asio: [ RUN ] AsioRados.AsyncReadCallback 2024-03-21T13:29:09.644 INFO:tasks.workunit.client.0.smithi097.stdout: api_asio: [ OK ] AsioRados.AsyncReadCallback (2 ms) 2024-03-21T13:29:09.644 INFO:tasks.workunit.client.0.smithi097.stdout: api_asio: [ RUN ] AsioRados.AsyncReadFuture 2024-03-21T13:29:09.644 INFO:tasks.workunit.client.0.smithi097.stdout: api_asio: [ OK ] AsioRados.AsyncReadFuture (1 ms) 2024-03-21T13:29:09.644 INFO:tasks.workunit.client.0.smithi097.stdout: api_asio: [ RUN ] AsioRados.AsyncReadYield 2024-03-21T13:29:09.644 INFO:tasks.workunit.client.0.smithi097.stdout: api_asio: [ OK ] AsioRados.AsyncReadYield (1 ms) 2024-03-21T13:29:09.644 INFO:tasks.workunit.client.0.smithi097.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteCallback 2024-03-21T13:29:09.644 INFO:tasks.workunit.client.0.smithi097.stdout: api_asio: [ OK ] AsioRados.AsyncWriteCallback (11 ms) 2024-03-21T13:29:09.644 INFO:tasks.workunit.client.0.smithi097.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteFuture 2024-03-21T13:29:09.644 INFO:tasks.workunit.client.0.smithi097.stdout: api_asio: [ OK ] AsioRados.AsyncWriteFuture (3 ms) 2024-03-21T13:29:09.644 INFO:tasks.workunit.client.0.smithi097.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteYield 2024-03-21T13:29:09.644 INFO:tasks.workunit.client.0.smithi097.stdout: api_asio: [ OK ] AsioRados.AsyncWriteYield (3 ms) 2024-03-21T13:29:09.644 INFO:tasks.workunit.client.0.smithi097.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationCallback 2024-03-21T13:29:09.644 INFO:tasks.workunit.client.0.smithi097.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationCallback (2 ms) 2024-03-21T13:29:09.645 INFO:tasks.workunit.client.0.smithi097.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationFuture 2024-03-21T13:29:09.645 INFO:tasks.workunit.client.0.smithi097.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationFuture (3 ms) 2024-03-21T13:29:09.645 INFO:tasks.workunit.client.0.smithi097.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationYield 2024-03-21T13:29:09.645 INFO:tasks.workunit.client.0.smithi097.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationYield (1 ms) 2024-03-21T13:29:09.645 INFO:tasks.workunit.client.0.smithi097.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationCallback 2024-03-21T13:29:09.645 INFO:tasks.workunit.client.0.smithi097.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationCallback (3 ms) 2024-03-21T13:29:09.645 INFO:tasks.workunit.client.0.smithi097.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationFuture 2024-03-21T13:29:09.645 INFO:tasks.workunit.client.0.smithi097.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationFuture (3 ms) 2024-03-21T13:29:09.645 INFO:tasks.workunit.client.0.smithi097.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationYield 2024-03-21T13:29:09.645 INFO:tasks.workunit.client.0.smithi097.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationYield (2 ms) 2024-03-21T13:29:09.645 INFO:tasks.workunit.client.0.smithi097.stdout: api_asio: [----------] 12 tests from AsioRados (35 ms total) 2024-03-21T13:29:09.645 INFO:tasks.workunit.client.0.smithi097.stdout: api_asio: 2024-03-21T13:29:09.645 INFO:tasks.workunit.client.0.smithi097.stdout: api_asio: [----------] Global test environment tear-down 2024-03-21T13:29:09.645 INFO:tasks.workunit.client.0.smithi097.stdout: api_asio: [==========] 12 tests from 1 test suite ran. (10155 ms total) 2024-03-21T13:29:09.645 INFO:tasks.workunit.client.0.smithi097.stdout: api_asio: [ PASSED ] 12 tests. 2024-03-21T13:29:09.980 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: [==========] Running 11 tests from 3 test suites. 2024-03-21T13:29:09.980 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: [----------] Global test environment set-up. 2024-03-21T13:29:09.980 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: [----------] 7 tests from LibRadosList 2024-03-21T13:29:09.980 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: [ RUN ] LibRadosList.ListObjects 2024-03-21T13:29:09.980 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: [ OK ] LibRadosList.ListObjects (438 ms) 2024-03-21T13:29:09.980 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: [ RUN ] LibRadosList.ListObjectsZeroInName 2024-03-21T13:29:09.981 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: [ OK ] LibRadosList.ListObjectsZeroInName (37 ms) 2024-03-21T13:29:09.981 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: [ RUN ] LibRadosList.ListObjectsNS 2024-03-21T13:29:09.981 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: myset foo1,foo2,foo3 2024-03-21T13:29:09.981 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: foo1 2024-03-21T13:29:09.981 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: foo2 2024-03-21T13:29:09.981 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: foo3 2024-03-21T13:29:09.981 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: myset foo1,foo4,foo5 2024-03-21T13:29:09.981 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: foo4 2024-03-21T13:29:09.981 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: foo5 2024-03-21T13:29:09.981 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: foo1 2024-03-21T13:29:09.981 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: myset foo6,foo7 2024-03-21T13:29:09.981 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: foo7 2024-03-21T13:29:09.981 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: foo6 2024-03-21T13:29:09.981 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-03-21T13:29:09.981 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: ns1:foo4 2024-03-21T13:29:09.982 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: ns1:foo5 2024-03-21T13:29:09.982 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: ns2:foo7 2024-03-21T13:29:09.982 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: ns2:foo6 2024-03-21T13:29:09.982 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: ns1:foo1 2024-03-21T13:29:09.982 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: :foo1 2024-03-21T13:29:09.982 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: :foo2 2024-03-21T13:29:09.982 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: :foo3 2024-03-21T13:29:09.982 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: [ OK ] LibRadosList.ListObjectsNS (86 ms) 2024-03-21T13:29:09.982 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: [ RUN ] LibRadosList.ListObjectsStart 2024-03-21T13:29:09.982 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: 1 0 2024-03-21T13:29:09.983 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: 10 0 2024-03-21T13:29:09.983 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: 13 0 2024-03-21T13:29:09.983 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: 7 0 2024-03-21T13:29:09.983 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: 14 0 2024-03-21T13:29:09.983 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: 0 0 2024-03-21T13:29:09.983 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: 15 0 2024-03-21T13:29:09.983 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: 11 0 2024-03-21T13:29:09.983 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: 5 0 2024-03-21T13:29:09.983 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: 8 0 2024-03-21T13:29:09.983 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: 6 0 2024-03-21T13:29:09.983 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: 3 0 2024-03-21T13:29:09.983 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: 4 0 2024-03-21T13:29:09.983 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: 12 0 2024-03-21T13:29:09.983 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: 9 0 2024-03-21T13:29:09.983 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: 2 0 2024-03-21T13:29:09.984 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-03-21T13:29:09.984 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: [ OK ] LibRadosList.ListObjectsStart (65 ms) 2024-03-21T13:29:09.984 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: [ RUN ] LibRadosList.ListObjectsCursor 2024-03-21T13:29:09.984 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: x cursor=MIN 2024-03-21T13:29:09.984 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: > oid=1 cursor=14:02547ec2:::1:head 2024-03-21T13:29:09.984 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: > oid=10 cursor=14:52ea6a34:::10:head 2024-03-21T13:29:09.984 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: > oid=13 cursor=14:566253c9:::13:head 2024-03-21T13:29:09.984 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: > oid=7 cursor=14:5c6b0b28:::7:head 2024-03-21T13:29:09.984 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: > oid=14 cursor=14:62a1935d:::14:head 2024-03-21T13:29:09.984 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: > oid=0 cursor=14:6cac518f:::0:head 2024-03-21T13:29:09.984 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: > oid=15 cursor=14:863748b0:::15:head 2024-03-21T13:29:09.985 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: > oid=11 cursor=14:89d3ae78:::11:head 2024-03-21T13:29:09.985 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: > oid=5 cursor=14:b29083e3:::5:head 2024-03-21T13:29:09.985 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: > oid=8 cursor=14:bd63b0f1:::8:head 2024-03-21T13:29:09.985 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: > oid=6 cursor=14:c4fdafeb:::6:head 2024-03-21T13:29:09.985 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: > oid=3 cursor=14:cfc208b3:::3:head 2024-03-21T13:29:09.985 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: > oid=4 cursor=14:d83876eb:::4:head 2024-03-21T13:29:09.985 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: > oid=12 cursor=14:de5d7c5f:::12:head 2024-03-21T13:29:09.985 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: > oid=9 cursor=14:e960b815:::9:head 2024-03-21T13:29:09.985 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: > oid=2 cursor=14:f905c69b:::2:head 2024-03-21T13:29:09.985 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: FIRST> seek to MIN oid=1 2024-03-21T13:29:09.985 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : oid=1 cursor=14:02547ec2:::1:head 2024-03-21T13:29:09.985 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : seek to 14:02547ec2:::1:head 2024-03-21T13:29:09.985 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: > 14:02547ec2:::1:head -> 1 2024-03-21T13:29:09.985 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : oid=10 cursor=14:52ea6a34:::10:head 2024-03-21T13:29:09.985 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : seek to 14:52ea6a34:::10:head 2024-03-21T13:29:09.986 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: > 14:52ea6a34:::10:head -> 10 2024-03-21T13:29:09.986 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : oid=13 cursor=14:566253c9:::13:head 2024-03-21T13:29:09.986 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : seek to 14:566253c9:::13:head 2024-03-21T13:29:09.986 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: > 14:566253c9:::13:head -> 13 2024-03-21T13:29:09.986 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : oid=7 cursor=14:5c6b0b28:::7:head 2024-03-21T13:29:09.986 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : seek to 14:5c6b0b28:::7:head 2024-03-21T13:29:09.986 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: > 14:5c6b0b28:::7:head -> 7 2024-03-21T13:29:10.053 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : oid=14 cursor=14:62a1935d:::14:head 2024-03-21T13:29:10.054 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : seek to 14:62a1935d:::14:head 2024-03-21T13:29:10.054 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: > 14:62a1935d:::14:head -> 14 2024-03-21T13:29:10.054 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : oid=0 cursor=14:6cac518f:::0:head 2024-03-21T13:29:10.054 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : seek to 14:6cac518f:::0:head 2024-03-21T13:29:10.054 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: > 14:6cac518f:::0:head -> 0 2024-03-21T13:29:10.054 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : oid=15 cursor=14:863748b0:::15:head 2024-03-21T13:29:10.054 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : seek to 14:863748b0:::15:head 2024-03-21T13:29:10.054 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: > 14:863748b0:::15:head -> 15 2024-03-21T13:29:10.055 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : oid=11 cursor=14:89d3ae78:::11:head 2024-03-21T13:29:10.055 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : seek to 14:89d3ae78:::11:head 2024-03-21T13:29:10.055 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: > 14:89d3ae78:::11:head -> 11 2024-03-21T13:29:10.055 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : oid=5 cursor=14:b29083e3:::5:head 2024-03-21T13:29:10.055 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : seek to 14:b29083e3:::5:head 2024-03-21T13:29:10.055 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: > 14:b29083e3:::5:head -> 5 2024-03-21T13:29:10.055 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : oid=8 cursor=14:bd63b0f1:::8:head 2024-03-21T13:29:10.055 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : seek to 14:bd63b0f1:::8:head 2024-03-21T13:29:10.055 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: > 14:bd63b0f1:::8:head -> 8 2024-03-21T13:29:10.055 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : oid=6 cursor=14:c4fdafeb:::6:head 2024-03-21T13:29:10.055 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : seek to 14:c4fdafeb:::6:head 2024-03-21T13:29:10.055 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: > 14:c4fdafeb:::6:head -> 6 2024-03-21T13:29:10.055 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : oid=3 cursor=14:cfc208b3:::3:head 2024-03-21T13:29:10.055 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : seek to 14:cfc208b3:::3:head 2024-03-21T13:29:10.056 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: > 14:cfc208b3:::3:head -> 3 2024-03-21T13:29:10.056 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : oid=4 cursor=14:d83876eb:::4:head 2024-03-21T13:29:10.056 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : seek to 14:d83876eb:::4:head 2024-03-21T13:29:10.056 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: > 14:d83876eb:::4:head -> 4 2024-03-21T13:29:10.056 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : oid=12 cursor=14:de5d7c5f:::12:head 2024-03-21T13:29:10.056 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : seek to 14:de5d7c5f:::12:head 2024-03-21T13:29:10.056 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: > 14:de5d7c5f:::12:head -> 12 2024-03-21T13:29:10.056 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : oid=9 cursor=14:e960b815:::9:head 2024-03-21T13:29:10.056 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : seek to 14:e960b815:::9:head 2024-03-21T13:29:10.056 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: > 14:e960b815:::9:head -> 9 2024-03-21T13:29:10.056 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : oid=2 cursor=14:f905c69b:::2:head 2024-03-21T13:29:10.056 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : seek to 14:f905c69b:::2:head 2024-03-21T13:29:10.056 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: > 14:f905c69b:::2:head -> 2 2024-03-21T13:29:10.056 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : seek to 14:de5d7c5f:::12:head 2024-03-21T13:29:10.056 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : cursor()=14:de5d7c5f:::12:head expected=14:de5d7c5f:::12:head 2024-03-21T13:29:10.057 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: > 14:de5d7c5f:::12:head -> 12 2024-03-21T13:29:10.057 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : entry=12 expected=12 2024-03-21T13:29:10.057 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : seek to 14:d83876eb:::4:head 2024-03-21T13:29:10.057 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : cursor()=14:d83876eb:::4:head expected=14:d83876eb:::4:head 2024-03-21T13:29:10.057 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: > 14:d83876eb:::4:head -> 4 2024-03-21T13:29:10.057 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : entry=4 expected=4 2024-03-21T13:29:10.057 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : seek to 14:cfc208b3:::3:head 2024-03-21T13:29:10.057 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : cursor()=14:cfc208b3:::3:head expected=14:cfc208b3:::3:head 2024-03-21T13:29:10.057 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: > 14:cfc208b3:::3:head -> 3 2024-03-21T13:29:10.057 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : entry=3 expected=3 2024-03-21T13:29:10.057 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : seek to 14:02547ec2:::1:head 2024-03-21T13:29:10.057 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : cursor()=14:02547ec2:::1:head expected=14:02547ec2:::1:head 2024-03-21T13:29:10.057 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: > 14:02547ec2:::1:head -> 1 2024-03-21T13:29:10.057 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : entry=1 expected=1 2024-03-21T13:29:10.057 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : seek to 14:52ea6a34:::10:head 2024-03-21T13:29:10.058 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : cursor()=14:52ea6a34:::10:head expected=14:52ea6a34:::10:head 2024-03-21T13:29:10.058 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: > 14:52ea6a34:::10:head -> 10 2024-03-21T13:29:10.058 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : entry=10 expected=10 2024-03-21T13:29:10.058 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : seek to 14:c4fdafeb:::6:head 2024-03-21T13:29:10.058 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : cursor()=14:c4fdafeb:::6:head expected=14:c4fdafeb:::6:head 2024-03-21T13:29:10.058 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: > 14:c4fdafeb:::6:head -> 6 2024-03-21T13:29:10.058 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : entry=6 expected=6 2024-03-21T13:29:10.058 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : seek to 14:bd63b0f1:::8:head 2024-03-21T13:29:10.058 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : cursor()=14:bd63b0f1:::8:head expected=14:bd63b0f1:::8:head 2024-03-21T13:29:10.059 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: > 14:bd63b0f1:::8:head -> 8 2024-03-21T13:29:10.059 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : entry=8 expected=8 2024-03-21T13:29:10.059 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : seek to 14:b29083e3:::5:head 2024-03-21T13:29:10.059 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : cursor()=14:b29083e3:::5:head expected=14:b29083e3:::5:head 2024-03-21T13:29:10.059 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: > 14:b29083e3:::5:head -> 5 2024-03-21T13:29:10.298 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : entry=5 list_parallel: process_1_[80536]: starting. 2024-03-21T13:29:10.298 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_1_[80536]: creating pool ceph_test_rados_list_parallel.smithi097-80461 2024-03-21T13:29:10.298 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_1_[80536]: created object 0... 2024-03-21T13:29:10.298 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_1_[80536]: created object 25... 2024-03-21T13:29:10.298 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_1_[80536]: created object 49... 2024-03-21T13:29:10.298 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_1_[80536]: finishing. 2024-03-21T13:29:10.299 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_1_[80536]: shutting down. 2024-03-21T13:29:10.299 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_2_[80538]: starting. 2024-03-21T13:29:10.299 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_2_[80538]: listing objects. 2024-03-21T13:29:10.299 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_2_[80538]: listed object 0... 2024-03-21T13:29:10.299 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_2_[80538]: listed object 25... 2024-03-21T13:29:10.299 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_2_[80538]: saw 50 objects 2024-03-21T13:29:10.299 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_2_[80538]: shutting down. 2024-03-21T13:29:10.299 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: ******************************* 2024-03-21T13:29:10.299 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_3_[81599]: starting. 2024-03-21T13:29:10.299 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_3_[81599]: creating pool ceph_test_rados_list_parallel.smithi097-80461 2024-03-21T13:29:10.299 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_3_[81599]: created object 0... 2024-03-21T13:29:10.299 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_3_[81599]: created object 25... 2024-03-21T13:29:10.299 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_3_[81599]: created object 49... 2024-03-21T13:29:10.300 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_3_[81599]: finishing. 2024-03-21T13:29:10.300 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_3_[81599]: shutting down. 2024-03-21T13:29:10.300 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: ******************************* 2024-03-21T13:29:10.300 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_4_[81600]: starting. 2024-03-21T13:29:10.300 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_4_[81600]: listing objects. 2024-03-21T13:29:10.300 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_4_[81600]: listed object 0... 2024-03-21T13:29:10.300 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_4_[81600]: listed object 25... 2024-03-21T13:29:10.300 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_4_[81600]: saw 42 objects 2024-03-21T13:29:10.300 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_4_[81600]: shutting down. 2024-03-21T13:29:10.300 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: ******************************* 2024-03-21T13:29:10.300 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_5_[81601]: starting. 2024-03-21T13:29:10.301 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_5_[81601]: removed 25 objects... 2024-03-21T13:29:10.301 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_5_[81601]: removed half of the objects 2024-03-21T13:29:10.301 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_5_[81601]: removed 50 objects... 2024-03-21T13:29:10.301 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_5_[81601]: removed 50 objects 2024-03-21T13:29:10.301 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_5_[81601]: shutting down. 2024-03-21T13:29:10.301 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: ******************************* 2024-03-21T13:29:10.301 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: ******************************* 2024-03-21T13:29:10.301 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_6_[81698]: starting. 2024-03-21T13:29:10.301 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_6_[81698]: creating pool ceph_test_rados_list_parallel.smithi097-80461 2024-03-21T13:29:10.301 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_6_[81698]: created object 0... 2024-03-21T13:29:10.301 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_6_[81698]: created object 25... 2024-03-21T13:29:10.301 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_6_[81698]: created object 49... 2024-03-21T13:29:10.301 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_6_[81698]: finishing. 2024-03-21T13:29:10.302 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_6_[81698]: shutting down. 2024-03-21T13:29:10.302 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: ******************************* 2024-03-21T13:29:10.302 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: ******************************* 2024-03-21T13:29:10.302 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_7_[81699]: starting. 2024-03-21T13:29:10.302 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_7_[81699]: listing objects. 2024-03-21T13:29:10.302 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_7_[81699]: listed object 0... 2024-03-21T13:29:10.302 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_7_[81699]: listed object 25... 2024-03-21T13:29:10.302 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_7_[81699]: listed object 50... 2024-03-21T13:29:10.302 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_7_[81699]: saw 56 objects 2024-03-21T13:29:10.302 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_7_[81699]: shutting down. 2024-03-21T13:29:10.302 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: ******************************* 2024-03-21T13:29:10.302 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: ******************************* 2024-03-21T13:29:10.302 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_8_[81700]: starting. 2024-03-21T13:29:10.302 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_8_[81700]: added 25 objects... 2024-03-21T13:29:10.303 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_8_[81700]: added half of the objects 2024-03-21T13:29:10.303 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_8_[81700]: added 50 objects... 2024-03-21T13:29:10.303 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_8_[81700]: added 50 objects 2024-03-21T13:29:10.303 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_8_[81700]: shutting down. 2024-03-21T13:29:10.303 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: ******************************* 2024-03-21T13:29:10.303 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: ******************************* 2024-03-21T13:29:10.760 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: ********************** handler_error: Running main() from gmock_main.cc 2024-03-21T13:29:10.760 INFO:tasks.workunit.client.0.smithi097.stdout: handler_error: [==========] Running 1 test from 1 test suite. 2024-03-21T13:29:10.760 INFO:tasks.workunit.client.0.smithi097.stdout: handler_error: [----------] Global test environment set-up. 2024-03-21T13:29:10.760 INFO:tasks.workunit.client.0.smithi097.stdout: handler_error: [----------] 1 test from neocls_handler_error 2024-03-21T13:29:10.760 INFO:tasks.workunit.client.0.smithi097.stdout: handler_error: [ RUN ] neocls_handler_error.test_handler_error 2024-03-21T13:29:10.760 INFO:tasks.workunit.client.0.smithi097.stdout: handler_error: [ OK ] neocls_handler_error.test_handler_error (10938 ms) 2024-03-21T13:29:10.760 INFO:tasks.workunit.client.0.smithi097.stdout: handler_error: [----------] 1 test from neocls_handler_error (10938 ms total) 2024-03-21T13:29:10.760 INFO:tasks.workunit.client.0.smithi097.stdout: handler_error: 2024-03-21T13:29:10.761 INFO:tasks.workunit.client.0.smithi097.stdout: handler_error: [----------] Global test environment tear-down 2024-03-21T13:29:10.761 INFO:tasks.workunit.client.0.smithi097.stdout: handler_error: [==========] 1 test from 1 test suite ran. (10938 ms total) 2024-03-21T13:29:10.761 INFO:tasks.workunit.client.0.smithi097.stdout: handler_error: [ PASSED ] 1 test. 2024-03-21T13:29:10.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:10 smithi097 ceph-mon[39416]: pgmap v74: 1284 pgs: 896 creating+peering, 11 creating+activating, 249 active+clean, 128 unknown; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 131 B/s wr, 1 op/s 2024-03-21T13:29:10.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:10 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi097-79906-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:10.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:10 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi097-79992-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:10.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:10 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi097-80014-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:10.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:10 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi097-80039-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:10.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:10 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi097-79828-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:10.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:10 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi097-80000-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:10.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:10 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80370-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:10.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:10 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80165-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:10.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:10 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi097-80011-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:10.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:10 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi097-79710-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:10.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:10 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80187-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:10.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:10 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi097-79910-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:10.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:10 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2555609167' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi097-80028-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:29:10.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:10 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3130993506' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi097-79740-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:10.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:10 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2920807904' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:10.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:10 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2881128302' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi097-80404-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:10.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:10 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:10.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:10 smithi097 ceph-mon[39416]: osdmap e75: 8 total, 8 up, 8 in 2024-03-21T13:29:10.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:10 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2555609167' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi097-80028-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi097-80028-1"}]: dispatch 2024-03-21T13:29:10.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:10 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:10.806 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:10 smithi097 ceph-mon[30040]: pgmap v74: 1284 pgs: 896 creating+peering, 11 creating+activating, 249 active+clean, 128 unknown; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 131 B/s wr, 1 op/s 2024-03-21T13:29:10.806 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:10 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi097-79906-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:10.806 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:10 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi097-79992-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:10.806 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:10 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi097-80014-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:10.806 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:10 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi097-80039-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:10.806 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:10 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi097-79828-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:10.806 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:10 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi097-80000-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:10.806 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:10 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80370-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:10.806 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:10 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80165-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:10.806 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:10 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi097-80011-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:10.806 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:10 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi097-79710-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:10.807 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:10 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80187-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:10.807 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:10 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi097-79910-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:10.807 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:10 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2555609167' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi097-80028-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:29:10.807 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:10 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3130993506' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi097-79740-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:10.807 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:10 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2920807904' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:10.807 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:10 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2881128302' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi097-80404-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:10.807 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:10 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:10.807 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:10 smithi097 ceph-mon[30040]: osdmap e75: 8 total, 8 up, 8 in 2024-03-21T13:29:10.807 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:10 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2555609167' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi097-80028-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi097-80028-1"}]: dispatch 2024-03-21T13:29:10.807 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:10 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:10.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:10 smithi144 ceph-mon[33633]: pgmap v74: 1284 pgs: 896 creating+peering, 11 creating+activating, 249 active+clean, 128 unknown; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 131 B/s wr, 1 op/s 2024-03-21T13:29:10.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:10 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi097-79906-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:10.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:10 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi097-79992-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:10.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:10 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi097-80014-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:10.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:10 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi097-80039-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:10.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:10 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi097-79828-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:10.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:10 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi097-80000-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:10.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:10 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80370-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:10.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:10 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80165-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:10.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:10 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi097-80011-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:10.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:10 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi097-79710-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:10.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:10 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80187-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:10.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:10 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi097-79910-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:10.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:10 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2555609167' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi097-80028-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:29:10.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:10 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3130993506' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi097-79740-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:10.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:10 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2920807904' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:10.873 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:10 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2881128302' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi097-80404-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:10.873 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:10 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:10.873 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:10 smithi144 ceph-mon[33633]: osdmap e75: 8 total, 8 up, 8 in 2024-03-21T13:29:10.873 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:10 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2555609167' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi097-80028-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi097-80028-1"}]: dispatch 2024-03-21T13:29:10.873 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:10 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:11.636 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd_pp: Running main() from gmock_main.cc 2024-03-21T13:29:11.637 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd_pp: [==========] Running 3 tests from 1 test suite. 2024-03-21T13:29:11.637 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd_pp: [----------] Global test environment set-up. 2024-03-21T13:29:11.637 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd 2024-03-21T13:29:11.637 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.MonDescribePP 2024-03-21T13:29:11.637 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd_pp: [ OK ] LibRadosCmd.MonDescribePP (42 ms) 2024-03-21T13:29:11.637 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.OSDCmdPP 2024-03-21T13:29:11.637 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd_pp: [ OK ] LibRadosCmd.OSDCmdPP (47 ms) 2024-03-21T13:29:11.637 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.PGCmdPP 2024-03-21T13:29:11.637 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd_pp: [ OK ] LibRadosCmd.PGCmdPP (11911 ms) 2024-03-21T13:29:11.637 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd (12000 ms total) 2024-03-21T13:29:11.637 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd_pp: 2024-03-21T13:29:11.637 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd_pp: [----------] Global test environment tear-down 2024-03-21T13:29:11.637 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd_pp: [==========] 3 tests from 1 test suite ran. (12000 ms total) 2024-03-21T13:29:11.638 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd_pp: [ PASSED ] 3 tests. 2024-03-21T13:29:11.715 INFO:tasks.workunit.client.0.smithi097.stdout:********* 2024-03-21T13:29:11.715 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_9_[81772]: starting. 2024-03-21T13:29:11.715 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_9_[81772]: creating pool ceph_test_rados_list_parallel.smithi097-80461 2024-03-21T13:29:11.715 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_9_[81772]: created object 0... 2024-03-21T13:29:11.715 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_9_[81772]: created object 25... 2024-03-21T13:29:11.715 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_9_[81772]: created object 49... 2024-03-21T13:29:11.715 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_9_[81772]: finishing. 2024-03-21T13:29:11.715 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_9_[81772]: shutting down. 2024-03-21T13:29:11.715 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: ******************************* 2024-03-21T13:29:11.715 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: ******************************* 2024-03-21T13:29:11.716 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: ******************************* 2024-03-21T13:29:11.716 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_10_[81773]: starting. 2024-03-21T13:29:11.716 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_10_[81773]: listing objects. 2024-03-21T13:29:11.716 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_10_[81773]: listed object 0... 2024-03-21T13:29:11.716 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_10_[81773]: listed object 25... 2024-03-21T13:29:11.716 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_10_[81773]: listed object 50... 2024-03-21T13:29:11.716 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_10_[81773]: listed object 75... 2024-03-21T13:29:11.716 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_10_[81773]: saw 93 objects 2024-03-21T13:29:11.716 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_10_[81773]: shutting down. 2024-03-21T13:29:11.716 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: ******************************* 2024-03-21T13:29:11.716 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: ******************************* 2024-03-21T13:29:11.716 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: ******************************* 2024-03-21T13:29:11.716 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_13_[81776]: starting. 2024-03-21T13:29:11.717 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_13_[81776]: removed 25 objects... 2024-03-21T13:29:11.717 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_13_[81776]: removed half of the objects 2024-03-21T13:29:11.717 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_13_[81776]: removed 50 objects... 2024-03-21T13:29:11.717 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_13_[81776]: removed 50 objects 2024-03-21T13:29:11.717 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_13_[81776]: shutting down. 2024-03-21T13:29:11.717 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: ******************************* 2024-03-21T13:29:11.717 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: ******************************* 2024-03-21T13:29:11.717 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: ******************************* 2024-03-21T13:29:11.717 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_11_[81774]: starting. 2024-03-21T13:29:11.717 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_11_[81774]: added 25 objects... 2024-03-21T13:29:11.717 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_11_[81774]: added half of the objects 2024-03-21T13:29:11.717 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_11_[81774]: added 50 objects... 2024-03-21T13:29:11.717 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_11_[81774]: added 50 objects 2024-03-21T13:29:11.717 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_11_[81774]: shutting down. 2024-03-21T13:29:11.718 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: ******************************* 2024-03-21T13:29:11.718 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: ******************************* 2024-03-21T13:29:11.718 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: ******************************* 2024-03-21T13:29:11.718 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_12_[81775]: starting. 2024-03-21T13:29:11.718 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_12_[81775]: added 25 objects... 2024-03-21T13:29:11.718 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_12_[81775]: added half of the objects 2024-03-21T13:29:11.718 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_12_[81775]: added 50 objects... 2024-03-21T13:29:11.718 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_12_[81775]: added 50 objects 2024-03-21T13:29:11.718 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_12_[81775]: shutting down. 2024-03-21T13:29:11.718 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: ******************************* 2024-03-21T13:29:11.718 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: ******************************* 2024-03-21T13:29:11.718 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: ******************************* 2024-03-21T13:29:11.718 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: ******************************* 2024-03-21T13:29:11.718 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_14_[81885]: starting. 2024-03-21T13:29:11.719 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_14_[81885]: creating pool ceph_test_rados_list_parallel.smithi097-80461 2024-03-21T13:29:11.719 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_14_[81885]: created object 0... 2024-03-21T13:29:11.719 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_14_[81885]: created object 25... 2024-03-21T13:29:11.719 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_14_[81885]: created object 49... 2024-03-21T13:29:11.719 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_14_[81885]: finishing. 2024-03-21T13:29:11.719 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_14_[81885]: shutting down. 2024-03-21T13:29:11.719 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: ******************************* 2024-03-21T13:29:11.719 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: ******************************* 2024-03-21T13:29:11.719 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: ******************************* 2024-03-21T13:29:11.719 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: ******************************* 2024-03-21T13:29:11.719 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_15_[81886]: starting. 2024-03-21T13:29:11.719 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_15_[81886]: listing objects. 2024-03-21T13:29:11.719 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_15_[81886]: listed object 0... 2024-03-21T13:29:11.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:11 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3464660016' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi097-79675-22"}]: dispatch 2024-03-21T13:29:11.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:11 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/725581604' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-03-21T13:29:11.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:11 smithi144 ceph-mon[33633]: pgmap v76: 1092 pgs: 800 creating+peering, 132 active+clean, 160 unknown; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:29:11.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:11 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/233801175' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi097-79664-16"}]: dispatch 2024-03-21T13:29:11.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:11 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi097-80861-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi097-80861-1"}]': finished 2024-03-21T13:29:11.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:11 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi097-80712-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi097-80712-1"}]': finished 2024-03-21T13:29:11.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:11 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:11.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:11 smithi144 ceph-mon[33633]: osdmap e76: 8 total, 8 up, 8 in 2024-03-21T13:29:11.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:11 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/233801175' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi097-79664-16"}]: dispatch 2024-03-21T13:29:11.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:11 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/233801175' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi097-79664-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:11.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:11 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3464660016' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi097-79675-22"}]: dispatch 2024-03-21T13:29:11.951 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:11 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/725581604' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-03-21T13:29:11.951 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:11 smithi097 ceph-mon[39416]: pgmap v76: 1092 pgs: 800 creating+peering, 132 active+clean, 160 unknown; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:29:11.951 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:11 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/233801175' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi097-79664-16"}]: dispatch 2024-03-21T13:29:11.951 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:11 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi097-80861-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi097-80861-1"}]': finished 2024-03-21T13:29:11.952 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:11 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi097-80712-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi097-80712-1"}]': finished 2024-03-21T13:29:11.952 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:11 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:11.952 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:11 smithi097 ceph-mon[39416]: osdmap e76: 8 total, 8 up, 8 in 2024-03-21T13:29:11.952 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:11 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/233801175' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi097-79664-16"}]: dispatch 2024-03-21T13:29:11.952 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:11 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/233801175' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi097-79664-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:11.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:11 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3464660016' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi097-79675-22"}]: dispatch 2024-03-21T13:29:11.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:11 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/725581604' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-03-21T13:29:11.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:11 smithi097 ceph-mon[30040]: pgmap v76: 1092 pgs: 800 creating+peering, 132 active+clean, 160 unknown; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:29:11.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:11 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/233801175' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi097-79664-16"}]: dispatch 2024-03-21T13:29:11.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:11 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi097-80861-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi097-80861-1"}]': finished 2024-03-21T13:29:11.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:11 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi097-80712-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi097-80712-1"}]': finished 2024-03-21T13:29:11.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:11 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:11.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:11 smithi097 ceph-mon[30040]: osdmap e76: 8 total, 8 up, 8 in 2024-03-21T13:29:11.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:11 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/233801175' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi097-79664-16"}]: dispatch 2024-03-21T13:29:11.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:11 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/233801175' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi097-79664-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:12.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:29:11 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:29:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:29:13.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:13 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2555609167' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi097-80028-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi097-80028-1"}]': finished 2024-03-21T13:29:13.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:13 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/233801175' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi097-79664-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:29:13.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:13 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/224160575' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi097-79636-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:13.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:13 smithi097 ceph-mon[30040]: osdmap e77: 8 total, 8 up, 8 in 2024-03-21T13:29:13.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:13 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/233801175' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi097-79664-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi097-79664-16"}]: dispatch 2024-03-21T13:29:13.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:13 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1666538629' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi097-79618-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:13.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:13 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi097-79636-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:13.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:13 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.0"}]: dispatch 2024-03-21T13:29:13.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:13 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.1"}]: dispatch 2024-03-21T13:29:13.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:13 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.2"}]: dispatch 2024-03-21T13:29:13.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:13 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.3"}]: dispatch 2024-03-21T13:29:13.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:13 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.4"}]: dispatch 2024-03-21T13:29:13.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:13 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.5"}]: dispatch 2024-03-21T13:29:13.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:13 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.6"}]: dispatch 2024-03-21T13:29:13.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:13 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.7"}]: dispatch 2024-03-21T13:29:13.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:13 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.8"}]: dispatch 2024-03-21T13:29:13.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:13 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.9"}]: dispatch 2024-03-21T13:29:13.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:13 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:13.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:13 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2555609167' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi097-80028-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi097-80028-1"}]': finished 2024-03-21T13:29:13.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:13 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/233801175' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi097-79664-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:29:13.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:13 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/224160575' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi097-79636-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:13.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:13 smithi097 ceph-mon[39416]: osdmap e77: 8 total, 8 up, 8 in 2024-03-21T13:29:13.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:13 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/233801175' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi097-79664-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi097-79664-16"}]: dispatch 2024-03-21T13:29:13.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:13 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1666538629' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi097-79618-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:13.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:13 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi097-79636-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:13.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:13 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.0"}]: dispatch 2024-03-21T13:29:13.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:13 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.1"}]: dispatch 2024-03-21T13:29:13.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:13 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.2"}]: dispatch 2024-03-21T13:29:13.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:13 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.3"}]: dispatch 2024-03-21T13:29:13.556 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:13 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.4"}]: dispatch 2024-03-21T13:29:13.556 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:13 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.5"}]: dispatch 2024-03-21T13:29:13.556 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:13 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.6"}]: dispatch 2024-03-21T13:29:13.556 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:13 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.7"}]: dispatch 2024-03-21T13:29:13.556 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:13 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.8"}]: dispatch 2024-03-21T13:29:13.556 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:13 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.9"}]: dispatch 2024-03-21T13:29:13.556 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:13 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:13.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:29:13 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:29:13.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:13 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2555609167' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi097-80028-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi097-80028-1"}]': finished 2024-03-21T13:29:13.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:13 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/233801175' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi097-79664-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:29:13.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:13 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/224160575' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi097-79636-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:13.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:13 smithi144 ceph-mon[33633]: osdmap e77: 8 total, 8 up, 8 in 2024-03-21T13:29:13.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:13 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/233801175' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi097-79664-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi097-79664-16"}]: dispatch 2024-03-21T13:29:13.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:13 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1666538629' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi097-79618-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:13.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:13 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi097-79636-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:13.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:13 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.0"}]: dispatch 2024-03-21T13:29:13.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:13 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.1"}]: dispatch 2024-03-21T13:29:13.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:13 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.2"}]: dispatch 2024-03-21T13:29:13.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:13 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.3"}]: dispatch 2024-03-21T13:29:13.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:13 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.4"}]: dispatch 2024-03-21T13:29:13.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:13 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.5"}]: dispatch 2024-03-21T13:29:13.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:13 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.6"}]: dispatch 2024-03-21T13:29:13.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:13 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.7"}]: dispatch 2024-03-21T13:29:13.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:13 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.8"}]: dispatch 2024-03-21T13:29:13.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:13 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.9"}]: dispatch 2024-03-21T13:29:13.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:13 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:13.695 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_15_[81886]: listed object 2 api_c_read_operations: Running main() from gmock_main.cc 2024-03-21T13:29:13.695 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_read_operations: [==========] Running 17 tests from 1 test suite. 2024-03-21T13:29:13.695 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_read_operations: [----------] Global test environment set-up. 2024-03-21T13:29:13.695 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_read_operations: [----------] 17 tests from CReadOpsTest 2024-03-21T13:29:13.695 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.NewDelete 2024-03-21T13:29:13.695 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_read_operations: [ OK ] CReadOpsTest.NewDelete (0 ms) 2024-03-21T13:29:13.695 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.SetOpFlags 2024-03-21T13:29:13.695 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_read_operations: [ OK ] CReadOpsTest.SetOpFlags (719 ms) 2024-03-21T13:29:13.695 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertExists 2024-03-21T13:29:13.696 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertExists (2194 ms) 2024-03-21T13:29:13.696 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertVersion 2024-03-21T13:29:13.696 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertVersion (3 ms) 2024-03-21T13:29:13.696 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpXattr 2024-03-21T13:29:13.696 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpXattr (11 ms) 2024-03-21T13:29:13.696 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Read 2024-03-21T13:29:13.696 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Read (6 ms) 2024-03-21T13:29:13.696 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Checksum 2024-03-21T13:29:13.696 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Checksum (6 ms) 2024-03-21T13:29:13.696 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.RWOrderedRead 2024-03-21T13:29:13.696 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_read_operations: [ OK ] CReadOpsTest.RWOrderedRead (4 ms) 2024-03-21T13:29:13.696 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ShortRead 2024-03-21T13:29:13.696 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ShortRead (6 ms) 2024-03-21T13:29:13.696 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Exec 2024-03-21T13:29:13.696 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Exec (2 ms) 2024-03-21T13:29:13.697 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ExecUserBuf 2024-03-21T13:29:13.697 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ExecUserBuf (3 ms) 2024-03-21T13:29:13.697 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat 2024-03-21T13:29:13.697 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat (3 ms) 2024-03-21T13:29:13.697 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat2 2024-03-21T13:29:13.697 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat2 (2 ms) 2024-03-21T13:29:13.697 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Omap 2024-03-21T13:29:13.697 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Omap (8 ms) 2024-03-21T13:29:13.697 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.OmapNuls 2024-03-21T13:29:13.697 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_read_operations: [ OK ] CReadOpsTest.OmapNuls (7 ms) 2024-03-21T13:29:13.697 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.GetXattrs 2024-03-21T13:29:13.697 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_read_operations: [ OK ] CReadOpsTest.GetXattrs (6 ms) 2024-03-21T13:29:13.697 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpExt 2024-03-21T13:29:13.697 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpExt (6 ms) 2024-03-21T13:29:13.698 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_read_operations: [----------] 17 tests from CReadOpsTest (2986 ms total) 2024-03-21T13:29:13.698 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_read_operations: 2024-03-21T13:29:13.698 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_read_operations: [----------] Global test environment tear-down 2024-03-21T13:29:13.698 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_read_operations: [==========] 17 tests from 1 test suite ran. (14000 ms total) 2024-03-21T13:29:13.698 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_read_operations: [ PASSED ] 17 tests. 2024-03-21T13:29:13.698 INFO:tasks.workunit.client.0.smithi097.stdout:5... 2024-03-21T13:29:13.698 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_15_[81886]: listed object 50... 2024-03-21T13:29:13.698 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_15_[81886]: listed object 75... 2024-03-21T13:29:13.698 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_15_[81886]: listed object 100... 2024-03-21T13:29:13.699 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_15_[81886]: listed object 125... 2024-03-21T13:29:13.699 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_15_[81886]: saw 150 objects 2024-03-21T13:29:13.699 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_15_[81886]: shutting down. 2024-03-21T13:29:13.699 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: ******************************* 2024-03-21T13:29:13.699 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: ******************************* 2024-03-21T13:29:13.699 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: ******************************* 2024-03-21T13:29:13.699 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: ******************************* 2024-03-21T13:29:13.699 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_16_[81887]: starting. 2024-03-21T13:29:13.699 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_16_[81887]: added 25 objects... 2024-03-21T13:29:13.699 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_16_[81887]: added half of the objects 2024-03-21T13:29:13.699 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_16_[81887]: added 50 objects... 2024-03-21T13:29:13.699 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_16_[81887]: added 50 objects 2024-03-21T13:29:13.699 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: process_16_[81887]: shutting down. 2024-03-21T13:29:13.699 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: ******************************* 2024-03-21T13:29:13.699 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: ******************************* 2024-03-21T13:29:13.700 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: ******************************* 2024-03-21T13:29:13.700 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: ******************************* 2024-03-21T13:29:13.700 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: ******************************* 2024-03-21T13:29:13.700 INFO:tasks.workunit.client.0.smithi097.stdout: list_parallel: ******* SUCCESS ********** 2024-03-21T13:29:14.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[30040]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.0"}]: dispatch 2024-03-21T13:29:14.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[30040]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.1"}]: dispatch 2024-03-21T13:29:14.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[30040]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.2"}]: dispatch 2024-03-21T13:29:14.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[30040]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.3"}]: dispatch 2024-03-21T13:29:14.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[30040]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.4"}]: dispatch 2024-03-21T13:29:14.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[30040]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.5"}]: dispatch 2024-03-21T13:29:14.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[30040]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.6"}]: dispatch 2024-03-21T13:29:14.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[30040]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.7"}]: dispatch 2024-03-21T13:29:14.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[30040]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.8"}]: dispatch 2024-03-21T13:29:14.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[30040]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.9"}]: dispatch 2024-03-21T13:29:14.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[30040]: pgmap v79: 1020 pgs: 608 creating+peering, 132 active+clean, 280 unknown; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:29:14.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[30040]: 15.6 deep-scrub starts 2024-03-21T13:29:14.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[30040]: 15.7 deep-scrub starts 2024-03-21T13:29:14.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[30040]: 15.7 deep-scrub ok 2024-03-21T13:29:14.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[30040]: 15.6 deep-scrub ok 2024-03-21T13:29:14.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1666538629' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi097-79618-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:14.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi097-79636-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:14.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[30040]: 15.0 deep-scrub starts 2024-03-21T13:29:14.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[30040]: 15.0 deep-scrub ok 2024-03-21T13:29:14.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:14.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[30040]: osdmap e78: 8 total, 8 up, 8 in 2024-03-21T13:29:14.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[30040]: Health check update: 16 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:29:14.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3695006278' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi097-79740-10"}]: dispatch 2024-03-21T13:29:14.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[30040]: 15.4 deep-scrub starts 2024-03-21T13:29:14.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[30040]: 15.4 deep-scrub ok 2024-03-21T13:29:14.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi097-79740-10"}]: dispatch 2024-03-21T13:29:14.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/233801175' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi097-79664-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi097-79664-16"}]': finished 2024-03-21T13:29:14.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[30040]: osdmap e79: 8 total, 8 up, 8 in 2024-03-21T13:29:14.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3695006278' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi097-79740-10"}]: dispatch 2024-03-21T13:29:14.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi097-79740-10"}]: dispatch 2024-03-21T13:29:14.556 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[39416]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.0"}]: dispatch 2024-03-21T13:29:14.556 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[39416]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.1"}]: dispatch 2024-03-21T13:29:14.556 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[39416]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.2"}]: dispatch 2024-03-21T13:29:14.556 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[39416]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.3"}]: dispatch 2024-03-21T13:29:14.556 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[39416]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.4"}]: dispatch 2024-03-21T13:29:14.556 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[39416]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.5"}]: dispatch 2024-03-21T13:29:14.556 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[39416]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.6"}]: dispatch 2024-03-21T13:29:14.556 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[39416]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.7"}]: dispatch 2024-03-21T13:29:14.556 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[39416]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.8"}]: dispatch 2024-03-21T13:29:14.557 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[39416]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.9"}]: dispatch 2024-03-21T13:29:14.557 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[39416]: pgmap v79: 1020 pgs: 608 creating+peering, 132 active+clean, 280 unknown; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:29:14.557 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[39416]: 15.6 deep-scrub starts 2024-03-21T13:29:14.557 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[39416]: 15.7 deep-scrub starts 2024-03-21T13:29:14.557 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[39416]: 15.7 deep-scrub ok 2024-03-21T13:29:14.557 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[39416]: 15.6 deep-scrub ok 2024-03-21T13:29:14.557 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1666538629' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi097-79618-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:14.557 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi097-79636-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:14.557 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[39416]: 15.0 deep-scrub starts 2024-03-21T13:29:14.557 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[39416]: 15.0 deep-scrub ok 2024-03-21T13:29:14.557 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:14.557 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[39416]: osdmap e78: 8 total, 8 up, 8 in 2024-03-21T13:29:14.557 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[39416]: Health check update: 16 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:29:14.557 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3695006278' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi097-79740-10"}]: dispatch 2024-03-21T13:29:14.557 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[39416]: 15.4 deep-scrub starts 2024-03-21T13:29:14.558 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[39416]: 15.4 deep-scrub ok 2024-03-21T13:29:14.558 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi097-79740-10"}]: dispatch 2024-03-21T13:29:14.558 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/233801175' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi097-79664-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi097-79664-16"}]': finished 2024-03-21T13:29:14.558 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[39416]: osdmap e79: 8 total, 8 up, 8 in 2024-03-21T13:29:14.558 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3695006278' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi097-79740-10"}]: dispatch 2024-03-21T13:29:14.558 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi097-79740-10"}]: dispatch 2024-03-21T13:29:14.558 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/224160575' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi097-79636-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:14.558 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1442988712' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:14.558 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3695006278' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi097-79740-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:14.558 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi097-79636-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:14.559 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:14.559 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi097-79740-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:14.559 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi097-79675-22"}]: dispatch 2024-03-21T13:29:14.559 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/224160575' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi097-79636-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:14.559 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1442988712' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:14.559 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3695006278' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi097-79740-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:14.559 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi097-79636-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:14.559 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:14.559 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi097-79740-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:14.559 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:14 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi097-79675-22"}]: dispatch 2024-03-21T13:29:14.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:14 smithi144 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.0"}]: dispatch 2024-03-21T13:29:14.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:14 smithi144 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.1"}]: dispatch 2024-03-21T13:29:14.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:14 smithi144 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.2"}]: dispatch 2024-03-21T13:29:14.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:14 smithi144 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.3"}]: dispatch 2024-03-21T13:29:14.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:14 smithi144 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.4"}]: dispatch 2024-03-21T13:29:14.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:14 smithi144 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.5"}]: dispatch 2024-03-21T13:29:14.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:14 smithi144 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.6"}]: dispatch 2024-03-21T13:29:14.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:14 smithi144 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.7"}]: dispatch 2024-03-21T13:29:14.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:14 smithi144 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.8"}]: dispatch 2024-03-21T13:29:14.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:14 smithi144 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.9"}]: dispatch 2024-03-21T13:29:14.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:14 smithi144 ceph-mon[33633]: pgmap v79: 1020 pgs: 608 creating+peering, 132 active+clean, 280 unknown; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:29:14.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:14 smithi144 ceph-mon[33633]: 15.6 deep-scrub starts 2024-03-21T13:29:14.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:14 smithi144 ceph-mon[33633]: 15.7 deep-scrub starts 2024-03-21T13:29:14.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:14 smithi144 ceph-mon[33633]: 15.7 deep-scrub ok 2024-03-21T13:29:14.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:14 smithi144 ceph-mon[33633]: 15.6 deep-scrub ok 2024-03-21T13:29:14.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:14 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1666538629' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi097-79618-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:14.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:14 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi097-79636-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:14.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:14 smithi144 ceph-mon[33633]: 15.0 deep-scrub starts 2024-03-21T13:29:14.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:14 smithi144 ceph-mon[33633]: 15.0 deep-scrub ok 2024-03-21T13:29:14.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:14 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:14.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:14 smithi144 ceph-mon[33633]: osdmap e78: 8 total, 8 up, 8 in 2024-03-21T13:29:14.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:14 smithi144 ceph-mon[33633]: Health check update: 16 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:29:14.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:14 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3695006278' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi097-79740-10"}]: dispatch 2024-03-21T13:29:14.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:14 smithi144 ceph-mon[33633]: 15.4 deep-scrub starts 2024-03-21T13:29:14.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:14 smithi144 ceph-mon[33633]: 15.4 deep-scrub ok 2024-03-21T13:29:14.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:14 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi097-79740-10"}]: dispatch 2024-03-21T13:29:14.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:14 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/233801175' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi097-79664-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi097-79664-16"}]': finished 2024-03-21T13:29:14.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:14 smithi144 ceph-mon[33633]: osdmap e79: 8 total, 8 up, 8 in 2024-03-21T13:29:14.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:14 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3695006278' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi097-79740-10"}]: dispatch 2024-03-21T13:29:14.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:14 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi097-79740-10"}]: dispatch 2024-03-21T13:29:14.624 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:14 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/224160575' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi097-79636-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:14.624 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:14 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1442988712' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:14.624 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:14 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3695006278' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi097-79740-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:14.624 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:14 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi097-79636-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:14.624 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:14 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:14.624 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:14 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi097-79740-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:14.624 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:14 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi097-79675-22"}]: dispatch 2024-03-21T13:29:14.711 INFO:tasks.workunit.client.0.smithi097.stdout: api_cls_remote_reads: [==========] Running 1 test from 1 test suite. 2024-03-21T13:29:14.711 INFO:tasks.workunit.client.0.smithi097.stdout: api_cls_remote_reads: [----------] Global test environment set-up. 2024-03-21T13:29:14.711 INFO:tasks.workunit.client.0.smithi097.stdout: api_cls_remote_reads: [----------] 1 test from ClsTestRemoteReads 2024-03-21T13:29:14.711 INFO:tasks.workunit.client.0.smithi097.stdout: api_cls_remote_reads: [ RUN ] ClsTestRemoteReads.TestGather 2024-03-21T13:29:14.711 INFO:tasks.workunit.client.0.smithi097.stdout: api_cls_remote_reads: [ OK ] ClsTestRemoteReads.TestGather (15013 ms) 2024-03-21T13:29:14.711 INFO:tasks.workunit.client.0.smithi097.stdout: api_cls_remote_reads: [----------] 1 test from ClsTestRemoteReads (15013 ms total) 2024-03-21T13:29:14.711 INFO:tasks.workunit.client.0.smithi097.stdout: api_cls_remote_reads: 2024-03-21T13:29:14.711 INFO:tasks.workunit.client.0.smithi097.stdout: api_cls_remote_reads: [----------] Global test environment tear-down 2024-03-21T13:29:14.711 INFO:tasks.workunit.client.0.smithi097.stdout: api_cls_remote_reads: [==========] 1 test from 1 test suite ran. (15013 ms total) 2024-03-21T13:29:14.711 INFO:tasks.workunit.client.0.smithi097.stdout: api_cls_remote_reads: [ PASSED ] 1 test. 2024-03-21T13:29:15.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[39416]: 15.3 deep-scrub starts 2024-03-21T13:29:15.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[39416]: 15.3 deep-scrub ok 2024-03-21T13:29:15.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[39416]: 15.9 deep-scrub starts 2024-03-21T13:29:15.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[39416]: 15.9 deep-scrub ok 2024-03-21T13:29:15.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[39416]: 15.1 deep-scrub starts 2024-03-21T13:29:15.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[39416]: 15.1 deep-scrub ok 2024-03-21T13:29:15.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:29:15.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[39416]: 15.8 deep-scrub starts 2024-03-21T13:29:15.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[39416]: 15.8 deep-scrub ok 2024-03-21T13:29:15.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3464660016' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi097-79675-22"}]: dispatch 2024-03-21T13:29:15.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2661409599' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi097-79828-10"}]: dispatch 2024-03-21T13:29:15.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:15.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi097-79828-10"}]: dispatch 2024-03-21T13:29:15.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2661409599' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi097-79828-10"}]: dispatch 2024-03-21T13:29:15.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi097-79828-10"}]: dispatch 2024-03-21T13:29:15.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2661409599' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi097-79828-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:15.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi097-79828-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:15.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi097-79675-22"}]: dispatch 2024-03-21T13:29:15.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3464660016' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi097-79675-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:15.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi097-79675-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:15.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi097-79636-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:15.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:15.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi097-79740-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:29:15.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi097-79828-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:29:15.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi097-79675-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:29:15.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[39416]: osdmap e80: 8 total, 8 up, 8 in 2024-03-21T13:29:15.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/224160575' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi097-79636-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-03-21T13:29:15.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3695006278' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi097-79740-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi097-79740-10"}]: dispatch 2024-03-21T13:29:15.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2661409599' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi097-79828-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi097-79828-10"}]: dispatch 2024-03-21T13:29:15.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3464660016' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi097-79675-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi097-79675-22"}]: dispatch 2024-03-21T13:29:15.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2488892529' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:15.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3213135837' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi097-80712-1"}]: dispatch 2024-03-21T13:29:15.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi097-79675-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi097-79675-22"}]: dispatch 2024-03-21T13:29:15.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi097-79636-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-03-21T13:29:15.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi097-79740-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi097-79740-10"}]: dispatch 2024-03-21T13:29:15.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:15.807 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi097-79828-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi097-79828-10"}]: dispatch 2024-03-21T13:29:15.807 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi097-80712-1"}]: dispatch 2024-03-21T13:29:15.807 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1371127508' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi097-80014-7"}]: dispatch 2024-03-21T13:29:15.807 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi097-80014-7"}]: dispatch 2024-03-21T13:29:15.807 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1371127508' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi097-80014-7"}]: dispatch 2024-03-21T13:29:15.807 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi097-80014-7"}]: dispatch 2024-03-21T13:29:15.807 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1371127508' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi097-80014-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:15.807 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[30040]: 15.3 deep-scrub starts 2024-03-21T13:29:15.808 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[30040]: 15.3 deep-scrub ok 2024-03-21T13:29:15.808 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[30040]: 15.9 deep-scrub starts 2024-03-21T13:29:15.808 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[30040]: 15.9 deep-scrub ok 2024-03-21T13:29:15.808 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[30040]: 15.1 deep-scrub starts 2024-03-21T13:29:15.808 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[30040]: 15.1 deep-scrub ok 2024-03-21T13:29:15.808 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:29:15.808 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[30040]: 15.8 deep-scrub starts 2024-03-21T13:29:15.808 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[30040]: 15.8 deep-scrub ok 2024-03-21T13:29:15.808 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3464660016' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi097-79675-22"}]: dispatch 2024-03-21T13:29:15.808 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2661409599' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi097-79828-10"}]: dispatch 2024-03-21T13:29:15.808 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:15.808 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi097-79828-10"}]: dispatch 2024-03-21T13:29:15.808 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2661409599' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi097-79828-10"}]: dispatch 2024-03-21T13:29:15.808 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi097-79828-10"}]: dispatch 2024-03-21T13:29:15.808 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2661409599' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi097-79828-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:15.809 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi097-79828-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:15.809 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi097-79675-22"}]: dispatch 2024-03-21T13:29:15.809 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3464660016' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi097-79675-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:15.809 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi097-79675-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:15.809 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi097-79636-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:15.809 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:15.809 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi097-79740-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:29:15.809 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi097-79828-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:29:15.809 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi097-79675-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:29:15.809 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[30040]: osdmap e80: 8 total, 8 up, 8 in 2024-03-21T13:29:15.809 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/224160575' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi097-79636-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-03-21T13:29:15.810 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3695006278' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi097-79740-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi097-79740-10"}]: dispatch 2024-03-21T13:29:15.810 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2661409599' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi097-79828-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi097-79828-10"}]: dispatch 2024-03-21T13:29:15.810 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3464660016' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi097-79675-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi097-79675-22"}]: dispatch 2024-03-21T13:29:15.810 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2488892529' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:15.810 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3213135837' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi097-80712-1"}]: dispatch 2024-03-21T13:29:15.810 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi097-79675-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi097-79675-22"}]: dispatch 2024-03-21T13:29:15.810 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi097-79636-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-03-21T13:29:15.810 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi097-79740-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi097-79740-10"}]: dispatch 2024-03-21T13:29:15.810 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:15.810 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi097-79828-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi097-79828-10"}]: dispatch 2024-03-21T13:29:15.810 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi097-80712-1"}]: dispatch 2024-03-21T13:29:15.810 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1371127508' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi097-80014-7"}]: dispatch 2024-03-21T13:29:15.810 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi097-80014-7"}]: dispatch 2024-03-21T13:29:15.810 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1371127508' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi097-80014-7"}]: dispatch 2024-03-21T13:29:15.810 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi097-80014-7"}]: dispatch 2024-03-21T13:29:15.811 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:15 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1371127508' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi097-80014-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:15.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:15 smithi144 ceph-mon[33633]: 15.3 deep-scrub starts 2024-03-21T13:29:15.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:15 smithi144 ceph-mon[33633]: 15.3 deep-scrub ok 2024-03-21T13:29:15.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:15 smithi144 ceph-mon[33633]: 15.9 deep-scrub starts 2024-03-21T13:29:15.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:15 smithi144 ceph-mon[33633]: 15.9 deep-scrub ok 2024-03-21T13:29:15.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:15 smithi144 ceph-mon[33633]: 15.1 deep-scrub starts 2024-03-21T13:29:15.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:15 smithi144 ceph-mon[33633]: 15.1 deep-scrub ok 2024-03-21T13:29:15.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:15 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:29:15.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:15 smithi144 ceph-mon[33633]: 15.8 deep-scrub starts 2024-03-21T13:29:15.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:15 smithi144 ceph-mon[33633]: 15.8 deep-scrub ok 2024-03-21T13:29:15.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:15 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3464660016' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi097-79675-22"}]: dispatch 2024-03-21T13:29:15.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:15 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2661409599' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi097-79828-10"}]: dispatch 2024-03-21T13:29:15.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:15 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:15.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:15 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi097-79828-10"}]: dispatch 2024-03-21T13:29:15.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:15 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2661409599' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi097-79828-10"}]: dispatch 2024-03-21T13:29:15.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:15 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi097-79828-10"}]: dispatch 2024-03-21T13:29:15.873 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:15 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2661409599' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi097-79828-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:15.873 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:15 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi097-79828-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:15.873 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:15 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi097-79675-22"}]: dispatch 2024-03-21T13:29:15.873 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:15 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3464660016' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi097-79675-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:15.873 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:15 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi097-79675-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:15.873 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:15 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi097-79636-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:15.873 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:15 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:15.873 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:15 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi097-79740-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:29:15.873 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:15 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi097-79828-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:29:15.874 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:15 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi097-79675-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:29:15.874 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:15 smithi144 ceph-mon[33633]: osdmap e80: 8 total, 8 up, 8 in 2024-03-21T13:29:15.874 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:15 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/224160575' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi097-79636-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-03-21T13:29:15.874 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:15 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3695006278' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi097-79740-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi097-79740-10"}]: dispatch 2024-03-21T13:29:15.874 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:15 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2661409599' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi097-79828-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi097-79828-10"}]: dispatch 2024-03-21T13:29:15.874 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:15 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3464660016' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi097-79675-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi097-79675-22"}]: dispatch 2024-03-21T13:29:15.874 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:15 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2488892529' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:15.874 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:15 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3213135837' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi097-80712-1"}]: dispatch 2024-03-21T13:29:15.874 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:15 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi097-79675-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi097-79675-22"}]: dispatch 2024-03-21T13:29:15.874 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:15 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi097-79636-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-03-21T13:29:15.875 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:15 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi097-79740-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi097-79740-10"}]: dispatch 2024-03-21T13:29:15.875 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:15 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:15.875 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:15 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi097-79828-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi097-79828-10"}]: dispatch 2024-03-21T13:29:15.875 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:15 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi097-80712-1"}]: dispatch 2024-03-21T13:29:15.875 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:15 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1371127508' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi097-80014-7"}]: dispatch 2024-03-21T13:29:15.875 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:15 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi097-80014-7"}]: dispatch 2024-03-21T13:29:15.875 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:15 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1371127508' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi097-80014-7"}]: dispatch 2024-03-21T13:29:15.875 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:15 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi097-80014-7"}]: dispatch 2024-03-21T13:29:15.875 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:15 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1371127508' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi097-80014-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:16.541 INFO:tasks.workunit.client.0.smithi097.stdout: cls: Running main() from gmock_main.cc 2024-03-21T13:29:16.541 INFO:tasks.workunit.client.0.smithi097.stdout: cls: [==========] Running 2 tests from 1 test suite. 2024-03-21T13:29:16.541 INFO:tasks.workunit.client.0.smithi097.stdout: cls: [----------] Global test environment set-up. 2024-03-21T13:29:16.541 INFO:tasks.workunit.client.0.smithi097.stdout: cls: [----------] 2 tests from NeoRadosCls 2024-03-21T13:29:16.541 INFO:tasks.workunit.client.0.smithi097.stdout: cls: [ RUN ] NeoRadosCls.DNE 2024-03-21T13:29:16.541 INFO:tasks.workunit.client.0.smithi097.stdout: cls: [ OK ] NeoRadosCls.DNE (12934 ms) 2024-03-21T13:29:16.541 INFO:tasks.workunit.client.0.smithi097.stdout: cls: [ RUN ] NeoRadosCls.RemoteReads 2024-03-21T13:29:16.542 INFO:tasks.workunit.client.0.smithi097.stdout: cls: [ OK ] NeoRadosCls.RemoteReads (3868 ms) 2024-03-21T13:29:16.542 INFO:tasks.workunit.client.0.smithi097.stdout: cls: [----------] 2 tests from NeoRadosCls (16802 ms total) 2024-03-21T13:29:16.542 INFO:tasks.workunit.client.0.smithi097.stdout: cls: 2024-03-21T13:29:16.542 INFO:tasks.workunit.client.0.smithi097.stdout: cls: [----------] Global test environment tear-down 2024-03-21T13:29:16.542 INFO:tasks.workunit.client.0.smithi097.stdout: cls: [==========] 2 tests from 1 test suite ran. (16802 ms total) 2024-03-21T13:29:16.542 INFO:tasks.workunit.client.0.smithi097.stdout: cls: [ PASSED ] 2 tests. 2024-03-21T13:29:16.542 INFO:tasks.workunit.client.0.smithi097.stdout: open_pools_parallel: process_1_[80562]: starting. 2024-03-21T13:29:16.542 INFO:tasks.workunit.client.0.smithi097.stdout: open_pools_parallel: process_1_[80562]: creating pool ceph_test_rados_open_pools_parallel.smithi097-80466 2024-03-21T13:29:16.542 INFO:tasks.workunit.client.0.smithi097.stdout: open_pools_parallel: process_1_[80562]: created object 0... 2024-03-21T13:29:16.542 INFO:tasks.workunit.client.0.smithi097.stdout: open_pools_parallel: process_1_[80562]: created object 25... 2024-03-21T13:29:16.542 INFO:tasks.workunit.client.0.smithi097.stdout: open_pools_parallel: process_1_[80562]: created object 49... 2024-03-21T13:29:16.543 INFO:tasks.workunit.client.0.smithi097.stdout: open_pools_parallel: process_1_[80562]: finishing. 2024-03-21T13:29:16.543 INFO:tasks.workunit.client.0.smithi097.stdout: open_pools_parallel: process_1_[80562]: shutting down. 2024-03-21T13:29:16.543 INFO:tasks.workunit.client.0.smithi097.stdout: open_pools_parallel: process_2_[80563]: starting. 2024-03-21T13:29:16.543 INFO:tasks.workunit.client.0.smithi097.stdout: open_pools_parallel: process_2_[80563]: rados_pool_create. 2024-03-21T13:29:16.543 INFO:tasks.workunit.client.0.smithi097.stdout: open_pools_parallel: process_2_[80563]: rados_ioctx_create. 2024-03-21T13:29:16.543 INFO:tasks.workunit.client.0.smithi097.stdout: open_pools_parallel: process_2_[80563]: shutting down. 2024-03-21T13:29:16.543 INFO:tasks.workunit.client.0.smithi097.stdout: open_pools_parallel: ******************************* 2024-03-21T13:29:16.543 INFO:tasks.workunit.client.0.smithi097.stdout: open_pools_parallel: process_3_[81938]: starting. 2024-03-21T13:29:16.543 INFO:tasks.workunit.client.0.smithi097.stdout: open_pools_parallel: process_3_[81938]: creating pool ceph_test_rados_open_pools_parallel.smithi097-80466 2024-03-21T13:29:16.543 INFO:tasks.workunit.client.0.smithi097.stdout: open_pools_parallel: process_3_[81938]: created object 0... 2024-03-21T13:29:16.543 INFO:tasks.workunit.client.0.smithi097.stdout: open_pools_parallel: process_3_[81938]: created object 25... 2024-03-21T13:29:16.543 INFO:tasks.workunit.client.0.smithi097.stdout: open_pools_parallel: process_3_[81938]: created object 49... 2024-03-21T13:29:16.543 INFO:tasks.workunit.client.0.smithi097.stdout: open_pools_parallel: process_3_[81938]: finishing. 2024-03-21T13:29:16.544 INFO:tasks.workunit.client.0.smithi097.stdout: open_pools_parallel: process_3_[81938]: shutting down. 2024-03-21T13:29:16.544 INFO:tasks.workunit.client.0.smithi097.stdout: open_pools_parallel: ******************************* 2024-03-21T13:29:16.544 INFO:tasks.workunit.client.0.smithi097.stdout: open_pools_parallel: process_4_[81939]: starting. 2024-03-21T13:29:16.544 INFO:tasks.workunit.client.0.smithi097.stdout: open_pools_parallel: process_4_[81939]: rados_pool_create. 2024-03-21T13:29:16.544 INFO:tasks.workunit.client.0.smithi097.stdout: open_pools_parallel: process_4_[81939]: rados_ioctx_create. 2024-03-21T13:29:16.544 INFO:tasks.workunit.client.0.smithi097.stdout: open_pools_parallel: process_4_[81939]: shutting down. 2024-03-21T13:29:16.544 INFO:tasks.workunit.client.0.smithi097.stdout: open_pools_parallel: ******************************* 2024-03-21T13:29:16.544 INFO:tasks.workunit.client.0.smithi097.stdout: open_pools_parallel: ******************************* 2024-03-21T13:29:16.544 INFO:tasks.workunit.client.0.smithi097.stdout: open_pools_parallel: ******* SUCCESS ********** 2024-03-21T13:29:16.544 INFO:tasks.workunit.client.0.smithi097.stdout: cmd: Running main() from gmock_main.cc 2024-03-21T13:29:16.545 INFO:tasks.workunit.client.0.smithi097.stdout: cmd: [==========] Running 3 tests from 1 test suite. 2024-03-21T13:29:16.545 INFO:tasks.workunit.client.0.smithi097.stdout: cmd: [----------] Global test environment set-up. 2024-03-21T13:29:16.545 INFO:tasks.workunit.client.0.smithi097.stdout: cmd: [----------] 3 tests from NeoRadosCmd 2024-03-21T13:29:16.545 INFO:tasks.workunit.client.0.smithi097.stdout: cmd: [ RUN ] NeoRadosCmd.MonDescribe 2024-03-21T13:29:16.545 INFO:tasks.workunit.client.0.smithi097.stdout: cmd: [ OK ] NeoRadosCmd.MonDescribe (9440 ms) 2024-03-21T13:29:16.545 INFO:tasks.workunit.client.0.smithi097.stdout: cmd: [ RUN ] NeoRadosCmd.OSDCmd 2024-03-21T13:29:16.545 INFO:tasks.workunit.client.0.smithi097.stdout: cmd: [ OK ] NeoRadosCmd.OSDCmd (2397 ms) 2024-03-21T13:29:16.545 INFO:tasks.workunit.client.0.smithi097.stdout: cmd: [ RUN ] NeoRadosCmd.PGCmd 2024-03-21T13:29:16.545 INFO:tasks.workunit.client.0.smithi097.stdout: cmd: [ OK ] NeoRadosCmd.PGCmd (4921 ms) 2024-03-21T13:29:16.545 INFO:tasks.workunit.client.0.smithi097.stdout: cmd: [----------] 3 tests from NeoRadosCmd (16758 ms total) 2024-03-21T13:29:16.545 INFO:tasks.workunit.client.0.smithi097.stdout: cmd: 2024-03-21T13:29:16.545 INFO:tasks.workunit.client.0.smithi097.stdout: cmd: [----------] Global test environment tear-down 2024-03-21T13:29:16.546 INFO:tasks.workunit.client.0.smithi097.stdout: cmd: [==========] 3 tests from 1 test suite ran. (16758 ms total) 2024-03-21T13:29:16.546 INFO:tasks.workunit.client.0.smithi097.stdout: cmd: [ PASSED ] 3 tests. 2024-03-21T13:29:16.800 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: Running main() from gmock_main.cc 2024-03-21T13:29:16.800 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: [==========] Running 4 tests from 1 test suite. 2024-03-21T13:29:16.800 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: [----------] Global test environment set-up. 2024-03-21T13:29:16.800 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: [----------] 4 tests from LibRadosCmd 2024-03-21T13:29:16.800 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: [ RUN ] LibRadosCmd.MonDescribe 2024-03-21T13:29:16.800 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: [ OK ] LibRadosCmd.MonDescribe (49 ms) 2024-03-21T13:29:16.800 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: [ RUN ] LibRadosCmd.OSDCmd 2024-03-21T13:29:16.800 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: [ OK ] LibRadosCmd.OSDCmd (55 ms) 2024-03-21T13:29:16.801 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: [ RUN ] LibRadosCmd.PGCmd 2024-03-21T13:29:16.801 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: [ OK ] LibRadosCmd.PGCmd (14973 ms) 2024-03-21T13:29:16.801 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: [ RUN ] LibRadosCmd.WatchLog 2024-03-21T13:29:16.801 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:13.976391+0000 mon.b [INF] from='client.? 172.21.15.97:0/3464660016' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi097-79675-22"}]: dispatch 2024-03-21T13:29:16.801 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:13.979938+0000 mon.c [INF] from='client.? 172.21.15.97:0/2661409599' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi097-79828-10"}]: dispatch 2024-03-21T13:29:16.801 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:14.212975+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi097-79828-10"}]: dispatch 2024-03-21T13:29:16.801 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:14.213553+0000 mon.c [INF] from='client.? 172.21.15.97:0/2661409599' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi097-79828-10"}]: dispatch 2024-03-21T13:29:16.801 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:14.218117+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi097-79828-10"}]: dispatch 2024-03-21T13:29:16.801 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:14.218608+0000 mon.c [INF] from='client.? 172.21.15.97:0/2661409599' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi097-79828-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:16.801 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:14.219044+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi097-79828-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:16.801 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:14.259196+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi097-79675-22"}]: dispatch 2024-03-21T13:29:16.801 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:14.259807+0000 mon.b [INF] from='client.? 172.21.15.97:0/3464660016' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi097-79675-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:16.802 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:14.260037+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi097-79675-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:16.802 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:14.686193+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi097-79636-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:16.802 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:14.686326+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:16.802 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:14.686415+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi097-79740-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:29:16.802 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:14.686499+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi097-79828-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:29:16.802 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:14.686584+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi097-79675-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:29:16.802 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:14.694273+0000 mon.c [INF] from='client.? 172.21.15.97:0/224160575' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi097-79636-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-03-21T13:29:16.802 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:14.694688+0000 mon.c [INF] from='client.? 172.21.15.97:0/3695006278' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi097-79740-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi097-79740-10"}]: dispatch 2024-03-21T13:29:16.802 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:14.695118+0000 mon.c [INF] from='client.? 172.21.15.97:0/2661409599' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi097-79828-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi097-79828-10"}]: dispatch 2024-03-21T13:29:16.802 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:14.697534+0000 mon.b [INF] from='client.? 172.21.15.97:0/3464660016' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi097-79675-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi097-79675-22"}]: dispatch 2024-03-21T13:29:16.802 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:14.700303+0000 mon.b [INF] from='client.? 172.21.15.97:0/2488892529' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:16.802 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:14.700781+0000 mon.b [INF] from='client.? 172.21.15.97:0/3213135837' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi097-80712-1"}]: dispatch 2024-03-21T13:29:16.803 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:14.702258+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi097-79675-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi097-79675-22"}]: dispatch 2024-03-21T13:29:16.803 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:14.702893+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi097-79636-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-03-21T13:29:16.803 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:14.703362+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi097-79740-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi097-79740-10"}]: dispatch 2024-03-21T13:29:16.803 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:14.703794+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:16.803 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:14.704203+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi097-79828-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi097-79828-10"}]: dispatch 2024-03-21T13:29:16.803 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:14.704615+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testp delete_pools_parallel: process_1_[80579]: starting. 2024-03-21T13:29:16.803 INFO:tasks.workunit.client.0.smithi097.stdout: delete_pools_parallel: process_1_[80579]: creating pool ceph_test_rados_delete_pools_parallel.smithi097-80490 2024-03-21T13:29:16.803 INFO:tasks.workunit.client.0.smithi097.stdout: delete_pools_parallel: process_1_[80579]: created object 0... 2024-03-21T13:29:16.803 INFO:tasks.workunit.client.0.smithi097.stdout: delete_pools_parallel: process_1_[80579]: created object 25... 2024-03-21T13:29:16.803 INFO:tasks.workunit.client.0.smithi097.stdout: delete_pools_parallel: process_1_[80579]: created object 49... 2024-03-21T13:29:16.804 INFO:tasks.workunit.client.0.smithi097.stdout: delete_pools_parallel: process_1_[80579]: finishing. 2024-03-21T13:29:16.804 INFO:tasks.workunit.client.0.smithi097.stdout: delete_pools_parallel: process_1_[80579]: shutting down. 2024-03-21T13:29:16.804 INFO:tasks.workunit.client.0.smithi097.stdout: delete_pools_parallel: process_2_[80582]: starting. 2024-03-21T13:29:16.804 INFO:tasks.workunit.client.0.smithi097.stdout: delete_pools_parallel: process_2_[80582]: deleting pool ceph_test_rados_delete_pools_parallel.smithi097-80490 2024-03-21T13:29:16.804 INFO:tasks.workunit.client.0.smithi097.stdout: delete_pools_parallel: process_2_[80582]: shutting down. 2024-03-21T13:29:16.804 INFO:tasks.workunit.client.0.smithi097.stdout: delete_pools_parallel: ******************************* 2024-03-21T13:29:16.804 INFO:tasks.workunit.client.0.smithi097.stdout: delete_pools_parallel: process_3_[81986]: starting. 2024-03-21T13:29:16.804 INFO:tasks.workunit.client.0.smithi097.stdout: delete_pools_parallel: process_3_[81986]: creating pool ceph_test_rados_delete_pools_parallel.smithi097-80490 2024-03-21T13:29:16.804 INFO:tasks.workunit.client.0.smithi097.stdout: delete_pools_parallel: process_3_[81986]: created object 0... 2024-03-21T13:29:16.804 INFO:tasks.workunit.client.0.smithi097.stdout: delete_pools_parallel: process_3_[81986]: created object 25... 2024-03-21T13:29:16.804 INFO:tasks.workunit.client.0.smithi097.stdout: delete_pools_parallel: process_3_[81986]: created object 49... 2024-03-21T13:29:16.804 INFO:tasks.workunit.client.0.smithi097.stdout: delete_pools_parallel: process_3_[81986]: finishing. 2024-03-21T13:29:16.804 INFO:tasks.workunit.client.0.smithi097.stdout: delete_pools_parallel: process_3_[81986]: shutting down. 2024-03-21T13:29:16.805 INFO:tasks.workunit.client.0.smithi097.stdout: delete_pools_parallel: ******************************* 2024-03-21T13:29:16.805 INFO:tasks.workunit.client.0.smithi097.stdout: delete_pools_parallel: process_5_[81988]: starting. 2024-03-21T13:29:16.805 INFO:tasks.workunit.client.0.smithi097.stdout: delete_pools_parallel: process_5_[81988]: listing objects. 2024-03-21T13:29:16.805 INFO:tasks.workunit.client.0.smithi097.stdout: delete_pools_parallel: process_5_[81988]: listed object 0... 2024-03-21T13:29:16.805 INFO:tasks.workunit.client.0.smithi097.stdout: delete_pools_parallel: process_5_[81988]: listed object 25... 2024-03-21T13:29:16.805 INFO:tasks.workunit.client.0.smithi097.stdout: delete_pools_parallel: process_5_[81988]: saw 50 objects 2024-03-21T13:29:16.805 INFO:tasks.workunit.client.0.smithi097.stdout: delete_pools_parallel: process_5_[81988]: shutting down. 2024-03-21T13:29:16.805 INFO:tasks.workunit.client.0.smithi097.stdout: delete_pools_parallel: ******************************* 2024-03-21T13:29:16.805 INFO:tasks.workunit.client.0.smithi097.stdout: delete_pools_parallel: process_4_[81987]: starting. 2024-03-21T13:29:16.805 INFO:tasks.workunit.client.0.smithi097.stdout: delete_pools_parallel: process_4_[81987]: deleting pool ceph_test_rados_delete_pools_parallel.smithi097-80490 2024-03-21T13:29:16.805 INFO:tasks.workunit.client.0.smithi097.stdout: delete_pools_parallel: process_4_[81987]: shutting down. 2024-03-21T13:29:16.805 INFO:tasks.workunit.client.0.smithi097.stdout: delete_pools_parallel: ******************************* 2024-03-21T13:29:16.805 INFO:tasks.workunit.client.0.smithi097.stdout: delete_pools_parallel: ******************************* 2024-03-21T13:29:16.805 INFO:tasks.workunit.client.0.smithi097.stdout: delete_pools_parallel: ******* SUCCESS ********** 2024-03-21T13:29:17.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:16 smithi144 ceph-mon[33633]: pgmap v82: 1092 pgs: 264 creating+peering, 724 active+clean, 104 unknown; 145 MiB data, 733 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 556 op/s 2024-03-21T13:29:17.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:16 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:17.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:16 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3714576222' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi097-80039-7"}]: dispatch 2024-03-21T13:29:17.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:16 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi097-80014-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:17.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:16 smithi097 ceph-mon[39416]: pgmap v82: 1092 pgs: 264 creating+peering, 724 active+clean, 104 unknown; 145 MiB data, 733 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 556 op/s 2024-03-21T13:29:17.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:16 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:17.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:16 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3714576222' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi097-80039-7"}]: dispatch 2024-03-21T13:29:17.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:16 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi097-80014-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:17.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:16 smithi097 ceph-mon[30040]: pgmap v82: 1092 pgs: 264 creating+peering, 724 active+clean, 104 unknown; 145 MiB data, 733 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 556 op/s 2024-03-21T13:29:17.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:16 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:17.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:16 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3714576222' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi097-80039-7"}]: dispatch 2024-03-21T13:29:17.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:16 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi097-80014-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:18.106 INFO:tasks.workunit.client.0.smithi097.stdout:rofile-SimpleWritesmithi097-80712-1"}]: dispatch 2024-03-21T13:29:18.106 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:14.751955+0000 mon.c [INF] from='client.? 172.21.15.97:0/1371127508' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi097-80014-7"}]: dispatch 2024-03-21T13:29:18.106 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:14.752543+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi097-80014-7"}]: dispatch 2024-03-21T13:29:18.107 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:14.761983+0000 mon.c [INF] from='client.? 172.21.15.97:0/1371127508' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi097-80014-7"}]: dispatch 2024-03-21T13:29:18.107 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:15.057440+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi097-80014-7"}]: dispatch 2024-03-21T13:29:18.107 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:15.058009+0000 mon.c [INF] from='client.? 172.21.15.97:0/1371127508' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi097-80014-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:18.107 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:15.790397+0000 mon.a [INF] from='client.? 172.21.15.97:0/3714576222' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi097-80039-7"}]: dispatch 2024-03-21T13:29:18.107 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:15.791898+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi097-80014-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:18.107 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:15.305250+0000 client.admin [INF] onexx 2024-03-21T13:29:18.107 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:15.792338+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi097-79636-3", "field": "max_bytes", "val": "4096"}]': finished 2024-03-21T13:29:18.107 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:15.792424+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:18.107 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:15.792541+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi097-80712-1"}]': finished 2024-03-21T13:29:18.107 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:15.799682+0000 mon.b [INF] from='client.? 172.21.15.97:0/3213135837' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi097-80712-1"}]: dispatch 2024-03-21T13:29:18.107 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:16.789050+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi097-79675-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi097-79675-22"}]': finished 2024-03-21T13:29:18.107 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:16.789160+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi097-79740-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi097-79740-10"}]': finished 2024-03-21T13:29:18.107 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:16.789244+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi097-79828-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi097-79828-10"}]': finished 2024-03-21T13:29:18.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:18 smithi144 ceph-mon[33633]: 15.5 deep-scrub starts 2024-03-21T13:29:18.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:18 smithi144 ceph-mon[33633]: 15.5 deep-scrub ok 2024-03-21T13:29:18.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:18 smithi144 ceph-mon[33633]: onexx 2024-03-21T13:29:18.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:18 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi097-79636-3", "field": "max_bytes", "val": "4096"}]': finished 2024-03-21T13:29:18.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:18 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:18.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:18 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi097-80712-1"}]': finished 2024-03-21T13:29:18.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:18 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3213135837' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi097-80712-1"}]: dispatch 2024-03-21T13:29:18.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:18 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:18.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:18 smithi144 ceph-mon[33633]: pgmap v85: 748 pgs: 98 creating+peering, 458 active+clean, 192 unknown; 145 MiB data, 733 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 547 op/s 2024-03-21T13:29:18.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:18 smithi144 ceph-mon[33633]: osdmap e81: 8 total, 8 up, 8 in 2024-03-21T13:29:18.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:18 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi097-79675-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi097-79675-22"}]': finished 2024-03-21T13:29:18.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:18 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi097-79740-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi097-79740-10"}]': finished 2024-03-21T13:29:18.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:18 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi097-79828-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi097-79828-10"}]': finished 2024-03-21T13:29:18.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:18 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi097-80014-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:29:18.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:18 smithi144 ceph-mon[33633]: osdmap e82: 8 total, 8 up, 8 in 2024-03-21T13:29:18.373 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:18 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1371127508' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi097-80014-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi097-80014-7"}]: dispatch 2024-03-21T13:29:18.373 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:18 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3714576222' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi097-80039-7"}]: dispatch 2024-03-21T13:29:18.373 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:18 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2555609167' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi097-80028-1"}]: dispatch 2024-03-21T13:29:18.373 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:18 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/233801175' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi097-79664-16"}]: dispatch 2024-03-21T13:29:18.373 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:18 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4069198980' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:18.373 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:18 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3714576222' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi097-80039-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:18.373 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:18 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1913222463' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-03-21T13:29:18.373 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:18 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi097-80014-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi097-80014-7"}]: dispatch 2024-03-21T13:29:18.373 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:18 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:18.373 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:18 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi097-80712-1"}]: dispatch 2024-03-21T13:29:18.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:18 smithi097 ceph-mon[39416]: 15.5 deep-scrub starts 2024-03-21T13:29:18.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:18 smithi097 ceph-mon[39416]: 15.5 deep-scrub ok 2024-03-21T13:29:18.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:18 smithi097 ceph-mon[39416]: onexx 2024-03-21T13:29:18.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:18 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi097-79636-3", "field": "max_bytes", "val": "4096"}]': finished 2024-03-21T13:29:18.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:18 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:18.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:18 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi097-80712-1"}]': finished 2024-03-21T13:29:18.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:18 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3213135837' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi097-80712-1"}]: dispatch 2024-03-21T13:29:18.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:18 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:18.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:18 smithi097 ceph-mon[39416]: pgmap v85: 748 pgs: 98 creating+peering, 458 active+clean, 192 unknown; 145 MiB data, 733 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 547 op/s 2024-03-21T13:29:18.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:18 smithi097 ceph-mon[39416]: osdmap e81: 8 total, 8 up, 8 in 2024-03-21T13:29:18.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:18 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi097-79675-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi097-79675-22"}]': finished 2024-03-21T13:29:18.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:18 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi097-79740-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi097-79740-10"}]': finished 2024-03-21T13:29:18.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:18 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi097-79828-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi097-79828-10"}]': finished 2024-03-21T13:29:18.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:18 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi097-80014-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:29:18.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:18 smithi097 ceph-mon[39416]: osdmap e82: 8 total, 8 up, 8 in 2024-03-21T13:29:18.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:18 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1371127508' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi097-80014-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi097-80014-7"}]: dispatch 2024-03-21T13:29:18.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:18 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3714576222' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi097-80039-7"}]: dispatch 2024-03-21T13:29:18.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:18 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2555609167' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi097-80028-1"}]: dispatch 2024-03-21T13:29:18.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:18 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/233801175' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi097-79664-16"}]: dispatch 2024-03-21T13:29:18.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:18 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4069198980' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:18.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:18 smithi097 ceph-mon[30040]: 15.5 deep-scrub starts 2024-03-21T13:29:18.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:18 smithi097 ceph-mon[30040]: 15.5 deep-scrub ok 2024-03-21T13:29:18.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:18 smithi097 ceph-mon[30040]: onexx 2024-03-21T13:29:18.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:18 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi097-79636-3", "field": "max_bytes", "val": "4096"}]': finished 2024-03-21T13:29:18.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:18 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:18.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:18 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi097-80712-1"}]': finished 2024-03-21T13:29:18.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:18 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3213135837' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi097-80712-1"}]: dispatch 2024-03-21T13:29:18.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:18 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:18.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:18 smithi097 ceph-mon[30040]: pgmap v85: 748 pgs: 98 creating+peering, 458 active+clean, 192 unknown; 145 MiB data, 733 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 547 op/s 2024-03-21T13:29:18.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:18 smithi097 ceph-mon[30040]: osdmap e81: 8 total, 8 up, 8 in 2024-03-21T13:29:18.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:18 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi097-79675-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi097-79675-22"}]': finished 2024-03-21T13:29:18.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:18 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi097-79740-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi097-79740-10"}]': finished 2024-03-21T13:29:18.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:18 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi097-79828-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi097-79828-10"}]': finished 2024-03-21T13:29:18.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:18 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi097-80014-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:29:18.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:18 smithi097 ceph-mon[30040]: osdmap e82: 8 total, 8 up, 8 in 2024-03-21T13:29:18.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:18 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1371127508' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi097-80014-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi097-80014-7"}]: dispatch 2024-03-21T13:29:18.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:18 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3714576222' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi097-80039-7"}]: dispatch 2024-03-21T13:29:18.557 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:18 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2555609167' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi097-80028-1"}]: dispatch 2024-03-21T13:29:18.557 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:18 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/233801175' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi097-79664-16"}]: dispatch 2024-03-21T13:29:18.557 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:18 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4069198980' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:18.557 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:18 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3714576222' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi097-80039-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:18.557 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:18 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1913222463' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-03-21T13:29:18.557 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:18 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi097-80014-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi097-80014-7"}]: dispatch 2024-03-21T13:29:18.557 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:18 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:18.557 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:18 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi097-80712-1"}]: dispatch 2024-03-21T13:29:18.557 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:18 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3714576222' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi097-80039-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:18.558 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:18 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1913222463' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-03-21T13:29:18.558 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:18 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi097-80014-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi097-80014-7"}]: dispatch 2024-03-21T13:29:18.558 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:18 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:18.558 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:18 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi097-80712-1"}]: dispatch 2024-03-21T13:29:18.965 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:16.789324+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi097-80014-7", "profile": [ "k=2 api_io_pp: Running main() from gmock_main.cc 2024-03-21T13:29:18.965 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [==========] Running 37 tests from 2 test suites. 2024-03-21T13:29:18.966 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [----------] Global test environment set-up. 2024-03-21T13:29:18.966 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP 2024-03-21T13:29:18.966 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: seed 79675 2024-03-21T13:29:18.966 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TooBigPP 2024-03-21T13:29:18.966 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ OK ] LibRadosIoPP.TooBigPP (0 ms) 2024-03-21T13:29:18.966 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SimpleWritePP 2024-03-21T13:29:18.966 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ OK ] LibRadosIoPP.SimpleWritePP (97 ms) 2024-03-21T13:29:18.966 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadOpPP 2024-03-21T13:29:18.966 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadOpPP (8 ms) 2024-03-21T13:29:18.966 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SparseReadOpPP 2024-03-21T13:29:18.966 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ OK ] LibRadosIoPP.SparseReadOpPP (6 ms) 2024-03-21T13:29:18.966 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP 2024-03-21T13:29:18.966 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP (2 ms) 2024-03-21T13:29:18.967 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP2 2024-03-21T13:29:18.967 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP2 (4 ms) 2024-03-21T13:29:18.967 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ RUN ] LibRadosIoPP.Checksum 2024-03-21T13:29:18.967 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ OK ] LibRadosIoPP.Checksum (2 ms) 2024-03-21T13:29:18.967 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadIntoBufferlist 2024-03-21T13:29:18.967 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadIntoBufferlist (3 ms) 2024-03-21T13:29:18.967 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ RUN ] LibRadosIoPP.OverlappingWriteRoundTripPP 2024-03-21T13:29:18.967 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ OK ] LibRadosIoPP.OverlappingWriteRoundTripPP (6 ms) 2024-03-21T13:29:18.967 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP 2024-03-21T13:29:18.967 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP (2 ms) 2024-03-21T13:29:18.967 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP2 2024-03-21T13:29:18.967 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP2 (2 ms) 2024-03-21T13:29:18.967 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ RUN ] LibRadosIoPP.AppendRoundTripPP 2024-03-21T13:29:18.967 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ OK ] LibRadosIoPP.AppendRoundTripPP (4 ms) 2024-03-21T13:29:18.967 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TruncTestPP 2024-03-21T13:29:18.968 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ OK ] LibRadosIoPP.TruncTestPP (3 ms) 2024-03-21T13:29:18.968 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RemoveTestPP 2024-03-21T13:29:18.968 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ OK ] LibRadosIoPP.RemoveTestPP (2 ms) 2024-03-21T13:29:18.968 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrsRoundTripPP 2024-03-21T13:29:18.968 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrsRoundTripPP (2 ms) 2024-03-21T13:29:18.968 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RmXattrPP 2024-03-21T13:29:18.968 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ OK ] LibRadosIoPP.RmXattrPP (7 ms) 2024-03-21T13:29:18.968 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-03-21T13:29:18.968 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrListPP (3 ms) 2024-03-21T13:29:18.968 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtPP 2024-03-21T13:29:18.968 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtPP (3 ms) 2024-03-21T13:29:18.968 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtDNEPP 2024-03-21T13:29:18.968 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtDNEPP (2 ms) 2024-03-21T13:29:18.968 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtMismatchPP 2024-03-21T13:29:18.968 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtMismatchPP (2 ms) 2024-03-21T13:29:18.969 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP (160 ms total) 2024-03-21T13:29:18.969 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: 2024-03-21T13:29:18.969 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP 2024-03-21T13:29:18.969 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SimpleWritePP 2024-03-21T13:29:18.969 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SimpleWritePP (2118 ms) 2024-03-21T13:29:18.969 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.ReadOpPP 2024-03-21T13:29:18.969 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ OK ] LibRadosIoECPP.ReadOpPP (11 ms) 2024-03-21T13:29:18.969 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SparseReadOpPP 2024-03-21T13:29:18.969 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SparseReadOpPP (5 ms) 2024-03-21T13:29:18.969 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP 2024-03-21T13:29:18.969 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP (5 ms) 2024-03-21T13:29:18.969 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP2 2024-03-21T13:29:18.969 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP2 (5 ms) 2024-03-21T13:29:19.132 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ RUN ] Li api_io: Running main() from gmock_main.cc 2024-03-21T13:29:19.132 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [==========] Running 24 tests from 2 test suites. 2024-03-21T13:29:19.132 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [----------] Global test environment set-up. 2024-03-21T13:29:19.132 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [----------] 14 tests from LibRadosIo 2024-03-21T13:29:19.132 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [ RUN ] LibRadosIo.SimpleWrite 2024-03-21T13:29:19.132 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [ OK ] LibRadosIo.SimpleWrite (4 ms) 2024-03-21T13:29:19.133 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [ RUN ] LibRadosIo.TooBig 2024-03-21T13:29:19.133 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [ OK ] LibRadosIo.TooBig (0 ms) 2024-03-21T13:29:19.133 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [ RUN ] LibRadosIo.ReadTimeout 2024-03-21T13:29:19.133 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: no timeout :/ 2024-03-21T13:29:19.133 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: no timeout :/ 2024-03-21T13:29:19.133 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: no timeout :/ 2024-03-21T13:29:19.133 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: no timeout :/ 2024-03-21T13:29:19.133 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: no timeout :/ 2024-03-21T13:29:19.133 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [ OK ] LibRadosIo.ReadTimeout (20 ms) 2024-03-21T13:29:19.133 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [ RUN ] LibRadosIo.RoundTrip 2024-03-21T13:29:19.133 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [ OK ] LibRadosIo.RoundTrip (7 ms) 2024-03-21T13:29:19.133 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [ RUN ] LibRadosIo.Checksum 2024-03-21T13:29:19.133 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [ OK ] LibRadosIo.Checksum (2 ms) 2024-03-21T13:29:19.133 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [ RUN ] LibRadosIo.OverlappingWriteRoundTrip 2024-03-21T13:29:19.133 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [ OK ] LibRadosIo.OverlappingWriteRoundTrip (4 ms) 2024-03-21T13:29:19.133 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [ RUN ] LibRadosIo.WriteFullRoundTrip 2024-03-21T13:29:19.134 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [ OK ] LibRadosIo.WriteFullRoundTrip (2 ms) 2024-03-21T13:29:19.134 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [ RUN ] LibRadosIo.AppendRoundTrip 2024-03-21T13:29:19.134 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [ OK ] LibRadosIo.AppendRoundTrip (3 ms) 2024-03-21T13:29:19.134 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [ RUN ] LibRadosIo.ZeroLenZero 2024-03-21T13:29:19.134 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [ OK ] LibRadosIo.ZeroLenZero (1 ms) 2024-03-21T13:29:19.134 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [ RUN ] LibRadosIo.TruncTest 2024-03-21T13:29:19.134 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [ OK ] LibRadosIo.TruncTest (3 ms) 2024-03-21T13:29:19.134 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [ RUN ] LibRadosIo.RemoveTest 2024-03-21T13:29:19.134 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [ OK ] LibRadosIo.RemoveTest (2 ms) 2024-03-21T13:29:19.134 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [ RUN ] LibRadosIo.XattrsRoundTrip 2024-03-21T13:29:19.134 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [ OK ] LibRadosIo.XattrsRoundTrip (3 ms) 2024-03-21T13:29:19.134 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [ RUN ] LibRadosIo.RmXattr 2024-03-21T13:29:19.134 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [ OK ] LibRadosIo.RmXattr (6 ms) 2024-03-21T13:29:19.134 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [ RUN ] LibRadosIo.XattrIter 2024-03-21T13:29:19.134 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [ OK ] LibRadosIo.XattrIter (4 ms) 2024-03-21T13:29:19.135 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [----------] 14 tests from LibRadosIo (61 ms total) 2024-03-21T13:29:19.135 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: 2024-03-21T13:29:19.135 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [----------] 10 tests from LibRadosIoEC 2024-03-21T13:29:19.135 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [ RUN ] LibRadosIoEC.SimpleWrite 2024-03-21T13:29:19.135 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [ OK ] LibRadosIoEC.SimpleWrite (1223 ms) 2024-03-21T13:29:19.135 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [ RUN ] LibRadosIoEC.RoundTrip 2024-03-21T13:29:19.135 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [ OK ] LibRadosIoEC.RoundTrip (27 ms) 2024-03-21T13:29:19.135 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [ RUN ] LibRadosIoEC.OverlappingWriteRoundTrip 2024-03-21T13:29:19.135 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [ OK ] LibRadosIoEC.OverlappingWriteRoundTrip (7 ms) 2024-03-21T13:29:19.135 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [ RUN ] LibRadosIoEC.WriteFullRoundTrip 2024-03-21T13:29:19.135 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [ OK ] LibRadosIoEC.WriteFullRoundTrip (6 ms) 2024-03-21T13:29:19.135 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [ RUN ] LibRadosIoEC.AppendRoundTrip 2024-03-21T13:29:19.135 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [ OK ] LibRadosIoEC.AppendRoundTrip (8 ms) 2024-03-21T13:29:19.135 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [ RUN ] LibRadosIoEC.TruncTest 2024-03-21T13:29:19.136 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [ OK ] LibRadosIoEC.TruncTest (6 ms) 2024-03-21T13:29:19.136 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [ RUN ] LibRadosIoEC.RemoveTest 2024-03-21T13:29:19.136 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [ OK ] LibRadosIoEC.RemoveTest (4 ms) 2024-03-21T13:29:19.136 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [ RUN ] LibRadosIoEC.XattrsRoundTrip 2024-03-21T13:29:19.136 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [ OK ] LibRadosIoEC.XattrsRoundTrip (4 ms) 2024-03-21T13:29:19.136 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [ RUN ] LibRadosIoEC.RmXattr 2024-03-21T13:29:19.136 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [ OK ] LibRadosIoEC.RmXattr (11 ms) 2024-03-21T13:29:19.136 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [ RUN ] LibRadosIoEC.XattrIter 2024-03-21T13:29:19.136 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [ OK ] LibRadosIoEC.XattrIter (5 ms) 2024-03-21T13:29:19.136 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [----------] 10 tests from LibRadosIoEC (1301 ms total) 2024-03-21T13:29:19.136 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: 2024-03-21T13:29:19.136 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [----------] Global test environment tear-down 2024-03-21T13:29:19.136 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [==========] 24 tests from 2 test suites ran. (19691 ms total) 2024-03-21T13:29:19.136 INFO:tasks.workunit.client.0.smithi097.stdout: api_io: [ PASSED ] 24 tests. 2024-03-21T13:29:19.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:19 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2555609167' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi097-80028-1"}]': finished 2024-03-21T13:29:19.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:19 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/233801175' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi097-79664-16"}]': finished 2024-03-21T13:29:19.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:19 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4069198980' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:19.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:19 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3714576222' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi097-80039-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:29:19.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:19 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi097-80712-1"}]': finished 2024-03-21T13:29:19.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:19 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1668470557' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi097-80712-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:19.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:19 smithi097 ceph-mon[39416]: osdmap e83: 8 total, 8 up, 8 in 2024-03-21T13:29:19.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:19 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1145239455' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi097-79618-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:19.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:19 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3605606612' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi097-80861-1"}]: dispatch 2024-03-21T13:29:19.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:19 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2488892529' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:19.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:19 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1913222463' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-03-21T13:29:19.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:19 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2555609167' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi097-80028-1"}]: dispatch 2024-03-21T13:29:19.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:19 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/233801175' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi097-79664-16"}]: dispatch 2024-03-21T13:29:19.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:19 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3714576222' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi097-80039-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi097-80039-7"}]: dispatch 2024-03-21T13:29:19.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:19 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi097-79618-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:19.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:19 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:19.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:19 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi097-80712-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:19.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:19 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi097-80861-1"}]: dispatch 2024-03-21T13:29:19.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:19 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2555609167' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi097-80028-1"}]': finished 2024-03-21T13:29:19.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:19 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/233801175' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi097-79664-16"}]': finished 2024-03-21T13:29:19.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:19 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4069198980' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:19.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:19 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3714576222' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi097-80039-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:29:19.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:19 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi097-80712-1"}]': finished 2024-03-21T13:29:19.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:19 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1668470557' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi097-80712-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:19.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:19 smithi097 ceph-mon[30040]: osdmap e83: 8 total, 8 up, 8 in 2024-03-21T13:29:19.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:19 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1145239455' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi097-79618-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:19.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:19 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3605606612' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi097-80861-1"}]: dispatch 2024-03-21T13:29:19.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:19 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2488892529' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:19.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:19 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1913222463' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-03-21T13:29:19.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:19 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2555609167' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi097-80028-1"}]: dispatch 2024-03-21T13:29:19.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:19 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/233801175' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi097-79664-16"}]: dispatch 2024-03-21T13:29:19.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:19 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3714576222' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi097-80039-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi097-80039-7"}]: dispatch 2024-03-21T13:29:19.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:19 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi097-79618-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:19.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:19 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:19.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:19 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi097-80712-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:19.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:19 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi097-80861-1"}]: dispatch 2024-03-21T13:29:19.556 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:19 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:19.556 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:19 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi097-80014-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi097-80014-7"}]': finished 2024-03-21T13:29:19.556 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:19 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2555609167' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi097-80028-1"}]': finished 2024-03-21T13:29:19.556 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:19 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/233801175' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi097-79664-16"}]': finished 2024-03-21T13:29:19.557 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:19 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi097-79618-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:19.557 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:19 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi097-80712-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:29:19.557 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:19 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3605606612' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi097-80861-1"}]: dispatch 2024-03-21T13:29:19.557 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:19 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1668470557' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi097-80712-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi097-80712-2"}]: dispatch 2024-03-21T13:29:19.557 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:19 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:19.557 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:19 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi097-80014-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi097-80014-7"}]': finished 2024-03-21T13:29:19.557 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:19 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2555609167' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi097-80028-1"}]': finished 2024-03-21T13:29:19.557 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:19 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/233801175' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi097-79664-16"}]': finished 2024-03-21T13:29:19.557 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:19 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi097-79618-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:19.557 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:19 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi097-80712-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:29:19.557 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:19 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3605606612' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi097-80861-1"}]: dispatch 2024-03-21T13:29:19.557 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:19 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1668470557' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi097-80712-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi097-80712-2"}]: dispatch 2024-03-21T13:29:19.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:19 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2555609167' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi097-80028-1"}]': finished 2024-03-21T13:29:19.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:19 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/233801175' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi097-79664-16"}]': finished 2024-03-21T13:29:19.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:19 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4069198980' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:19.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:19 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3714576222' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi097-80039-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:29:19.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:19 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi097-80712-1"}]': finished 2024-03-21T13:29:19.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:19 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1668470557' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi097-80712-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:19.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:19 smithi144 ceph-mon[33633]: osdmap e83: 8 total, 8 up, 8 in 2024-03-21T13:29:19.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:19 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1145239455' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi097-79618-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:19.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:19 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3605606612' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi097-80861-1"}]: dispatch 2024-03-21T13:29:19.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:19 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2488892529' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:19.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:19 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1913222463' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-03-21T13:29:19.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:19 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2555609167' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi097-80028-1"}]: dispatch 2024-03-21T13:29:19.624 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:19 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/233801175' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi097-79664-16"}]: dispatch 2024-03-21T13:29:19.624 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:19 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3714576222' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi097-80039-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi097-80039-7"}]: dispatch 2024-03-21T13:29:19.624 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:19 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi097-79618-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:19.624 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:19 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:19.624 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:19 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi097-80712-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:19.624 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:19 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi097-80861-1"}]: dispatch 2024-03-21T13:29:19.624 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:19 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:19.624 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:19 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi097-80014-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi097-80014-7"}]': finished 2024-03-21T13:29:19.624 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:19 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2555609167' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi097-80028-1"}]': finished 2024-03-21T13:29:19.624 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:19 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/233801175' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi097-79664-16"}]': finished 2024-03-21T13:29:19.624 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:19 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi097-79618-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:19.624 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:19 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi097-80712-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:29:19.624 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:19 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3605606612' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi097-80861-1"}]: dispatch 2024-03-21T13:29:19.624 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:19 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1668470557' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi097-80712-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi097-80712-2"}]: dispatch 2024-03-21T13:29:20.406 INFO:tasks.workunit.client.0.smithi097.stdout: watch_notify: Running main() from gmock_main.cc 2024-03-21T13:29:20.406 INFO:tasks.workunit.client.0.smithi097.stdout: watch_notify: [==========] Running 2 tests from 1 test suite. 2024-03-21T13:29:20.406 INFO:tasks.workunit.client.0.smithi097.stdout: watch_notify: [----------] Global test environment set-up. 2024-03-21T13:29:20.406 INFO:tasks.workunit.client.0.smithi097.stdout: watch_notify: [----------] 2 tests from NeoRadosWatchNotify 2024-03-21T13:29:20.406 INFO:tasks.workunit.client.0.smithi097.stdout: watch_notify: [ RUN ] NeoRadosWatchNotify.WatchNotify 2024-03-21T13:29:20.406 INFO:tasks.workunit.client.0.smithi097.stdout: watch_notify: handle_notify cookie 94452011315104 notify_id 322122547200 notifier_gid 24842 2024-03-21T13:29:20.407 INFO:tasks.workunit.client.0.smithi097.stdout: watch_notify: [ OK ] NeoRadosWatchNotify.WatchNotify (11725 ms) 2024-03-21T13:29:20.407 INFO:tasks.workunit.client.0.smithi097.stdout: watch_notify: [ RUN ] NeoRadosWatchNotify.WatchNotifyTimeout 2024-03-21T13:29:20.407 INFO:tasks.workunit.client.0.smithi097.stdout: watch_notify: Trying... 2024-03-21T13:29:20.407 INFO:tasks.workunit.client.0.smithi097.stdout: watch_notify: handle_notify cookie 94452024052752 notify_id 343597383682 notifier_gid 25130 2024-03-21T13:29:20.407 INFO:tasks.workunit.client.0.smithi097.stdout: watch_notify: Waiting for 3.000000000s 2024-03-21T13:29:20.407 INFO:tasks.workunit.client.0.smithi097.stdout: watch_notify: Timed out. 2024-03-21T13:29:20.407 INFO:tasks.workunit.client.0.smithi097.stdout: watch_notify: Flushing... 2024-03-21T13:29:20.407 INFO:tasks.workunit.client.0.smithi097.stdout: watch_notify: Flushed... 2024-03-21T13:29:20.407 INFO:tasks.workunit.client.0.smithi097.stdout: watch_notify: [ OK ] NeoRadosWatchNotify.WatchNotifyTimeout (8788 ms) 2024-03-21T13:29:20.407 INFO:tasks.workunit.client.0.smithi097.stdout: watch_notify: [----------] 2 tests from NeoRadosWatchNotify (20513 ms total) 2024-03-21T13:29:20.407 INFO:tasks.workunit.client.0.smithi097.stdout: watch_notify: 2024-03-21T13:29:20.407 INFO:tasks.workunit.client.0.smithi097.stdout: watch_notify: [----------] Global test environment tear-down 2024-03-21T13:29:20.407 INFO:tasks.workunit.client.0.smithi097.stdout: watch_notify: [==========] 2 tests from 1 test suite ran. (20513 ms total) 2024-03-21T13:29:20.407 INFO:tasks.workunit.client.0.smithi097.stdout: watch_notify: [ PASSED ] 2 tests. 2024-03-21T13:29:20.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:20 smithi097 ceph-mon[30040]: pgmap v88: 860 pgs: 157 creating+peering, 703 active+clean; 145 MiB data, 741 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:29:20.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:20 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi097-80861-1"}]': finished 2024-03-21T13:29:20.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:20 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:20.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:20 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2488892529' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-4", "tierpool": "test-rados-api-smithi097-79906-4-cache"}]: dispatch 2024-03-21T13:29:20.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:20 smithi097 ceph-mon[30040]: twoxx 2024-03-21T13:29:20.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:20 smithi097 ceph-mon[30040]: osdmap e84: 8 total, 8 up, 8 in 2024-03-21T13:29:20.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:20 smithi097 ceph-mon[30040]: Health check update: 15 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:29:20.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:20 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/657726548' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:20.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:20 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi097-80861-1"}]: dispatch 2024-03-21T13:29:20.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:20 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1913222463' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-03-21T13:29:20.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:20 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi097-80712-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi097-80712-2"}]: dispatch 2024-03-21T13:29:20.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:20 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-4", "tierpool": "test-rados-api-smithi097-79906-4-cache"}]: dispatch 2024-03-21T13:29:20.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:20 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:20.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:20 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:20.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:20 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3714576222' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi097-80039-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi097-80039-7"}]': finished 2024-03-21T13:29:20.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:20 smithi097 ceph-mon[39416]: pgmap v88: 860 pgs: 157 creating+peering, 703 active+clean; 145 MiB data, 741 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:29:20.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:20 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi097-80861-1"}]': finished 2024-03-21T13:29:20.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:20 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:20.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:20 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2488892529' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-4", "tierpool": "test-rados-api-smithi097-79906-4-cache"}]: dispatch 2024-03-21T13:29:20.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:20 smithi097 ceph-mon[39416]: twoxx 2024-03-21T13:29:20.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:20 smithi097 ceph-mon[39416]: osdmap e84: 8 total, 8 up, 8 in 2024-03-21T13:29:20.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:20 smithi097 ceph-mon[39416]: Health check update: 15 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:29:20.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:20 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/657726548' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:20.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:20 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi097-80861-1"}]: dispatch 2024-03-21T13:29:20.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:20 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1913222463' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-03-21T13:29:20.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:20 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi097-80712-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi097-80712-2"}]: dispatch 2024-03-21T13:29:20.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:20 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-4", "tierpool": "test-rados-api-smithi097-79906-4-cache"}]: dispatch 2024-03-21T13:29:20.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:20 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:20.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:20 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:20.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:20 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3714576222' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi097-80039-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi097-80039-7"}]': finished 2024-03-21T13:29:20.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:20 smithi144 ceph-mon[33633]: pgmap v88: 860 pgs: 157 creating+peering, 703 active+clean; 145 MiB data, 741 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:29:20.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:20 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi097-80861-1"}]': finished 2024-03-21T13:29:20.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:20 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:20.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:20 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2488892529' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-4", "tierpool": "test-rados-api-smithi097-79906-4-cache"}]: dispatch 2024-03-21T13:29:20.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:20 smithi144 ceph-mon[33633]: twoxx 2024-03-21T13:29:20.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:20 smithi144 ceph-mon[33633]: osdmap e84: 8 total, 8 up, 8 in 2024-03-21T13:29:20.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:20 smithi144 ceph-mon[33633]: Health check update: 15 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:29:20.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:20 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/657726548' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:20.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:20 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi097-80861-1"}]: dispatch 2024-03-21T13:29:20.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:20 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1913222463' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-03-21T13:29:20.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:20 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi097-80712-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi097-80712-2"}]: dispatch 2024-03-21T13:29:20.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:20 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-4", "tierpool": "test-rados-api-smithi097-79906-4-cache"}]: dispatch 2024-03-21T13:29:20.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:20 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:20.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:20 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:20.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:20 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3714576222' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi097-80039-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi097-80039-7"}]': finished 2024-03-21T13:29:21.186 INFO:tasks.workunit.client.0.smithi097.stdout: list: Running main() from gmock_main.cc 2024-03-21T13:29:21.186 INFO:tasks.workunit.client.0.smithi097.stdout: list: [==========] Running 3 tests from 1 test suite. 2024-03-21T13:29:21.186 INFO:tasks.workunit.client.0.smithi097.stdout: list: [----------] Global test environment set-up. 2024-03-21T13:29:21.186 INFO:tasks.workunit.client.0.smithi097.stdout: list: [----------] 3 tests from NeoradosList 2024-03-21T13:29:21.186 INFO:tasks.workunit.client.0.smithi097.stdout: list: [ RUN ] NeoradosList.ListObjects 2024-03-21T13:29:21.187 INFO:tasks.workunit.client.0.smithi097.stdout: list: [ OK ] NeoradosList.ListObjects (11453 ms) 2024-03-21T13:29:21.187 INFO:tasks.workunit.client.0.smithi097.stdout: list: [ RUN ] NeoradosList.ListObjectsNS 2024-03-21T13:29:21.187 INFO:tasks.workunit.client.0.smithi097.stdout: list: [ OK ] NeoradosList.ListObjectsNS (5255 ms) 2024-03-21T13:29:21.187 INFO:tasks.workunit.client.0.smithi097.stdout: list: [ RUN ] NeoradosList.ListObjectsMany 2024-03-21T13:29:21.187 INFO:tasks.workunit.client.0.smithi097.stdout: list: [ OK ] NeoradosList.ListObjectsMany (4646 ms) 2024-03-21T13:29:21.187 INFO:tasks.workunit.client.0.smithi097.stdout: list: [----------] 3 tests from NeoradosList (21355 ms total) 2024-03-21T13:29:21.187 INFO:tasks.workunit.client.0.smithi097.stdout: list: 2024-03-21T13:29:21.187 INFO:tasks.workunit.client.0.smithi097.stdout: list: [----------] Global test environment tear-down 2024-03-21T13:29:21.187 INFO:tasks.workunit.client.0.smithi097.stdout: list: [==========] 3 tests from 1 test suite ran. (21355 ms total) 2024-03-21T13:29:21.187 INFO:tasks.workunit.client.0.smithi097.stdout: list: [ PASSED ] 3 tests. 2024-03-21T13:29:22.001 INFO:tasks.workunit.client.0.smithi097.stdout:bRadosIoECPP.OverlappingWriteRoundTripPP 2024-03-21T13:29:22.001 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ OK ] LibRadosIoECPP.OverlappingWriteRoundTripPP (5 ms) 2024-03-21T13:29:22.001 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP 2024-03-21T13:29:22.001 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP (6 ms) 2024-03-21T13:29:22.002 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP2 2024-03-21T13:29:22.002 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP2 (5 ms) 2024-03-21T13:29:22.002 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.AppendRoundTripPP 2024-03-21T13:29:22.002 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ OK ] LibRadosIoECPP.AppendRoundTripPP (6 ms) 2024-03-21T13:29:22.002 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.TruncTestPP 2024-03-21T13:29:22.002 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ OK ] LibRadosIoECPP.TruncTestPP (4 ms) 2024-03-21T13:29:22.002 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RemoveTestPP 2024-03-21T13:29:22.002 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RemoveTestPP (2 ms) 2024-03-21T13:29:22.002 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrsRoundTripPP 2024-03-21T13:29:22.002 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrsRoundTripPP (3 ms) 2024-03-21T13:29:22.002 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RmXattrPP 2024-03-21T13:29:22.002 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RmXattrPP (9 ms) 2024-03-21T13:29:22.002 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrListPP 2024-03-21T13:29:22.002 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrListPP (4 ms) 2024-03-21T13:29:22.003 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtPP 2024-03-21T13:29:22.003 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtPP (7 ms) 2024-03-21T13:29:22.003 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtDNEPP 2024-03-21T13:29:22.003 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtDNEPP (4 ms) 2024-03-21T13:29:22.003 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtMismatchPP 2024-03-21T13:29:22.003 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtMismatchPP (6 ms) 2024-03-21T13:29:22.003 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP (2205 ms total) 2024-03-21T13:29:22.003 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: 2024-03-21T13:29:22.003 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [----------] Global test environment tear-down 2024-03-21T13:29:22.003 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [==========] 37 tests from 2 test suites ran. (22557 ms total) 2024-03-21T13:29:22.003 INFO:tasks.workunit.client.0.smithi097.stdout: api_io_pp: [ PASSED ] 37 tests. 2024-03-21T13:29:22.053 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:29:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:29:22.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi097-80861-1"}]': finished 2024-03-21T13:29:22.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-4", "tierpool": "test-rados-api-smithi097-79906-4-cache"}]': finished 2024-03-21T13:29:22.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:22.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2488892529' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-4", "overlaypool": "test-rados-api-smithi097-79906-4-cache"}]: dispatch 2024-03-21T13:29:22.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:22.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[39416]: pgmap v91: 836 pgs: 176 unknown, 73 creating+peering, 587 active+clean; 145 MiB data, 741 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 4.0 KiB/s wr, 4 op/s 2024-03-21T13:29:22.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[39416]: osdmap e85: 8 total, 8 up, 8 in 2024-03-21T13:29:22.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3464660016' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi097-79675-22"}]: dispatch 2024-03-21T13:29:22.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2696340779' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:22.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/795380094' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:22.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-4", "overlaypool": "test-rados-api-smithi097-79906-4-cache"}]: dispatch 2024-03-21T13:29:22.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1913222463' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-03-21T13:29:22.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:22.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1626342189' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi097-80861-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:22.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi097-79675-22"}]: dispatch 2024-03-21T13:29:22.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:22.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:22.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi097-80712-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi097-80712-2"}]': finished 2024-03-21T13:29:22.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi097-80861-1"}]': finished 2024-03-21T13:29:22.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-4", "tierpool": "test-rados-api-smithi097-79906-4-cache"}]': finished 2024-03-21T13:29:22.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:22.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2488892529' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-4", "overlaypool": "test-rados-api-smithi097-79906-4-cache"}]: dispatch 2024-03-21T13:29:22.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:22.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[30040]: pgmap v91: 836 pgs: 176 unknown, 73 creating+peering, 587 active+clean; 145 MiB data, 741 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 4.0 KiB/s wr, 4 op/s 2024-03-21T13:29:22.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[30040]: osdmap e85: 8 total, 8 up, 8 in 2024-03-21T13:29:22.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3464660016' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi097-79675-22"}]: dispatch 2024-03-21T13:29:22.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2696340779' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:22.056 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/795380094' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:22.056 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-4", "overlaypool": "test-rados-api-smithi097-79906-4-cache"}]: dispatch 2024-03-21T13:29:22.056 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1913222463' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-03-21T13:29:22.056 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:22.056 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1626342189' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi097-80861-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:22.056 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi097-79675-22"}]: dispatch 2024-03-21T13:29:22.056 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:22.056 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:22.056 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi097-80712-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi097-80712-2"}]': finished 2024-03-21T13:29:22.056 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-4", "overlaypool": "test-rados-api-smithi097-79906-4-cache"}]': finished 2024-03-21T13:29:22.056 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:22.056 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1626342189' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi097-80861-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:29:22.056 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi097-79675-22"}]': finished 2024-03-21T13:29:22.056 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:22.056 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2488892529' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-4-cache", "mode": "writeback"}]: dispatch 2024-03-21T13:29:22.057 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3464660016' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi097-79675-22"}]: dispatch 2024-03-21T13:29:22.057 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[30040]: osdmap e86: 8 total, 8 up, 8 in 2024-03-21T13:29:22.057 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2207685738' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi097-79618-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:22.057 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1371127508' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi097-80014-7"}]: dispatch 2024-03-21T13:29:22.057 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1626342189' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi097-80861-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi097-80861-2"}]: dispatch 2024-03-21T13:29:22.057 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-4-cache", "mode": "writeback"}]: dispatch 2024-03-21T13:29:22.057 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi097-79675-22"}]: dispatch 2024-03-21T13:29:22.057 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi097-79618-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:22.057 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi097-80014-7"}]: dispatch 2024-03-21T13:29:22.057 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3480244863' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi097-80011-12"}]: dispatch 2024-03-21T13:29:22.057 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi097-80011-12"}]: dispatch 2024-03-21T13:29:22.057 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3480244863' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi097-80011-12"}]: dispatch 2024-03-21T13:29:22.057 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi097-80011-12"}]: dispatch 2024-03-21T13:29:22.057 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3480244863' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi097-80011-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:22.057 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi097-80011-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:22.058 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-4", "overlaypool": "test-rados-api-smithi097-79906-4-cache"}]': finished 2024-03-21T13:29:22.058 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:22.058 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1626342189' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi097-80861-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:29:22.058 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi097-79675-22"}]': finished 2024-03-21T13:29:22.058 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:22.058 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2488892529' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-4-cache", "mode": "writeback"}]: dispatch 2024-03-21T13:29:22.058 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3464660016' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi097-79675-22"}]: dispatch 2024-03-21T13:29:22.058 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[39416]: osdmap e86: 8 total, 8 up, 8 in 2024-03-21T13:29:22.058 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2207685738' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi097-79618-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:22.058 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1371127508' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi097-80014-7"}]: dispatch 2024-03-21T13:29:22.058 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1626342189' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi097-80861-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi097-80861-2"}]: dispatch 2024-03-21T13:29:22.058 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-4-cache", "mode": "writeback"}]: dispatch 2024-03-21T13:29:22.059 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi097-79675-22"}]: dispatch 2024-03-21T13:29:22.059 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi097-79618-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:22.059 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi097-80014-7"}]: dispatch 2024-03-21T13:29:22.059 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3480244863' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi097-80011-12"}]: dispatch 2024-03-21T13:29:22.059 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi097-80011-12"}]: dispatch 2024-03-21T13:29:22.059 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3480244863' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi097-80011-12"}]: dispatch 2024-03-21T13:29:22.059 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi097-80011-12"}]: dispatch 2024-03-21T13:29:22.059 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3480244863' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi097-80011-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:22.059 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:21 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi097-80011-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:22.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:21 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi097-80861-1"}]': finished 2024-03-21T13:29:22.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:21 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-4", "tierpool": "test-rados-api-smithi097-79906-4-cache"}]': finished 2024-03-21T13:29:22.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:21 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:22.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:21 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2488892529' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-4", "overlaypool": "test-rados-api-smithi097-79906-4-cache"}]: dispatch 2024-03-21T13:29:22.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:21 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:22.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:21 smithi144 ceph-mon[33633]: pgmap v91: 836 pgs: 176 unknown, 73 creating+peering, 587 active+clean; 145 MiB data, 741 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 4.0 KiB/s wr, 4 op/s 2024-03-21T13:29:22.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:21 smithi144 ceph-mon[33633]: osdmap e85: 8 total, 8 up, 8 in 2024-03-21T13:29:22.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:21 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3464660016' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi097-79675-22"}]: dispatch 2024-03-21T13:29:22.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:21 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2696340779' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:22.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:21 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/795380094' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:22.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:21 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-4", "overlaypool": "test-rados-api-smithi097-79906-4-cache"}]: dispatch 2024-03-21T13:29:22.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:21 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1913222463' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-03-21T13:29:22.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:21 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:22.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:21 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1626342189' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi097-80861-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:22.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:21 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi097-79675-22"}]: dispatch 2024-03-21T13:29:22.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:21 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:22.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:21 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:22.123 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:21 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi097-80712-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi097-80712-2"}]': finished 2024-03-21T13:29:22.123 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:21 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-4", "overlaypool": "test-rados-api-smithi097-79906-4-cache"}]': finished 2024-03-21T13:29:22.123 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:21 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:22.123 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:21 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1626342189' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi097-80861-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:29:22.123 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:21 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi097-79675-22"}]': finished 2024-03-21T13:29:22.123 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:21 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:22.123 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:21 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2488892529' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-4-cache", "mode": "writeback"}]: dispatch 2024-03-21T13:29:22.123 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:21 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3464660016' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi097-79675-22"}]: dispatch 2024-03-21T13:29:22.123 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:21 smithi144 ceph-mon[33633]: osdmap e86: 8 total, 8 up, 8 in 2024-03-21T13:29:22.123 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:21 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2207685738' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi097-79618-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:22.123 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:21 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1371127508' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi097-80014-7"}]: dispatch 2024-03-21T13:29:22.123 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:21 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1626342189' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi097-80861-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi097-80861-2"}]: dispatch 2024-03-21T13:29:22.123 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:21 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-4-cache", "mode": "writeback"}]: dispatch 2024-03-21T13:29:22.123 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:21 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi097-79675-22"}]: dispatch 2024-03-21T13:29:22.124 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:21 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi097-79618-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:22.124 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:21 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi097-80014-7"}]: dispatch 2024-03-21T13:29:22.124 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:21 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3480244863' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi097-80011-12"}]: dispatch 2024-03-21T13:29:22.124 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:21 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi097-80011-12"}]: dispatch 2024-03-21T13:29:22.124 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:21 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3480244863' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi097-80011-12"}]: dispatch 2024-03-21T13:29:22.124 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:21 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi097-80011-12"}]: dispatch 2024-03-21T13:29:22.124 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:21 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3480244863' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi097-80011-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:22.124 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:21 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi097-80011-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:22.759 INFO:tasks.workunit.client.0.smithi097.stdout:", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:29:22.760 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:16.795157+0000 mon.c [INF] from='client.? 172.21.15.97:0/1371127508' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi097-80014-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi097-80014-7"}]: dispatch 2024-03-21T13:29:22.760 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:16.799916+0000 mon.a [INF] from='client.? 172.21.15.97:0/3714576222' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi097-80039-7"}]: dispatch 2024-03-21T13:29:22.760 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:16.802658+0000 mon.a [INF] from='client.? 172.21.15.97:0/2555609167' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi097-80028-1"}]: dispatch 2024-03-21T13:29:22.760 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:16.802910+0000 mon.a [INF] from='client.? 172.21.15.97:0/233801175' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi097-79664-16"}]: dispatch 2024-03-21T13:29:22.760 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:16.804130+0000 mon.a [INF] from='client.? 172.21.15.97:0/4069198980' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:22.760 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:16.815140+0000 mon.a [INF] from='client.? 172.21.15.97:0/3714576222' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi097-80039-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:22.760 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:16.815469+0000 mon.a [INF] from='client.? 172.21.15.97:0/1913222463' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-03-21T13:29:22.760 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:17.068790+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi097-80014-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi097-80014-7"}]: dispatch 2024-03-21T13:29:22.760 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:17.329371+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi097-80712-1"}]: dispatch 2024-03-21T13:29:22.760 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:20.137629+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi097-80861-1"}]': finished 2024-03-21T13:29:22.760 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:20.137689+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-4", "tierpool": "test-rados-api-smithi097-79906-4-cache"}]': finished 2024-03-21T13:29:22.760 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:20.137739+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:22.761 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:20.144621+0000 mon.b [INF] from='client.? 172.21.15.97:0/2488892529' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-4", "overlaypool": "test-rados-api-smithi097-79906-4-cache"}]: dispatch 2024-03-21T13:29:22.761 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:20.403641+0000 mon.b [INF] from='client.? 172.21.15.97:0/3464660016' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi097-79675-22"}]: dispatch 2024-03-21T13:29:22.761 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:20.404557+0000 mon.c [INF] from='client.? 172.21.15.97:0/2696340779' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:22.761 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:20.405084+0000 mon.b [INF] from='client.? 172.21.15.97:0/795380094' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:22.761 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:20.409829+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-4", "overlaypool": "test-rados-api-smithi097-79906-4-cache"}]: dispatch 2024-03-21T13:29:22.761 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:20.418226+0000 mon.a [INF] from='client.? 172.21.15.97:0/1913222463' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-03-21T13:29:22.761 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:20.423739+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:22.761 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:20.664054+0000 mon.a [INF] from='client.? 172.21.15.97:0/1626342189' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi097-80861-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:22.761 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:20.911397+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi097-79675-22"}]: dispatch 2024-03-21T13:29:22.761 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:20.911749+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:22.761 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:21.160166+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi097-80712-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi097-80712-2"}]': finished 2024-03-21T13:29:22.761 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:21.160244+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-4", "overlaypool": "test-rados-api-smithi097-79906-4-cache"}]': finished 2024-03-21T13:29:22.761 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:21.160314+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:22.761 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:21.160377+0000 mon.a [INF] from='client.? 172.21.15.97:0/1626342189' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi097-80861-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:29:22.762 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:21.160438+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi097-79675-22"}]': finished 2024-03-21T13:29:22.762 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:21.160499+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:22.762 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:21.165178+0000 mon.b [INF] from='client.? 172.21.15.97:0/2488892529' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-4-cache", "mode": "writeback"}]: dispatch 2024-03-21T13:29:22.762 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:21.165289+0000 mon.b [INF] from='client.? 172.21.15.97:0/3464660016' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi097-79675-22"}]: dispatch 2024-03-21T13:29:23.007 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:21.175492+0000 mon.c [INF] from='client.? 172.21.15.97:0/2207685738' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi097-79618-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:23.007 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:21.175660+0000 mon.c [INF] from='client.? 172.21.15.97:0/1371127508' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi097-80014-7"}]: dispatch 2024-03-21T13:29:23.008 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:21.182747+0000 mon.a [INF] from='client.? 172.21.15.97:0/1626342189' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi097-80861-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi097-80861-2"}]: dispatch 2024-03-21T13:29:23.008 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:21.183827+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-4-cache", "mode": "writeback"}]: dispatch 2024-03-21T13:29:23.008 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:21.184138+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi097-79675-22"}]: dispatch 2024-03-21T13:29:23.008 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:21.206436+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi097-79618-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:23.008 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:21.208438+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi097-80014-7"}]: dispatch 2024-03-21T13:29:23.008 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:21.227192+0000 mon.c [INF] from='client.? 172.21.15.97:0/3480244863' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi097-80011-12"}]: dispatch 2024-03-21T13:29:23.008 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:21.228733+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi097-80011-12"}]: dispatch 2024-03-21T13:29:23.008 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:21.230528+0000 mon.c [INF] from='client.? 172.21.15.97:0/3480244863' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi097-80011-12"}]: dispatch 2024-03-21T13:29:23.008 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:21.231205+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi097-80011-12"}]: dispatch 2024-03-21T13:29:23.008 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:21.236885+0000 mon.c [INF] from='client.? 172.21.15.97:0/3480244863' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi097-80011-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:23.008 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:21.237732+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi097-80011-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:23.008 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:21.313369+0000 mon.b [INF] from='client.? 172.21.15.97:0/795380094' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi097-80028-6", "pg_num": 4}]: dispatch 2024-03-21T13:29:23.009 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:21.748037+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi097-80028-6", "pg_num": 4}]: dispatch 2024-03-21T13:29:23.009 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:21.984889+0000 mon.a [WRN] pool 'PoolQuotaPP_smithi097-79636-3' is full (reached quota's max_bytes: 4 KiB) 2024-03-21T13:29:23.009 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:21.986553+0000 mon.a [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:29:23.009 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:21.986575+0000 mon.a [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-21T13:29:23.009 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:21.988335+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-4-cache", "mode": "writeback"}]': finished 2024-03-21T13:29:23.009 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:21.988401+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi097-79675-22"}]': finished 2024-03-21T13:29:23.009 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:21.988469+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi097-79618-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:23.009 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:21.988526+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi097-80014-7"}]': finished 2024-03-21T13:29:23.009 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:21.988594+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi097-80011-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:29:23.009 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:21.988651+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi097-80028-6", "pg_num": 4}]': finished 2024-03-21T13:29:23.009 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:21.993029+0000 mon.c [INF] from='client.? 172.21.15.97:0/1371127508' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi097-80014-7"}]: dispatch 2024-03-21T13:29:23.009 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:21.993190+0000 mon.c [INF] from='client.? 172.21.15.97:0/3480244863' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi097-80011-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi097-80011-12"}]: dispatch 2024-03-21T13:29:23.009 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:21.993419+0000 mon.b [INF] from='client.? 172.21.15.97:0/795380094' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4", "tierpool": "test-rados-api-smithi097-80028-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:29:23.009 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:22.002498+0000 mon.c [INF] from='client.? 172.21.15.97:0/3695006278' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi097-79740-10"}]: dispatch 2024-03-21T13:29:23.010 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:22.002909+0000 mon.c [INF] from='client.? 172.21.15.97:0/2661409599' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi097-79828-10"}]: dispatch 2024-03-21T13:29:23.010 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:22.088789+0000 mon.b [INF] from='client.? 172.21.15.97:0/2488892529' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-4"}]: dispatch 2024-03-21T13:29:23.010 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:22.261394+0000 mon.c [INF] from='client.? 172.21.15.97:0/3484349219' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:23.010 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:22.264234+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4", "tierpool": "test-rados-api-smithi097-80028-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:29:23.010 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:22.264480+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi097-80014-7"}]: dispatch 2024-03-21T13:29:23.029 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:22.264949+0000 mon.a [INF] from='client.? api_stat: Running main() from gmock_main.cc 2024-03-21T13:29:23.029 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat: [==========] Running 9 tests from 2 test suites. 2024-03-21T13:29:23.029 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat: [----------] Global test environment set-up. 2024-03-21T13:29:23.029 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat: [----------] 5 tests from LibRadosStat 2024-03-21T13:29:23.029 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat: [ RUN ] LibRadosStat.Stat 2024-03-21T13:29:23.029 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat: [ OK ] LibRadosStat.Stat (347 ms) 2024-03-21T13:29:23.029 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat: [ RUN ] LibRadosStat.Stat2 2024-03-21T13:29:23.029 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat: [ OK ] LibRadosStat.Stat2 (43 ms) 2024-03-21T13:29:23.029 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat: [ RUN ] LibRadosStat.StatNS 2024-03-21T13:29:23.029 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat: [ OK ] LibRadosStat.StatNS (25 ms) 2024-03-21T13:29:23.029 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat: [ RUN ] LibRadosStat.ClusterStat 2024-03-21T13:29:23.029 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat: [ OK ] LibRadosStat.ClusterStat (0 ms) 2024-03-21T13:29:23.030 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat: [ RUN ] LibRadosStat.PoolStat 2024-03-21T13:29:23.030 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat: [ OK ] LibRadosStat.PoolStat (2 ms) 2024-03-21T13:29:23.030 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat: [----------] 5 tests from LibRadosStat (417 ms total) 2024-03-21T13:29:23.030 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat: 2024-03-21T13:29:23.030 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat: [----------] 4 tests from LibRadosStatEC 2024-03-21T13:29:23.030 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat: [ RUN ] LibRadosStatEC.Stat 2024-03-21T13:29:23.030 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat: [ OK ] LibRadosStatEC.Stat (1383 ms) 2024-03-21T13:29:23.030 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat: [ RUN ] LibRadosStatEC.StatNS 2024-03-21T13:29:23.030 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat: [ OK ] LibRadosStatEC.StatNS (6 ms) 2024-03-21T13:29:23.030 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat: [ RUN ] LibRadosStatEC.ClusterStat 2024-03-21T13:29:23.030 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat: [ OK ] LibRadosStatEC.ClusterStat (0 ms) 2024-03-21T13:29:23.030 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat: [ RUN ] LibRadosStatEC.PoolStat 2024-03-21T13:29:23.030 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat: [ OK ] LibRadosStatEC.PoolStat (2 ms) 2024-03-21T13:29:23.030 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat: [----------] 4 tests from LibRadosStatEC (1391 ms total) 2024-03-21T13:29:23.031 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat: 2024-03-21T13:29:23.031 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat: [----------] Global test environment tear-down 2024-03-21T13:29:23.031 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat: [==========] 9 tests from 2 test suites ran. (23462 ms total) 2024-03-21T13:29:23.031 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat: [ PASSED ] 9 tests. 2024-03-21T13:29:23.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[39416]: 15.2 deep-scrub starts 2024-03-21T13:29:23.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[39416]: 15.2 deep-scrub ok 2024-03-21T13:29:23.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/795380094' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi097-80028-6", "pg_num": 4}]: dispatch 2024-03-21T13:29:23.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi097-80028-6", "pg_num": 4}]: dispatch 2024-03-21T13:29:23.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[39416]: pool 'PoolQuotaPP_smithi097-79636-3' is full (reached quota's max_bytes: 4 KiB) 2024-03-21T13:29:23.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[39416]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:29:23.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[39416]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-21T13:29:23.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-4-cache", "mode": "writeback"}]': finished 2024-03-21T13:29:23.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi097-79675-22"}]': finished 2024-03-21T13:29:23.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi097-79618-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:23.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi097-80014-7"}]': finished 2024-03-21T13:29:23.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi097-80011-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:29:23.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi097-80028-6", "pg_num": 4}]': finished 2024-03-21T13:29:23.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1371127508' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi097-80014-7"}]: dispatch 2024-03-21T13:29:23.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3480244863' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi097-80011-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi097-80011-12"}]: dispatch 2024-03-21T13:29:23.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/795380094' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4", "tierpool": "test-rados-api-smithi097-80028-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:29:23.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3695006278' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi097-79740-10"}]: dispatch 2024-03-21T13:29:23.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2661409599' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi097-79828-10"}]: dispatch 2024-03-21T13:29:23.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2488892529' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-4"}]: dispatch 2024-03-21T13:29:23.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:23.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[39416]: osdmap e87: 8 total, 8 up, 8 in 2024-03-21T13:29:23.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3484349219' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:23.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4", "tierpool": "test-rados-api-smithi097-80028-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:29:23.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi097-80014-7"}]: dispatch 2024-03-21T13:29:23.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi097-80011-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi097-80011-12"}]: dispatch 2024-03-21T13:29:23.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi097-79740-10"}]: dispatch 2024-03-21T13:29:23.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi097-79828-10"}]: dispatch 2024-03-21T13:29:23.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:23.306 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3714576222' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi097-80039-7"}]: dispatch 2024-03-21T13:29:23.306 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-4"}]: dispatch 2024-03-21T13:29:23.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[30040]: 15.2 deep-scrub starts 2024-03-21T13:29:23.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[30040]: 15.2 deep-scrub ok 2024-03-21T13:29:23.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/795380094' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi097-80028-6", "pg_num": 4}]: dispatch 2024-03-21T13:29:23.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi097-80028-6", "pg_num": 4}]: dispatch 2024-03-21T13:29:23.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[30040]: pool 'PoolQuotaPP_smithi097-79636-3' is full (reached quota's max_bytes: 4 KiB) 2024-03-21T13:29:23.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[30040]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:29:23.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[30040]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-21T13:29:23.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-4-cache", "mode": "writeback"}]': finished 2024-03-21T13:29:23.307 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi097-79675-22"}]': finished 2024-03-21T13:29:23.307 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi097-79618-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:23.307 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi097-80014-7"}]': finished 2024-03-21T13:29:23.307 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi097-80011-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:29:23.307 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi097-80028-6", "pg_num": 4}]': finished 2024-03-21T13:29:23.307 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1371127508' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi097-80014-7"}]: dispatch 2024-03-21T13:29:23.307 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3480244863' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi097-80011-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi097-80011-12"}]: dispatch 2024-03-21T13:29:23.307 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/795380094' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4", "tierpool": "test-rados-api-smithi097-80028-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:29:23.307 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3695006278' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi097-79740-10"}]: dispatch 2024-03-21T13:29:23.307 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2661409599' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi097-79828-10"}]: dispatch 2024-03-21T13:29:23.307 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2488892529' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-4"}]: dispatch 2024-03-21T13:29:23.307 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:23.307 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[30040]: osdmap e87: 8 total, 8 up, 8 in 2024-03-21T13:29:23.307 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3484349219' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:23.307 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4", "tierpool": "test-rados-api-smithi097-80028-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:29:23.307 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi097-80014-7"}]: dispatch 2024-03-21T13:29:23.308 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi097-80011-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi097-80011-12"}]: dispatch 2024-03-21T13:29:23.308 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi097-79740-10"}]: dispatch 2024-03-21T13:29:23.308 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi097-79828-10"}]: dispatch 2024-03-21T13:29:23.308 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:23.308 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3714576222' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi097-80039-7"}]: dispatch 2024-03-21T13:29:23.308 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:23 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-4"}]: dispatch 2024-03-21T13:29:23.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:23 smithi144 ceph-mon[33633]: 15.2 deep-scrub starts 2024-03-21T13:29:23.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:23 smithi144 ceph-mon[33633]: 15.2 deep-scrub ok 2024-03-21T13:29:23.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:23 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/795380094' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi097-80028-6", "pg_num": 4}]: dispatch 2024-03-21T13:29:23.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:23 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi097-80028-6", "pg_num": 4}]: dispatch 2024-03-21T13:29:23.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:23 smithi144 ceph-mon[33633]: pool 'PoolQuotaPP_smithi097-79636-3' is full (reached quota's max_bytes: 4 KiB) 2024-03-21T13:29:23.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:23 smithi144 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:29:23.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:23 smithi144 ceph-mon[33633]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-21T13:29:23.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:23 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-4-cache", "mode": "writeback"}]': finished 2024-03-21T13:29:23.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:23 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi097-79675-22"}]': finished 2024-03-21T13:29:23.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:23 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi097-79618-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:23.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:23 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi097-80014-7"}]': finished 2024-03-21T13:29:23.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:23 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi097-80011-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:29:23.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:23 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi097-80028-6", "pg_num": 4}]': finished 2024-03-21T13:29:23.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:23 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1371127508' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi097-80014-7"}]: dispatch 2024-03-21T13:29:23.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:23 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3480244863' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi097-80011-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi097-80011-12"}]: dispatch 2024-03-21T13:29:23.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:23 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/795380094' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4", "tierpool": "test-rados-api-smithi097-80028-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:29:23.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:23 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3695006278' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi097-79740-10"}]: dispatch 2024-03-21T13:29:23.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:23 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2661409599' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi097-79828-10"}]: dispatch 2024-03-21T13:29:23.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:23 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2488892529' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-4"}]: dispatch 2024-03-21T13:29:23.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:23 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:23.373 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:23 smithi144 ceph-mon[33633]: osdmap e87: 8 total, 8 up, 8 in 2024-03-21T13:29:23.373 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:23 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3484349219' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:23.373 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:23 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4", "tierpool": "test-rados-api-smithi097-80028-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:29:23.373 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:23 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi097-80014-7"}]: dispatch 2024-03-21T13:29:23.373 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:23 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi097-80011-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi097-80011-12"}]: dispatch 2024-03-21T13:29:23.373 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:23 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi097-79740-10"}]: dispatch 2024-03-21T13:29:23.373 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:23 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi097-79828-10"}]: dispatch 2024-03-21T13:29:23.373 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:23 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:23.373 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:23 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3714576222' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi097-80039-7"}]: dispatch 2024-03-21T13:29:23.373 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:23 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-4"}]: dispatch 2024-03-21T13:29:23.871 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:29:23 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:29:24.035 INFO:tasks.workunit.client.0.smithi097.stdout: ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi097-80011-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi097-80011-12"}]: dispatch 2024-03-21T13:29:24.035 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:22.265438+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi097-79740-10"}]: dispatch 2024-03-21T13:29:24.035 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:22.265827+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi097-79828-10"}]: dispatch 2024-03-21T13:29:24.035 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:22.266511+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:24.035 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:22.267087+0000 mon.a [INF] from='client.? 172.21.15.97:0/3714576222' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi097-80039-7"}]: dispatch 2024-03-21T13:29:24.035 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:22.277805+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-4"}]: dispatch 2024-03-21T13:29:24.035 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:22.419251+0000 client.admin [INF] threexx 2024-03-21T13:29:24.036 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:23.006306+0000 mon.a [INF] from='client.? 172.21.15.97:0/1913222463' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-03-21T13:29:24.036 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:23.013175+0000 mon.a [INF] from='client.? 172.21.15.97:0/1626342189' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi097-80861-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi097-80861-2"}]': finished 2024-03-21T13:29:24.036 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:23.013234+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4", "tierpool": "test-rados-api-smithi097-80028-6", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:29:24.036 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:23.013300+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi097-80014-7"}]': finished 2024-03-21T13:29:24.036 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:23.013363+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi097-79740-10"}]': finished 2024-03-21T13:29:24.036 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:23.013423+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi097-79828-10"}]': finished 2024-03-21T13:29:24.036 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:23.013473+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:24.036 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:23.013519+0000 mon.a [INF] from='client.? 172.21.15.97:0/3714576222' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi097-80039-7"}]': finished 2024-03-21T13:29:24.036 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:23.013555+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-4"}]': finished 2024-03-21T13:29:24.051 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:23.018675+0000 mon.b [INF] from='client.? 172.21.15.97:0/795380094' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4", "overlaypool": "test-rados-api api_lock_pp: Running main() from gmock_main.cc 2024-03-21T13:29:24.051 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock_pp: [==========] Running 16 tests from 2 test suites. 2024-03-21T13:29:24.051 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock_pp: [----------] Global test environment set-up. 2024-03-21T13:29:24.051 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP 2024-03-21T13:29:24.051 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock_pp: seed 79740 2024-03-21T13:29:24.051 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusivePP 2024-03-21T13:29:24.051 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusivePP (8 ms) 2024-03-21T13:29:24.052 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedPP 2024-03-21T13:29:24.052 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedPP (3 ms) 2024-03-21T13:29:24.052 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusiveDurPP 2024-03-21T13:29:24.052 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusiveDurPP (1003 ms) 2024-03-21T13:29:24.052 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedDurPP 2024-03-21T13:29:24.052 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedDurPP (1004 ms) 2024-03-21T13:29:24.052 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockMayRenewPP 2024-03-21T13:29:24.052 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockMayRenewPP (3 ms) 2024-03-21T13:29:24.052 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.UnlockPP 2024-03-21T13:29:24.052 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock_pp: [ OK ] LibRadosLockPP.UnlockPP (4 ms) 2024-03-21T13:29:24.052 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.ListLockersPP 2024-03-21T13:29:24.052 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock_pp: [ OK ] LibRadosLockPP.ListLockersPP (3 ms) 2024-03-21T13:29:24.052 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.BreakLockPP 2024-03-21T13:29:24.052 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock_pp: [ OK ] LibRadosLockPP.BreakLockPP (3 ms) 2024-03-21T13:29:24.052 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP (2031 ms total) 2024-03-21T13:29:24.053 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock_pp: 2024-03-21T13:29:24.053 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP 2024-03-21T13:29:24.053 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusivePP 2024-03-21T13:29:24.053 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusivePP (1126 ms) 2024-03-21T13:29:24.053 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedPP 2024-03-21T13:29:24.053 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedPP (2 ms) 2024-03-21T13:29:24.053 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusiveDurPP 2024-03-21T13:29:24.053 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusiveDurPP (1978 ms) 2024-03-21T13:29:24.053 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedDurPP 2024-03-21T13:29:24.053 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedDurPP (1004 ms) 2024-03-21T13:29:24.053 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockMayRenewPP 2024-03-21T13:29:24.053 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockMayRenewPP (4 ms) 2024-03-21T13:29:24.053 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.UnlockPP 2024-03-21T13:29:24.053 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.UnlockPP (3 ms) 2024-03-21T13:29:24.053 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.ListLockersPP 2024-03-21T13:29:24.054 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.ListLockersPP (4 ms) 2024-03-21T13:29:24.054 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.BreakLockPP 2024-03-21T13:29:24.054 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.BreakLockPP (3 ms) 2024-03-21T13:29:24.054 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP (4124 ms total) 2024-03-21T13:29:24.054 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock_pp: 2024-03-21T13:29:24.054 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock_pp: [----------] Global test environment tear-down 2024-03-21T13:29:24.054 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock_pp: [==========] 16 tests from 2 test suites ran. (24581 ms total) 2024-03-21T13:29:24.054 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock_pp: [ PASSED ] 16 tests. 2024-03-21T13:29:24.054 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock: Running main() from gmock_main.cc 2024-03-21T13:29:24.054 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock: [==========] Running 16 tests from 2 test suites. 2024-03-21T13:29:24.055 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock: [----------] Global test environment set-up. 2024-03-21T13:29:24.055 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock: [----------] 8 tests from LibRadosLock 2024-03-21T13:29:24.055 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusive 2024-03-21T13:29:24.055 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock: [ OK ] LibRadosLock.LockExclusive (290 ms) 2024-03-21T13:29:24.055 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock: [ RUN ] LibRadosLock.LockShared 2024-03-21T13:29:24.055 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock: [ OK ] LibRadosLock.LockShared (51 ms) 2024-03-21T13:29:24.055 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusiveDur 2024-03-21T13:29:24.055 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock: [ OK ] LibRadosLock.LockExclusiveDur (1016 ms) 2024-03-21T13:29:24.055 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock: [ RUN ] LibRadosLock.LockSharedDur 2024-03-21T13:29:24.055 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock: [ OK ] LibRadosLock.LockSharedDur (1037 ms) 2024-03-21T13:29:24.055 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock: [ RUN ] LibRadosLock.LockMayRenew 2024-03-21T13:29:24.055 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock: [ OK ] LibRadosLock.LockMayRenew (31 ms) 2024-03-21T13:29:24.055 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock: [ RUN ] LibRadosLock.Unlock 2024-03-21T13:29:24.055 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock: [ OK ] LibRadosLock.Unlock (73 ms) 2024-03-21T13:29:24.056 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock: [ RUN ] LibRadosLock.ListLockers 2024-03-21T13:29:24.056 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock: [ OK ] LibRadosLock.ListLockers (5 ms) 2024-03-21T13:29:24.056 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock: [ RUN ] LibRadosLock.BreakLock 2024-03-21T13:29:24.056 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock: [ OK ] LibRadosLock.BreakLock (3 ms) 2024-03-21T13:29:24.056 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock: [----------] 8 tests from LibRadosLock (2506 ms total) 2024-03-21T13:29:24.056 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock: 2024-03-21T13:29:24.056 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock: [----------] 8 tests from LibRadosLockEC 2024-03-21T13:29:24.056 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusive 2024-03-21T13:29:24.056 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusive (2141 ms) 2024-03-21T13:29:24.056 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock: [ RUN ] LibRadosLockEC.LockShared 2024-03-21T13:29:24.056 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock: [ OK ] LibRadosLockEC.LockShared (5 ms) 2024-03-21T13:29:24.056 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusiveDur 2024-03-21T13:29:24.056 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusiveDur (1005 ms) 2024-03-21T13:29:24.056 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock: [ RUN ] LibRadosLockEC.LockSharedDur 2024-03-21T13:29:24.056 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock: [ OK ] LibRadosLockEC.LockSharedDur (1004 ms) 2024-03-21T13:29:24.056 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock: [ RUN ] LibRadosLockEC.LockMayRenew 2024-03-21T13:29:24.057 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock: [ OK ] LibRadosLockEC.LockMayRenew (3 ms) 2024-03-21T13:29:24.057 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock: [ RUN ] LibRadosLockEC.Unlock 2024-03-21T13:29:24.057 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock: [ OK ] LibRadosLockEC.Unlock (4 ms) 2024-03-21T13:29:24.057 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock: [ RUN ] LibRadosLockEC.ListLockers 2024-03-21T13:29:24.057 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock: [ OK ] LibRadosLockEC.ListLockers (4 ms) 2024-03-21T13:29:24.057 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock: [ RUN ] LibRadosLockEC.BreakLock 2024-03-21T13:29:24.057 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock: [ OK ] LibRadosLockEC.BreakLock (3 ms) 2024-03-21T13:29:24.057 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock: [----------] 8 tests from LibRadosLockEC (4169 ms total) 2024-03-21T13:29:24.057 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock: 2024-03-21T13:29:24.057 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock: [----------] Global test environment tear-down 2024-03-21T13:29:24.057 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock: [==========] 16 tests from 2 test suites ran. (24558 ms total) 2024-03-21T13:29:24.057 INFO:tasks.workunit.client.0.smithi097.stdout: api_lock: [ PASSED ] 16 tests. 2024-03-21T13:29:24.060 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat_pp: Running main() from gmock_main.cc 2024-03-21T13:29:24.060 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat_pp: [==========] Running 9 tests from 2 test suites. 2024-03-21T13:29:24.060 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat_pp: [----------] Global test environment set-up. 2024-03-21T13:29:24.060 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP 2024-03-21T13:29:24.060 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat_pp: seed 80039 2024-03-21T13:29:24.060 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPP 2024-03-21T13:29:24.060 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPP (398 ms) 2024-03-21T13:29:24.060 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.Stat2Mtime2PP 2024-03-21T13:29:24.060 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat_pp: [ OK ] LibRadosStatPP.Stat2Mtime2PP (8 ms) 2024-03-21T13:29:24.060 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.ClusterStatPP 2024-03-21T13:29:24.060 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat_pp: [ OK ] LibRadosStatPP.ClusterStatPP (1 ms) 2024-03-21T13:29:24.060 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.PoolStatPP 2024-03-21T13:29:24.060 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat_pp: [ OK ] LibRadosStatPP.PoolStatPP (5 ms) 2024-03-21T13:29:24.061 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPPNS 2024-03-21T13:29:24.061 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPPNS (14 ms) 2024-03-21T13:29:24.061 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP (426 ms total) 2024-03-21T13:29:24.061 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat_pp: 2024-03-21T13:29:24.061 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP 2024-03-21T13:29:24.061 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPP 2024-03-21T13:29:24.061 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPP (888 ms) 2024-03-21T13:29:24.061 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.ClusterStatPP 2024-03-21T13:29:24.061 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.ClusterStatPP (202 ms) 2024-03-21T13:29:24.061 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.PoolStatPP 2024-03-21T13:29:24.061 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.PoolStatPP (236 ms) 2024-03-21T13:29:24.061 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPPNS 2024-03-21T13:29:24.061 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPPNS (8 ms) 2024-03-21T13:29:24.061 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP (1334 ms total) 2024-03-21T13:29:24.062 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat_pp: 2024-03-21T13:29:24.062 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat_pp: [----------] Global test environment tear-down 2024-03-21T13:29:24.062 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat_pp: [==========] 9 tests from 2 test suites ran. (24484 ms total) 2024-03-21T13:29:24.062 INFO:tasks.workunit.client.0.smithi097.stdout: api_stat_pp: [ PASSED ] 9 tests. 2024-03-21T13:29:24.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:24 smithi097 ceph-mon[39416]: pgmap v94: 720 pgs: 172 unknown, 32 creating+peering, 516 active+clean; 145 MiB data, 741 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:29:24.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:24 smithi097 ceph-mon[39416]: threexx 2024-03-21T13:29:24.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:24 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1913222463' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-03-21T13:29:24.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:24 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1626342189' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi097-80861-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi097-80861-2"}]': finished 2024-03-21T13:29:24.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:24 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4", "tierpool": "test-rados-api-smithi097-80028-6", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:29:24.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:24 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi097-80014-7"}]': finished 2024-03-21T13:29:24.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:24 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi097-79740-10"}]': finished 2024-03-21T13:29:24.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:24 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi097-79828-10"}]': finished 2024-03-21T13:29:24.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:24 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:24.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:24 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3714576222' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi097-80039-7"}]': finished 2024-03-21T13:29:24.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:24 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-4"}]': finished 2024-03-21T13:29:24.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:24 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/795380094' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4", "overlaypool": "test-rados-api-smithi097-80028-6"}]: dispatch 2024-03-21T13:29:24.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:24 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2488892529' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-4", "tierpool": "test-rados-api-smithi097-79906-4-cache"}]: dispatch 2024-03-21T13:29:24.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:24 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3695006278' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi097-79740-10"}]: dispatch 2024-03-21T13:29:24.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:24 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2661409599' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi097-79828-10"}]: dispatch 2024-03-21T13:29:24.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:24 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:24.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:24 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1668470557' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi097-80712-2"}]: dispatch 2024-03-21T13:29:24.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:24 smithi097 ceph-mon[39416]: osdmap e88: 8 total, 8 up, 8 in 2024-03-21T13:29:24.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:24 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3714576222' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi097-80039-7"}]: dispatch 2024-03-21T13:29:24.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:24 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4", "overlaypool": "test-rados-api-smithi097-80028-6"}]: dispatch 2024-03-21T13:29:24.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:24 smithi097 ceph-mon[30040]: pgmap v94: 720 pgs: 172 unknown, 32 creating+peering, 516 active+clean; 145 MiB data, 741 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:29:24.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:24 smithi097 ceph-mon[30040]: threexx 2024-03-21T13:29:24.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:24 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1913222463' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-03-21T13:29:24.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:24 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1626342189' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi097-80861-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi097-80861-2"}]': finished 2024-03-21T13:29:24.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:24 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4", "tierpool": "test-rados-api-smithi097-80028-6", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:29:24.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:24 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi097-80014-7"}]': finished 2024-03-21T13:29:24.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:24 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi097-79740-10"}]': finished 2024-03-21T13:29:24.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:24 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi097-79828-10"}]': finished 2024-03-21T13:29:24.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:24 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:24.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:24 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3714576222' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi097-80039-7"}]': finished 2024-03-21T13:29:24.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:24 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-4"}]': finished 2024-03-21T13:29:24.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:24 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/795380094' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4", "overlaypool": "test-rados-api-smithi097-80028-6"}]: dispatch 2024-03-21T13:29:24.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:24 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2488892529' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-4", "tierpool": "test-rados-api-smithi097-79906-4-cache"}]: dispatch 2024-03-21T13:29:24.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:24 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3695006278' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi097-79740-10"}]: dispatch 2024-03-21T13:29:24.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:24 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2661409599' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi097-79828-10"}]: dispatch 2024-03-21T13:29:24.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:24 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:24.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:24 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1668470557' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi097-80712-2"}]: dispatch 2024-03-21T13:29:24.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:24 smithi097 ceph-mon[30040]: osdmap e88: 8 total, 8 up, 8 in 2024-03-21T13:29:24.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:24 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3714576222' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi097-80039-7"}]: dispatch 2024-03-21T13:29:24.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:24 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4", "overlaypool": "test-rados-api-smithi097-80028-6"}]: dispatch 2024-03-21T13:29:24.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:24 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi097-79740-10"}]: dispatch 2024-03-21T13:29:24.307 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:24 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-4", "tierpool": "test-rados-api-smithi097-79906-4-cache"}]: dispatch 2024-03-21T13:29:24.307 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:24 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi097-79828-10"}]: dispatch 2024-03-21T13:29:24.307 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:24 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:29:24.307 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:24 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi097-79740-10"}]: dispatch 2024-03-21T13:29:24.307 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:24 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-4", "tierpool": "test-rados-api-smithi097-79906-4-cache"}]: dispatch 2024-03-21T13:29:24.307 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:24 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi097-79828-10"}]: dispatch 2024-03-21T13:29:24.307 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:24 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:29:24.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:24 smithi144 ceph-mon[33633]: pgmap v94: 720 pgs: 172 unknown, 32 creating+peering, 516 active+clean; 145 MiB data, 741 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:29:24.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:24 smithi144 ceph-mon[33633]: threexx 2024-03-21T13:29:24.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:24 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1913222463' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-03-21T13:29:24.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:24 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1626342189' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi097-80861-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi097-80861-2"}]': finished 2024-03-21T13:29:24.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:24 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4", "tierpool": "test-rados-api-smithi097-80028-6", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:29:24.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:24 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi097-80014-7"}]': finished 2024-03-21T13:29:24.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:24 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi097-79740-10"}]': finished 2024-03-21T13:29:24.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:24 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi097-79828-10"}]': finished 2024-03-21T13:29:24.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:24 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:24.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:24 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3714576222' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi097-80039-7"}]': finished 2024-03-21T13:29:24.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:24 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-4"}]': finished 2024-03-21T13:29:24.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:24 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/795380094' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4", "overlaypool": "test-rados-api-smithi097-80028-6"}]: dispatch 2024-03-21T13:29:24.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:24 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2488892529' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-4", "tierpool": "test-rados-api-smithi097-79906-4-cache"}]: dispatch 2024-03-21T13:29:24.373 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:24 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3695006278' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi097-79740-10"}]: dispatch 2024-03-21T13:29:24.373 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:24 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2661409599' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi097-79828-10"}]: dispatch 2024-03-21T13:29:24.373 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:24 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:24.373 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:24 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1668470557' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi097-80712-2"}]: dispatch 2024-03-21T13:29:24.373 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:24 smithi144 ceph-mon[33633]: osdmap e88: 8 total, 8 up, 8 in 2024-03-21T13:29:24.373 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:24 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3714576222' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi097-80039-7"}]: dispatch 2024-03-21T13:29:24.373 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:24 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4", "overlaypool": "test-rados-api-smithi097-80028-6"}]: dispatch 2024-03-21T13:29:24.373 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:24 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi097-79740-10"}]: dispatch 2024-03-21T13:29:24.373 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:24 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-4", "tierpool": "test-rados-api-smithi097-79906-4-cache"}]: dispatch 2024-03-21T13:29:24.373 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:24 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi097-79828-10"}]: dispatch 2024-03-21T13:29:24.373 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:24 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:29:25.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:25 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:29:25.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:25 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:29:25.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:25 smithi097 ceph-mon[39416]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:29:25.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:25 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi097-80011-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi097-80011-12"}]': finished 2024-03-21T13:29:25.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:25 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3714576222' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi097-80039-7"}]': finished 2024-03-21T13:29:25.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:25 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4", "overlaypool": "test-rados-api-smithi097-80028-6"}]': finished 2024-03-21T13:29:25.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:25 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi097-79740-10"}]': finished 2024-03-21T13:29:25.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:25 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-4", "tierpool": "test-rados-api-smithi097-79906-4-cache"}]': finished 2024-03-21T13:29:25.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:25 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi097-79828-10"}]': finished 2024-03-21T13:29:25.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:25 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1913222463' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-03-21T13:29:25.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:25 smithi097 ceph-mon[39416]: osdmap e89: 8 total, 8 up, 8 in 2024-03-21T13:29:25.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:25 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/795380094' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-80028-6", "mode": "writeback"}]: dispatch 2024-03-21T13:29:25.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:25 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2955392173' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:25.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:25 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1440999728' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi097-79618-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:25.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:25 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:25.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:25 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi097-79618-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:25.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:25 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi097-80712-2"}]: dispatch 2024-03-21T13:29:25.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:25 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-80028-6", "mode": "writeback"}]: dispatch 2024-03-21T13:29:25.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:25 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:25.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:25 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:29:25.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:25 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:29:25.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:25 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:29:25.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:25 smithi097 ceph-mon[30040]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:29:25.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:25 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi097-80011-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi097-80011-12"}]': finished 2024-03-21T13:29:25.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:25 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3714576222' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi097-80039-7"}]': finished 2024-03-21T13:29:25.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:25 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4", "overlaypool": "test-rados-api-smithi097-80028-6"}]': finished 2024-03-21T13:29:25.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:25 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi097-79740-10"}]': finished 2024-03-21T13:29:25.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:25 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-4", "tierpool": "test-rados-api-smithi097-79906-4-cache"}]': finished 2024-03-21T13:29:25.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:25 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi097-79828-10"}]': finished 2024-03-21T13:29:25.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:25 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1913222463' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-03-21T13:29:25.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:25 smithi097 ceph-mon[30040]: osdmap e89: 8 total, 8 up, 8 in 2024-03-21T13:29:25.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:25 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/795380094' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-80028-6", "mode": "writeback"}]: dispatch 2024-03-21T13:29:25.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:25 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2955392173' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:25.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:25 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1440999728' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi097-79618-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:25.307 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:25 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:25.307 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:25 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi097-79618-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:25.307 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:25 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi097-80712-2"}]: dispatch 2024-03-21T13:29:25.307 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:25 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-80028-6", "mode": "writeback"}]: dispatch 2024-03-21T13:29:25.307 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:25 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:25.307 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:25 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:29:25.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:25 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:29:25.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:25 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:29:25.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:25 smithi144 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:29:25.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:25 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi097-80011-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi097-80011-12"}]': finished 2024-03-21T13:29:25.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:25 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3714576222' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi097-80039-7"}]': finished 2024-03-21T13:29:25.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:25 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4", "overlaypool": "test-rados-api-smithi097-80028-6"}]': finished 2024-03-21T13:29:25.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:25 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi097-79740-10"}]': finished 2024-03-21T13:29:25.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:25 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-4", "tierpool": "test-rados-api-smithi097-79906-4-cache"}]': finished 2024-03-21T13:29:25.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:25 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi097-79828-10"}]': finished 2024-03-21T13:29:25.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:25 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1913222463' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-03-21T13:29:25.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:25 smithi144 ceph-mon[33633]: osdmap e89: 8 total, 8 up, 8 in 2024-03-21T13:29:25.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:25 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/795380094' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-80028-6", "mode": "writeback"}]: dispatch 2024-03-21T13:29:25.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:25 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2955392173' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:25.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:25 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1440999728' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi097-79618-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:25.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:25 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:25.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:25 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi097-79618-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:25.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:25 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi097-80712-2"}]: dispatch 2024-03-21T13:29:25.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:25 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-80028-6", "mode": "writeback"}]: dispatch 2024-03-21T13:29:25.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:25 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:25.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:25 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:29:26.065 INFO:tasks.workunit.client.0.smithi097.stdout: ec_io: Running main() from gmock_main.cc 2024-03-21T13:29:26.065 INFO:tasks.workunit.client.0.smithi097.stdout: ec_io: [==========] Running 2 tests from 1 test suite. 2024-03-21T13:29:26.065 INFO:tasks.workunit.client.0.smithi097.stdout: ec_io: [----------] Global test environment set-up. 2024-03-21T13:29:26.065 INFO:tasks.workunit.client.0.smithi097.stdout: ec_io: [----------] 2 tests from NeoRadosECIo 2024-03-21T13:29:26.065 INFO:tasks.workunit.client.0.smithi097.stdout: ec_io: [ RUN ] NeoRadosECIo.SimpleWrite 2024-03-21T13:29:26.065 INFO:tasks.workunit.client.0.smithi097.stdout: ec_io: [ OK ] NeoRadosECIo.SimpleWrite (18027 ms) 2024-03-21T13:29:26.065 INFO:tasks.workunit.client.0.smithi097.stdout: ec_io: [ RUN ] NeoRadosECIo.ReadOp 2024-03-21T13:29:26.065 INFO:tasks.workunit.client.0.smithi097.stdout: ec_io: [ OK ] NeoRadosECIo.ReadOp (8226 ms) 2024-03-21T13:29:26.065 INFO:tasks.workunit.client.0.smithi097.stdout: ec_io: [----------] 2 tests from NeoRadosECIo (26253 ms total) 2024-03-21T13:29:26.065 INFO:tasks.workunit.client.0.smithi097.stdout: ec_io: 2024-03-21T13:29:26.065 INFO:tasks.workunit.client.0.smithi097.stdout: ec_io: [----------] Global test environment tear-down 2024-03-21T13:29:26.065 INFO:tasks.workunit.client.0.smithi097.stdout: ec_io: [==========] 2 tests from 1 test suite ran. (26254 ms total) 2024-03-21T13:29:26.065 INFO:tasks.workunit.client.0.smithi097.stdout: ec_io: [ PASSED ] 2 tests. 2024-03-21T13:29:26.086 INFO:tasks.workunit.client.0.smithi097.stdout: pool: Running main() from gmock_main.cc 2024-03-21T13:29:26.086 INFO:tasks.workunit.client.0.smithi097.stdout: pool: [==========] Running 6 tests from 1 test suite. 2024-03-21T13:29:26.086 INFO:tasks.workunit.client.0.smithi097.stdout: pool: [----------] Global test environment set-up. 2024-03-21T13:29:26.086 INFO:tasks.workunit.client.0.smithi097.stdout: pool: [----------] 6 tests from NeoRadosPools 2024-03-21T13:29:26.086 INFO:tasks.workunit.client.0.smithi097.stdout: pool: [ RUN ] NeoRadosPools.PoolList 2024-03-21T13:29:26.086 INFO:tasks.workunit.client.0.smithi097.stdout: pool: [ OK ] NeoRadosPools.PoolList (9372 ms) 2024-03-21T13:29:26.087 INFO:tasks.workunit.client.0.smithi097.stdout: pool: [ RUN ] NeoRadosPools.PoolLookup 2024-03-21T13:29:26.087 INFO:tasks.workunit.client.0.smithi097.stdout: pool: [ OK ] NeoRadosPools.PoolLookup (5492 ms) 2024-03-21T13:29:26.087 INFO:tasks.workunit.client.0.smithi097.stdout: pool: [ RUN ] NeoRadosPools.PoolLookupOtherInstance 2024-03-21T13:29:26.087 INFO:tasks.workunit.client.0.smithi097.stdout: pool: [ OK ] NeoRadosPools.PoolLookupOtherInstance (3399 ms) 2024-03-21T13:29:26.087 INFO:tasks.workunit.client.0.smithi097.stdout: pool: [ RUN ] NeoRadosPools.PoolDelete 2024-03-21T13:29:26.087 INFO:tasks.workunit.client.0.smithi097.stdout: pool: [ OK ] NeoRadosPools.PoolDelete (4147 ms) 2024-03-21T13:29:26.087 INFO:tasks.workunit.client.0.smithi097.stdout: pool: [ RUN ] NeoRadosPools.PoolCreateDelete 2024-03-21T13:29:26.087 INFO:tasks.workunit.client.0.smithi097.stdout: pool: [ OK ] NeoRadosPools.PoolCreateDelete (1792 ms) 2024-03-21T13:29:26.087 INFO:tasks.workunit.client.0.smithi097.stdout: pool: [ RUN ] NeoRadosPools.PoolCreateWithCrushRule 2024-03-21T13:29:26.087 INFO:tasks.workunit.client.0.smithi097.stdout: pool: [ OK ] NeoRadosPools.PoolCreateWithCrushRule (2030 ms) 2024-03-21T13:29:26.087 INFO:tasks.workunit.client.0.smithi097.stdout: pool: [----------] 6 tests from NeoRadosPools (26232 ms total) 2024-03-21T13:29:26.087 INFO:tasks.workunit.client.0.smithi097.stdout: pool: 2024-03-21T13:29:26.087 INFO:tasks.workunit.client.0.smithi097.stdout: pool: [----------] Global test environment tear-down 2024-03-21T13:29:26.087 INFO:tasks.workunit.client.0.smithi097.stdout: pool: [==========] 6 tests from 1 test suite ran. (26232 ms total) 2024-03-21T13:29:26.088 INFO:tasks.workunit.client.0.smithi097.stdout: pool: [ PASSED ] 6 tests. 2024-03-21T13:29:26.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:26 smithi097 ceph-mon[39416]: pgmap v97: 824 pgs: 41 creating+peering, 135 unknown, 648 active+clean; 145 MiB data, 751 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 4.7 KiB/s wr, 2 op/s 2024-03-21T13:29:26.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:26 smithi097 ceph-mon[39416]: Health check update: 12 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:29:26.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:26 smithi097 ceph-mon[39416]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:29:26.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:26 smithi097 ceph-mon[39416]: fourxx 2024-03-21T13:29:26.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:26 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1913222463' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-03-21T13:29:26.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:26 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:26.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:26 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi097-79618-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:26.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:26 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi097-80712-2"}]': finished 2024-03-21T13:29:26.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:26 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-80028-6", "mode": "writeback"}]': finished 2024-03-21T13:29:26.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:26 smithi097 ceph-mon[39416]: osdmap e90: 8 total, 8 up, 8 in 2024-03-21T13:29:26.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:26 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1668470557' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi097-80712-2"}]: dispatch 2024-03-21T13:29:26.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:26 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi097-80712-2"}]: dispatch 2024-03-21T13:29:26.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:26 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:26.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:26 smithi097 ceph-mon[30040]: pgmap v97: 824 pgs: 41 creating+peering, 135 unknown, 648 active+clean; 145 MiB data, 751 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 4.7 KiB/s wr, 2 op/s 2024-03-21T13:29:26.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:26 smithi097 ceph-mon[30040]: Health check update: 12 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:29:26.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:26 smithi097 ceph-mon[30040]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:29:26.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:26 smithi097 ceph-mon[30040]: fourxx 2024-03-21T13:29:26.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:26 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1913222463' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-03-21T13:29:26.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:26 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:26.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:26 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi097-79618-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:26.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:26 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi097-80712-2"}]': finished 2024-03-21T13:29:26.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:26 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-80028-6", "mode": "writeback"}]': finished 2024-03-21T13:29:26.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:26 smithi097 ceph-mon[30040]: osdmap e90: 8 total, 8 up, 8 in 2024-03-21T13:29:26.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:26 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1668470557' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi097-80712-2"}]: dispatch 2024-03-21T13:29:26.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:26 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi097-80712-2"}]: dispatch 2024-03-21T13:29:26.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:26 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:26.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:26 smithi144 ceph-mon[33633]: pgmap v97: 824 pgs: 41 creating+peering, 135 unknown, 648 active+clean; 145 MiB data, 751 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 4.7 KiB/s wr, 2 op/s 2024-03-21T13:29:26.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:26 smithi144 ceph-mon[33633]: Health check update: 12 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:29:26.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:26 smithi144 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:29:26.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:26 smithi144 ceph-mon[33633]: fourxx 2024-03-21T13:29:26.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:26 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1913222463' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-03-21T13:29:26.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:26 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:26.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:26 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi097-79618-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:26.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:26 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi097-80712-2"}]': finished 2024-03-21T13:29:26.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:26 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-80028-6", "mode": "writeback"}]': finished 2024-03-21T13:29:26.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:26 smithi144 ceph-mon[33633]: osdmap e90: 8 total, 8 up, 8 in 2024-03-21T13:29:26.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:26 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1668470557' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi097-80712-2"}]: dispatch 2024-03-21T13:29:26.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:26 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi097-80712-2"}]: dispatch 2024-03-21T13:29:26.373 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:26 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:27.373 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:27 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1913222463' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-03-21T13:29:27.373 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:27 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi097-80712-2"}]': finished 2024-03-21T13:29:27.373 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:27 smithi144 ceph-mon[33633]: osdmap e91: 8 total, 8 up, 8 in 2024-03-21T13:29:27.374 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:27 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3480244863' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi097-80011-12"}]: dispatch 2024-03-21T13:29:27.374 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:27 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3707358668' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi097-80000-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:27.374 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:27 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi097-80011-12"}]: dispatch 2024-03-21T13:29:27.374 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:27 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1626342189' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi097-80861-2"}]: dispatch 2024-03-21T13:29:27.374 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:27 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/157919924' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:27.374 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:27 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi097-80000-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:27.374 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:27 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:27.442 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:27 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1913222463' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-03-21T13:29:27.442 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:27 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi097-80712-2"}]': finished 2024-03-21T13:29:27.442 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:27 smithi097 ceph-mon[39416]: osdmap e91: 8 total, 8 up, 8 in 2024-03-21T13:29:27.443 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:27 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3480244863' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi097-80011-12"}]: dispatch 2024-03-21T13:29:27.443 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:27 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3707358668' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi097-80000-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:27.443 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:27 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi097-80011-12"}]: dispatch 2024-03-21T13:29:27.443 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:27 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1626342189' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi097-80861-2"}]: dispatch 2024-03-21T13:29:27.443 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:27 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/157919924' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:27.443 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:27 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi097-80000-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:27.443 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:27 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:27.443 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:27 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1913222463' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-03-21T13:29:27.443 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:27 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi097-80712-2"}]': finished 2024-03-21T13:29:27.443 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:27 smithi097 ceph-mon[30040]: osdmap e91: 8 total, 8 up, 8 in 2024-03-21T13:29:27.443 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:27 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3480244863' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi097-80011-12"}]: dispatch 2024-03-21T13:29:27.443 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:27 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3707358668' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi097-80000-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:27.443 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:27 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi097-80011-12"}]: dispatch 2024-03-21T13:29:27.444 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:27 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1626342189' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi097-80861-2"}]: dispatch 2024-03-21T13:29:27.444 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:27 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/157919924' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:27.444 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:27 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi097-80000-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:27.444 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:27 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:28.041 INFO:tasks.workunit.client.0.smithi097.stdout:-smithi097-80028-6"}]: dispatch 2024-03-21T13:29:28.041 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:23.018816+0000 mon.b [INF] from='client.? 172.21.15.97:0/2488892529' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-4", "tierpool": "test-rados-api-smithi097-79906-4-cache"}]: dispatch 2024-03-21T13:29:28.041 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:23.024240+0000 mon.c [INF] from='client.? 172.21.15.97:0/3695006278' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi097-79740-10"}]: dispatch 2024-03-21T13:29:28.041 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:23.024503+0000 mon.c [INF] from='client.? 172.21.15.97:0/2661409599' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi097-79828-10"}]: dispatch 2024-03-21T13:29:28.041 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:23.283273+0000 mon.b [INF] from='client.? 172.21.15.97:0/1668470557' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi097-80712-2"}]: dispatch 2024-03-21T13:29:28.041 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:23.533090+0000 mon.a [INF] from='client.? 172.21.15.97:0/3714576222' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi097-80039-7"}]: dispatch 2024-03-21T13:29:28.041 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:23.533608+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4", "overlaypool": "test-rados-api-smithi097-80028-6"}]: dispatch 2024-03-21T13:29:28.041 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:23.533880+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi097-79740-10"}]: dispatch 2024-03-21T13:29:28.041 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:23.534548+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-4", "tierpool": "test-rados-api-smithi097-79906-4-cache"}]: dispatch 2024-03-21T13:29:28.041 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:23.534742+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi097-79828-10"}]: dispatch 2024-03-21T13:29:28.042 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: got: 2024-03-21T13:29:23.778430+0000 mon.a [INF] from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:29:28.042 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: [ OK ] LibRadosCmd.WatchLog (13336 ms) 2024-03-21T13:29:28.042 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: [----------] 4 tests from LibRadosCmd (28414 ms total) 2024-03-21T13:29:28.042 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: 2024-03-21T13:29:28.042 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: [----------] Global test environment tear-down 2024-03-21T13:29:28.042 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: [==========] 4 tests from 1 test suite ran. (28414 ms total) 2024-03-21T13:29:28.042 INFO:tasks.workunit.client.0.smithi097.stdout: api_cmd: [ PASSED ] 4 tests. 2024-03-21T13:29:28.167 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify: Running main() from gmock_main.cc 2024-03-21T13:29:28.167 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify: [==========] Running 11 tests from 2 test suites. 2024-03-21T13:29:28.167 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify: [----------] Global test environment set-up. 2024-03-21T13:29:28.167 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify 2024-03-21T13:29:28.167 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify 2024-03-21T13:29:28.167 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify: watch_notify_test_cb 2024-03-21T13:29:28.167 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify (411 ms) 2024-03-21T13:29:28.167 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch2Delete 2024-03-21T13:29:28.167 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94143299530672 err -107 2024-03-21T13:29:28.167 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-03-21T13:29:28.168 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch2Delete (12 ms) 2024-03-21T13:29:28.168 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete 2024-03-21T13:29:28.168 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-03-21T13:29:28.168 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94143299531792 err -107 2024-03-21T13:29:28.168 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete (1016 ms) 2024-03-21T13:29:28.168 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2 2024-03-21T13:29:28.168 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify: watch_notify2_test_cb from 24643 notify_id 326417514496 cookie 94143299535424 2024-03-21T13:29:28.168 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2 (5 ms) 2024-03-21T13:29:28.168 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchNotify2 2024-03-21T13:29:28.168 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify: watch_notify2_test_cb from 24643 notify_id 326417514497 cookie 94143299554496 2024-03-21T13:29:28.168 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchNotify2 (4 ms) 2024-03-21T13:29:28.168 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioNotify 2024-03-21T13:29:28.168 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify: watch_notify2_test_cb from 24643 notify_id 326417514497 cookie 94143299568128 2024-03-21T13:29:28.168 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioNotify (4 ms) 2024-03-21T13:29:28.168 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Multi 2024-03-21T13:29:28.168 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify: watch_notify2_test_cb from 24643 notify_id 326417514497 cookie 94143299574528 2024-03-21T13:29:28.169 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify: watch_notify2_test_cb from 24643 notify_id 326417514497 cookie 94143299586576 2024-03-21T13:29:28.169 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Multi (7 ms) 2024-03-21T13:29:28.169 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Timeout 2024-03-21T13:29:28.169 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify: watch_notify2_test_cb from 24643 notify_id 326417514498 cookie 94143299588864 2024-03-21T13:29:28.169 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify: watch_notify2_test_cb from 24643 notify_id 330712481795 cookie 94143299588864 2024-03-21T13:29:28.169 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Timeout (3004 ms) 2024-03-21T13:29:28.169 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch3Timeout 2024-03-21T13:29:28.169 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify: waiting up to 1024 for osd to time us out ... 2024-03-21T13:29:28.169 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94143299586576 err -107 2024-03-21T13:29:28.169 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify: watch_notify2_test_cb from 24643 notify_id 356482285573 cookie 94143299586576 2024-03-21T13:29:28.169 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch3Timeout (5011 ms) 2024-03-21T13:29:28.169 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete2 2024-03-21T13:29:28.169 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify: waiting up to 30 for disconnect notification ... 2024-03-21T13:29:28.169 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94143299586576 err -107 2024-03-21T13:29:28.169 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete2 (1004 ms) 2024-03-21T13:29:28.170 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify (10479 ms total) 2024-03-21T13:29:28.170 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify: 2024-03-21T13:29:28.170 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC 2024-03-21T13:29:28.170 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotifyEC.WatchNotify 2024-03-21T13:29:28.170 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify: watch_notify_test_cb 2024-03-21T13:29:28.170 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify: [ OK ] LibRadosWatchNotifyEC.WatchNotify (1133 ms) 2024-03-21T13:29:28.170 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC (1133 ms total) 2024-03-21T13:29:28.170 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify: 2024-03-21T13:29:28.170 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify: [----------] Global test environment tear-down 2024-03-21T13:29:28.170 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify: [==========] 11 tests from 2 test suites ran. (28593 ms total) 2024-03-21T13:29:28.173 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify: [ PASSED ] 11 tests. 2024-03-21T13:29:28.232 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: Running main() from gmock_main.cc 2024-03-21T13:29:28.232 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: [==========] Running 16 tests from 2 test suites. 2024-03-21T13:29:28.232 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: [----------] Global test environment set-up. 2024-03-21T13:29:28.232 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP 2024-03-21T13:29:28.232 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotify 2024-03-21T13:29:28.232 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: notify 2024-03-21T13:29:28.232 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotify (3027 ms) 2024-03-21T13:29:28.232 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotifyTimeout 2024-03-21T13:29:28.232 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotifyTimeout (8 ms) 2024-03-21T13:29:28.232 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP (3035 ms total) 2024-03-21T13:29:28.233 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: 2024-03-21T13:29:28.233 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP 2024-03-21T13:29:28.233 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 2024-03-21T13:29:28.233 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: notify 2024-03-21T13:29:28.233 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 (137 ms) 2024-03-21T13:29:28.233 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 2024-03-21T13:29:28.233 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: notify 2024-03-21T13:29:28.233 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 (3866 ms) 2024-03-21T13:29:28.233 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 2024-03-21T13:29:28.233 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 (5 ms) 2024-03-21T13:29:28.233 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 2024-03-21T13:29:28.233 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 (6 ms) 2024-03-21T13:29:28.233 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 2024-03-21T13:29:28.233 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: handle_notify cookie 94810145041136 notify_id 386547056641 notifier_gid 25153 2024-03-21T13:29:28.233 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 (5 ms) 2024-03-21T13:29:28.234 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 2024-03-21T13:29:28.234 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: handle_notify cookie 94810143949184 notify_id 386547056642 notifier_gid 25153 2024-03-21T13:29:28.234 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 (7 ms) 2024-03-21T13:29:28.234 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 2024-03-21T13:29:28.234 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: handle_notify cookie 94810143965792 notify_id 386547056643 notifier_gid 25153 2024-03-21T13:29:28.234 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 (6 ms) 2024-03-21T13:29:28.234 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 2024-03-21T13:29:28.234 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: handle_notify cookie 94810143965792 notify_id 386547056644 notifier_gid 25153 2024-03-21T13:29:28.234 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 (5 ms) 2024-03-21T13:29:28.234 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 2024-03-21T13:29:28.234 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: handle_notify cookie 94810143965792 notify_id 386547056645 notifier_gid 25153 2024-03-21T13:29:28.234 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 (5 ms) 2024-03-21T13:29:28.234 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 2024-03-21T13:29:28.234 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: handle_notify cookie 94810143965712 notify_id 386547056646 notifier_gid 25153 2024-03-21T13:29:28.234 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 (6 ms) 2024-03-21T13:29:28.235 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 2024-03-21T13:29:28.235 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: trying... 2024-03-21T13:29:28.235 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: handle_notify cookie 94810143965712 notify_id 386547056642 notifier_gid 25153 2024-03-21T13:29:28.235 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: timed out 2024-03-21T13:29:28.235 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: flushing 2024-03-21T13:29:28.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:28 smithi097 ceph-mon[30040]: pgmap v100: 552 pgs: 96 unknown, 456 active+clean; 145 MiB data, 751 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-03-21T13:29:28.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:28 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi097-80011-12"}]': finished 2024-03-21T13:29:28.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:28 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1626342189' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi097-80861-2"}]': finished 2024-03-21T13:29:28.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:28 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/157919924' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:28.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:28 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi097-80000-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:28.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:28 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3480244863' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi097-80011-12"}]: dispatch 2024-03-21T13:29:28.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:28 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3712598369' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:28.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:28 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:28.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:28 smithi097 ceph-mon[30040]: osdmap e92: 8 total, 8 up, 8 in 2024-03-21T13:29:28.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:28 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1626342189' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi097-80861-2"}]: dispatch 2024-03-21T13:29:28.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:28 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi097-80011-12"}]: dispatch 2024-03-21T13:29:28.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:28 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:28.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:28 smithi097 ceph-mon[39416]: pgmap v100: 552 pgs: 96 unknown, 456 active+clean; 145 MiB data, 751 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-03-21T13:29:28.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:28 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi097-80011-12"}]': finished 2024-03-21T13:29:28.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:28 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1626342189' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi097-80861-2"}]': finished 2024-03-21T13:29:28.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:28 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/157919924' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:28.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:28 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi097-80000-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:28.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:28 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3480244863' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi097-80011-12"}]: dispatch 2024-03-21T13:29:28.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:28 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3712598369' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:28.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:28 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:28.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:28 smithi097 ceph-mon[39416]: osdmap e92: 8 total, 8 up, 8 in 2024-03-21T13:29:28.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:28 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1626342189' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi097-80861-2"}]: dispatch 2024-03-21T13:29:28.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:28 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi097-80011-12"}]: dispatch 2024-03-21T13:29:28.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:28 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:28.586 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:28 smithi144 ceph-mon[33633]: pgmap v100: 552 pgs: 96 unknown, 456 active+clean; 145 MiB data, 751 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-03-21T13:29:28.586 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:28 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi097-80011-12"}]': finished 2024-03-21T13:29:28.586 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:28 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1626342189' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi097-80861-2"}]': finished 2024-03-21T13:29:28.586 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:28 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/157919924' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:28.586 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:28 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi097-80000-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:28.586 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:28 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3480244863' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi097-80011-12"}]: dispatch 2024-03-21T13:29:28.586 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:28 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3712598369' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:28.587 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:28 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:28.587 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:28 smithi144 ceph-mon[33633]: osdmap e92: 8 total, 8 up, 8 in 2024-03-21T13:29:28.587 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:28 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1626342189' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi097-80861-2"}]: dispatch 2024-03-21T13:29:28.587 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:28 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi097-80011-12"}]: dispatch 2024-03-21T13:29:28.587 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:28 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:29.499 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:29 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:29:29.499 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:29 smithi144 ceph-mon[33633]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-21T13:29:29.499 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:29 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1626342189' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi097-80861-2"}]': finished 2024-03-21T13:29:29.500 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:29 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi097-80011-12"}]': finished 2024-03-21T13:29:29.500 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:29 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:29.500 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:29 smithi144 ceph-mon[33633]: osdmap e93: 8 total, 8 up, 8 in 2024-03-21T13:29:29.500 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:29 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:29.500 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:29 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/196111487' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi097-79992-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:29.500 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:29 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1082248310' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi097-79618-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:29.500 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:29 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:29.500 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:29 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi097-79992-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:29.500 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:29 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:29.500 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:29 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi097-79618-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:29.500 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:29 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:29:29.500 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:29 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/440005378' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi097-80861-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:29.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:29 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:29:29.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:29 smithi097 ceph-mon[39416]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-21T13:29:29.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:29 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1626342189' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi097-80861-2"}]': finished 2024-03-21T13:29:29.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:29 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi097-80011-12"}]': finished 2024-03-21T13:29:29.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:29 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:29.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:29 smithi097 ceph-mon[39416]: osdmap e93: 8 total, 8 up, 8 in 2024-03-21T13:29:29.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:29 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:29.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:29 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/196111487' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi097-79992-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:29.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:29 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1082248310' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi097-79618-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:29.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:29 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:29.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:29 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi097-79992-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:29.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:29 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:29.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:29 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi097-79618-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:29.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:29 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:29:29.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:29 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/440005378' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi097-80861-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:29.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:29 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:29:29.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:29 smithi097 ceph-mon[30040]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-21T13:29:29.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:29 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1626342189' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi097-80861-2"}]': finished 2024-03-21T13:29:29.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:29 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi097-80011-12"}]': finished 2024-03-21T13:29:29.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:29 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:29.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:29 smithi097 ceph-mon[30040]: osdmap e93: 8 total, 8 up, 8 in 2024-03-21T13:29:29.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:29 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:29.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:29 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/196111487' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi097-79992-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:29.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:29 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1082248310' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi097-79618-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:29.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:29 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:29.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:29 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi097-79992-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:29.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:29 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:29.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:29 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi097-79618-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:29.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:29 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:29:29.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:29 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/440005378' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi097-80861-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:30.316 INFO:tasks.workunit.client.0.smithi097.stdout: api_watc api_service: [==========] Running 4 tests from 1 test suite. 2024-03-21T13:29:30.316 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: [----------] Global test environment set-up. 2024-03-21T13:29:30.317 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: [----------] 4 tests from LibRadosService 2024-03-21T13:29:30.317 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: [ RUN ] LibRadosService.RegisterEarly 2024-03-21T13:29:30.317 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: [ OK ] LibRadosService.RegisterEarly (5078 ms) 2024-03-21T13:29:30.317 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: [ RUN ] LibRadosService.RegisterLate 2024-03-21T13:29:30.317 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: [ OK ] LibRadosService.RegisterLate (28 ms) 2024-03-21T13:29:30.317 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: [ RUN ] LibRadosService.StatusFormat 2024-03-21T13:29:30.317 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: cluster: 2024-03-21T13:29:30.317 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: id: a5dd1f32-e785-11ee-95cd-87774f69a715 2024-03-21T13:29:30.317 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: health: HEALTH_WARN 2024-03-21T13:29:30.317 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: 4 pool(s) do not have an application enabled 2024-03-21T13:29:30.317 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: 2024-03-21T13:29:30.317 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: services: 2024-03-21T13:29:30.317 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 6m) 2024-03-21T13:29:30.317 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: mgr: y(active, since 2m), standbys: x 2024-03-21T13:29:30.317 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: osd: 8 osds: 8 up (since 3m), 8 in (since 3m) 2024-03-21T13:29:30.318 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: laundry: 2 daemons active (1 hosts) 2024-03-21T13:29:30.318 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-03-21T13:29:30.318 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: 2024-03-21T13:29:30.318 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: data: 2024-03-21T13:29:30.318 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: pools: 10 pools, 260 pgs 2024-03-21T13:29:30.318 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: objects: 198 objects, 582 KiB 2024-03-21T13:29:30.318 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: usage: 243 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:29:30.318 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: pgs: 23.846% pgs unknown 2024-03-21T13:29:30.318 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: 25.385% pgs not active 2024-03-21T13:29:30.318 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: 132 active+clean 2024-03-21T13:29:30.318 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: 66 creating+peering 2024-03-21T13:29:30.318 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: 62 unknown 2024-03-21T13:29:30.318 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: 2024-03-21T13:29:30.318 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: io: 2024-03-21T13:29:30.319 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: client: 1023 B/s rd, 0 op/s rd, 0 op/s wr 2024-03-21T13:29:30.319 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: 2024-03-21T13:29:30.319 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: 2024-03-21T13:29:30.319 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: cluster: 2024-03-21T13:29:30.319 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: id: a5dd1f32-e785-11ee-95cd-87774f69a715 2024-03-21T13:29:30.319 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: health: HEALTH_WARN 2024-03-21T13:29:30.319 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: 32 pool(s) do not have an application enabled 2024-03-21T13:29:30.319 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: 2024-03-21T13:29:30.319 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: services: 2024-03-21T13:29:30.319 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 6m) 2024-03-21T13:29:30.319 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: mgr: y(active, since 2m), standbys: x 2024-03-21T13:29:30.319 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: osd: 8 osds: 8 up (since 3m), 8 in (since 3m) 2024-03-21T13:29:30.319 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: foo: 16 portals active (1 hosts, 3 zones) 2024-03-21T13:29:30.319 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: laundry: 1 daemon active (1 hosts) 2024-03-21T13:29:30.319 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-03-21T13:29:30.320 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: 2024-03-21T13:29:30.320 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: data: 2024-03-21T13:29:30.320 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: pools: 42 pools, 1284 pgs 2024-03-21T13:29:30.320 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: objects: 199 objects, 583 KiB 2024-03-21T13:29:30.320 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: usage: 243 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:29:30.320 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: pgs: 9.969% pgs unknown 2024-03-21T13:29:30.320 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: 70.639% pgs not active 2024-03-21T13:29:30.320 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: 896 creating+peering 2024-03-21T13:29:30.320 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: 249 active+clean 2024-03-21T13:29:30.320 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: 128 unknown 2024-03-21T13:29:30.320 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: 11 creating+activating 2024-03-21T13:29:30.320 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: 2024-03-21T13:29:30.320 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: io: 2024-03-21T13:29:30.320 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: client: 1.3 KiB/s rd, 131 B/s wr, 1 op/s rd, 0 op/s wr 2024-03-21T13:29:30.320 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: 2024-03-21T13:29:30.321 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: 2024-03-21T13:29:30.321 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: [ OK ] LibRadosService.StatusFormat (5525 ms) 2024-03-21T13:29:30.321 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: [ RUN ] LibRadosService.Status 2024-03-21T13:29:30.321 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: [ OK ] LibRadosService.Status (20040 ms) 2024-03-21T13:29:30.321 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: [----------] 4 tests from LibRadosService (30671 ms total) 2024-03-21T13:29:30.321 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: 2024-03-21T13:29:30.321 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: [----------] Global test environment tear-down 2024-03-21T13:29:30.321 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: [==========] 4 tests from 1 test suite ran. (30671 ms total) 2024-03-21T13:29:30.321 INFO:tasks.workunit.client.0.smithi097.stdout: api_service: [ PASSED ] 4 tests. 2024-03-21T13:29:30.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:30 smithi097 ceph-mon[39416]: pgmap v103: 648 pgs: 96 creating+peering, 552 active+clean; 145 MiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T13:29:30.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:30 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi097-79992-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:30.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:30 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:30.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:30 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi097-79618-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:30.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:30 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/440005378' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi097-80861-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:29:30.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:30 smithi097 ceph-mon[39416]: osdmap e94: 8 total, 8 up, 8 in 2024-03-21T13:29:30.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:30 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/440005378' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi097-80861-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi097-80861-3"}]: dispatch 2024-03-21T13:29:30.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:30 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2621926389' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:30.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:30 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:30.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:30 smithi097 ceph-mon[30040]: pgmap v103: 648 pgs: 96 creating+peering, 552 active+clean; 145 MiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T13:29:30.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:30 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi097-79992-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:30.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:30 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:30.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:30 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi097-79618-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:30.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:30 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/440005378' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi097-80861-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:29:30.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:30 smithi097 ceph-mon[30040]: osdmap e94: 8 total, 8 up, 8 in 2024-03-21T13:29:30.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:30 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/440005378' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi097-80861-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi097-80861-3"}]: dispatch 2024-03-21T13:29:30.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:30 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2621926389' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:30.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:30 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:30.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:30 smithi144 ceph-mon[33633]: pgmap v103: 648 pgs: 96 creating+peering, 552 active+clean; 145 MiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T13:29:30.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:30 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi097-79992-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:30.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:30 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:30.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:30 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi097-79618-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:30.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:30 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/440005378' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi097-80861-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:29:30.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:30 smithi144 ceph-mon[33633]: osdmap e94: 8 total, 8 up, 8 in 2024-03-21T13:29:30.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:30 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/440005378' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi097-80861-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi097-80861-3"}]: dispatch 2024-03-21T13:29:30.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:30 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2621926389' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:30.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:30 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:31.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:31 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2621926389' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:31.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:31 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:31.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:31 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:31.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:31 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3124960777' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi097-79636-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:31.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:31 smithi097 ceph-mon[39416]: osdmap e95: 8 total, 8 up, 8 in 2024-03-21T13:29:31.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:31 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:29:31.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:31 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:31.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:31 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi097-79636-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:31.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:31 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:29:31.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:31 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2621926389' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:31.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:31 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:31.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:31 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:31.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:31 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3124960777' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi097-79636-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:31.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:31 smithi097 ceph-mon[30040]: osdmap e95: 8 total, 8 up, 8 in 2024-03-21T13:29:31.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:31 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:29:31.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:31 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:31.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:31 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi097-79636-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:31.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:31 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:29:31.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:31 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2621926389' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:31.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:31 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:31.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:31 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:31.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:31 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3124960777' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi097-79636-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:31.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:31 smithi144 ceph-mon[33633]: osdmap e95: 8 total, 8 up, 8 in 2024-03-21T13:29:31.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:31 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:29:31.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:31 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:31.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:31 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi097-79636-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:31.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:31 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:29:32.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:29:31 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:29:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:29:32.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:32 smithi144 ceph-mon[33633]: pgmap v106: 712 pgs: 192 unknown, 96 creating+peering, 424 active+clean; 145 MiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-21T13:29:32.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:32 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/440005378' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi097-80861-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi097-80861-3"}]': finished 2024-03-21T13:29:32.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:32 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:32.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:32 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi097-79636-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:32.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:32 smithi144 ceph-mon[33633]: osdmap e96: 8 total, 8 up, 8 in 2024-03-21T13:29:32.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:32 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:32.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:32 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/821107064' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:32.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:32 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:29:32.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:32 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:29:32.873 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:32 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:32.873 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:32 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:29:32.873 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:32 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:29:32.873 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:32 smithi144 ceph-mon[33633]: pgmap v108: 624 pgs: 168 unknown, 64 creating+peering, 392 active+clean; 145 MiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.6 KiB/s wr, 7 op/s 2024-03-21T13:29:32.873 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:32 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:29:32.873 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:32 smithi144 ceph-mon[33633]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-03-21T13:29:32.874 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:32 smithi144 ceph-mon[33633]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:29:32.874 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:32 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:32.874 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:32 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:32.874 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:32 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-7", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:29:32.874 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:32 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-7"}]: dispatch 2024-03-21T13:29:32.874 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:32 smithi144 ceph-mon[33633]: osdmap e97: 8 total, 8 up, 8 in 2024-03-21T13:29:32.875 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:32 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-7"}]: dispatch 2024-03-21T13:29:32.875 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:32 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3929544138' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi097-79618-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:33.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:32 smithi097 ceph-mon[39416]: pgmap v106: 712 pgs: 192 unknown, 96 creating+peering, 424 active+clean; 145 MiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-21T13:29:33.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:32 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/440005378' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi097-80861-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi097-80861-3"}]': finished 2024-03-21T13:29:33.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:32 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:33.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:32 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi097-79636-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:33.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:32 smithi097 ceph-mon[39416]: osdmap e96: 8 total, 8 up, 8 in 2024-03-21T13:29:33.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:32 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:33.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:32 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/821107064' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:33.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:32 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:29:33.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:32 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:29:33.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:32 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:33.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:32 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:29:33.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:32 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:29:33.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:32 smithi097 ceph-mon[39416]: pgmap v108: 624 pgs: 168 unknown, 64 creating+peering, 392 active+clean; 145 MiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.6 KiB/s wr, 7 op/s 2024-03-21T13:29:33.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:32 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:29:33.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:32 smithi097 ceph-mon[39416]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-03-21T13:29:33.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:32 smithi097 ceph-mon[39416]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:29:33.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:32 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:33.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:32 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:33.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:32 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-7", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:29:33.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:32 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-7"}]: dispatch 2024-03-21T13:29:33.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:32 smithi097 ceph-mon[39416]: osdmap e97: 8 total, 8 up, 8 in 2024-03-21T13:29:33.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:32 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-7"}]: dispatch 2024-03-21T13:29:33.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:32 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3929544138' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi097-79618-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:33.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:32 smithi097 ceph-mon[30040]: pgmap v106: 712 pgs: 192 unknown, 96 creating+peering, 424 active+clean; 145 MiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-21T13:29:33.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:32 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/440005378' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi097-80861-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi097-80861-3"}]': finished 2024-03-21T13:29:33.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:32 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:33.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:32 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi097-79636-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:33.056 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:32 smithi097 ceph-mon[30040]: osdmap e96: 8 total, 8 up, 8 in 2024-03-21T13:29:33.056 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:32 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:33.056 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:32 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/821107064' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:33.056 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:32 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:29:33.056 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:32 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:29:33.056 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:32 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:33.056 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:32 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:29:33.056 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:32 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:29:33.056 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:32 smithi097 ceph-mon[30040]: pgmap v108: 624 pgs: 168 unknown, 64 creating+peering, 392 active+clean; 145 MiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.6 KiB/s wr, 7 op/s 2024-03-21T13:29:33.056 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:32 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:29:33.056 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:32 smithi097 ceph-mon[30040]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-03-21T13:29:33.056 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:32 smithi097 ceph-mon[30040]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:29:33.056 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:32 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:33.056 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:32 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:33.057 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:32 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-7", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:29:33.057 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:32 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-7"}]: dispatch 2024-03-21T13:29:33.057 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:32 smithi097 ceph-mon[30040]: osdmap e97: 8 total, 8 up, 8 in 2024-03-21T13:29:33.057 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:32 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-7"}]: dispatch 2024-03-21T13:29:33.057 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:32 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3929544138' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi097-79618-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:33.534 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: [==========] Running 4 tests from 1 test suite. 2024-03-21T13:29:33.534 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: [----------] Global test environment set-up. 2024-03-21T13:29:33.534 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP 2024-03-21T13:29:33.534 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterEarly 2024-03-21T13:29:33.534 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterEarly (5050 ms) 2024-03-21T13:29:33.534 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterLate 2024-03-21T13:29:33.534 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterLate (1042 ms) 2024-03-21T13:29:33.535 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Status 2024-03-21T13:29:33.535 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: [ OK ] LibRadosServicePP.Status (22451 ms) 2024-03-21T13:29:33.535 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Close 2024-03-21T13:29:33.535 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: attempt 0 of 20 2024-03-21T13:29:33.535 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: failed to deregister: 2024-03-21T13:29:33.535 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: { 2024-03-21T13:29:33.535 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "epoch": 21, 2024-03-21T13:29:33.535 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "modified": "2024-03-21T13:29:30.323371+0000", 2024-03-21T13:29:33.535 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "services": { 2024-03-21T13:29:33.535 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "laundry": { 2024-03-21T13:29:33.535 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "daemons": { 2024-03-21T13:29:33.535 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "summary": "", 2024-03-21T13:29:33.535 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "close-test-pid80277": { 2024-03-21T13:29:33.535 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "start_epoch": 21, 2024-03-21T13:29:33.535 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "start_stamp": "2024-03-21T13:29:28.510908+0000", 2024-03-21T13:29:33.536 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "gid": 15429, 2024-03-21T13:29:33.536 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "addr": "172.21.15.97:0/4037969855", 2024-03-21T13:29:33.536 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "metadata": { 2024-03-21T13:29:33.536 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "arch": "x86_64", 2024-03-21T13:29:33.536 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "ceph_release": "squid", 2024-03-21T13:29:33.536 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "ceph_version": "ceph version 19.0.0-2267-g960fbbe4 (960fbbe4645c2becc8084669be194c95183c7753) squid (dev)", 2024-03-21T13:29:33.536 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "ceph_version_short": "19.0.0-2267-g960fbbe4", 2024-03-21T13:29:33.536 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz", 2024-03-21T13:29:33.536 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "distro": "centos", 2024-03-21T13:29:33.536 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "distro_description": "CentOS Stream 9", 2024-03-21T13:29:33.536 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "distro_version": "9", 2024-03-21T13:29:33.536 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "foo": "bar", 2024-03-21T13:29:33.536 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "hostname": "smithi097", 2024-03-21T13:29:33.536 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024", 2024-03-21T13:29:33.537 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "kernel_version": "5.14.0-430.el9.x86_64", 2024-03-21T13:29:33.537 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "mem_swap_kb": "0", 2024-03-21T13:29:33.537 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "mem_total_kb": "32488448", 2024-03-21T13:29:33.537 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "os": "Linux", 2024-03-21T13:29:33.537 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "this": "that" 2024-03-21T13:29:33.537 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: }, 2024-03-21T13:29:33.537 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "task_status": {} 2024-03-21T13:29:33.537 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: } 2024-03-21T13:29:33.537 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: } 2024-03-21T13:29:33.537 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: }, 2024-03-21T13:29:33.537 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "rgw": { 2024-03-21T13:29:33.537 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "daemons": { 2024-03-21T13:29:33.537 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "summary": "", 2024-03-21T13:29:33.537 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "24365": { 2024-03-21T13:29:33.537 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "start_epoch": 12, 2024-03-21T13:29:33.537 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "start_stamp": "2024-03-21T13:26:18.338171+0000", 2024-03-21T13:29:33.538 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "gid": 24365, 2024-03-21T13:29:33.538 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "addr": "172.21.15.97:0/110547329", 2024-03-21T13:29:33.538 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "metadata": { 2024-03-21T13:29:33.538 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "arch": "x86_64", 2024-03-21T13:29:33.538 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "ceph_release": "squid", 2024-03-21T13:29:33.871 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:29:33 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:29:34.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:34 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:34.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:34 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-7"}]': finished 2024-03-21T13:29:34.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:34 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3929544138' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi097-79618-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:34.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:34 smithi097 ceph-mon[39416]: osdmap e98: 8 total, 8 up, 8 in 2024-03-21T13:29:34.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:34 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/63390924' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi097-79636-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:34.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:34 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/440005378' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi097-80861-3"}]: dispatch 2024-03-21T13:29:34.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:34 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:29:34.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:34 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:34.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:34 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-7"}]': finished 2024-03-21T13:29:34.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:34 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3929544138' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi097-79618-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:34.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:34 smithi097 ceph-mon[30040]: osdmap e98: 8 total, 8 up, 8 in 2024-03-21T13:29:34.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:34 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/63390924' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi097-79636-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:34.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:34 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/440005378' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi097-80861-3"}]: dispatch 2024-03-21T13:29:34.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:34 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:29:34.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:34 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:34.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:34 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-7"}]': finished 2024-03-21T13:29:34.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:34 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3929544138' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi097-79618-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:34.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:34 smithi144 ceph-mon[33633]: osdmap e98: 8 total, 8 up, 8 in 2024-03-21T13:29:34.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:34 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/63390924' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi097-79636-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:34.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:34 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/440005378' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi097-80861-3"}]: dispatch 2024-03-21T13:29:34.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:34 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:29:35.229 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "ceph_version": "ceph version 19.0.0-2267-g960fbbe4 (960fbbe4645c2becc8084669be194c9 ec_list: Running main() from gmock_main.cc 2024-03-21T13:29:35.229 INFO:tasks.workunit.client.0.smithi097.stdout: ec_list: [==========] Running 3 tests from 1 test suite. 2024-03-21T13:29:35.229 INFO:tasks.workunit.client.0.smithi097.stdout: ec_list: [----------] Global test environment set-up. 2024-03-21T13:29:35.229 INFO:tasks.workunit.client.0.smithi097.stdout: ec_list: [----------] 3 tests from NeoradosECList 2024-03-21T13:29:35.229 INFO:tasks.workunit.client.0.smithi097.stdout: ec_list: [ RUN ] NeoradosECList.ListObjects 2024-03-21T13:29:35.230 INFO:tasks.workunit.client.0.smithi097.stdout: ec_list: [ OK ] NeoradosECList.ListObjects (20302 ms) 2024-03-21T13:29:35.230 INFO:tasks.workunit.client.0.smithi097.stdout: ec_list: [ RUN ] NeoradosECList.ListObjectsNS 2024-03-21T13:29:35.230 INFO:tasks.workunit.client.0.smithi097.stdout: ec_list: [ OK ] NeoradosECList.ListObjectsNS (8006 ms) 2024-03-21T13:29:35.230 INFO:tasks.workunit.client.0.smithi097.stdout: ec_list: [ RUN ] NeoradosECList.ListObjectsMany 2024-03-21T13:29:35.230 INFO:tasks.workunit.client.0.smithi097.stdout: ec_list: [ OK ] NeoradosECList.ListObjectsMany (7076 ms) 2024-03-21T13:29:35.230 INFO:tasks.workunit.client.0.smithi097.stdout: ec_list: [----------] 3 tests from NeoradosECList (35384 ms total) 2024-03-21T13:29:35.230 INFO:tasks.workunit.client.0.smithi097.stdout: ec_list: 2024-03-21T13:29:35.230 INFO:tasks.workunit.client.0.smithi097.stdout: ec_list: [----------] Global test environment tear-down 2024-03-21T13:29:35.230 INFO:tasks.workunit.client.0.smithi097.stdout: ec_list: [==========] 3 tests from 1 test suite ran. (35384 ms total) 2024-03-21T13:29:35.230 INFO:tasks.workunit.client.0.smithi097.stdout: ec_list: [ PASSED ] 3 tests. 2024-03-21T13:29:35.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:35 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:29:35.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:35 smithi097 ceph-mon[39416]: pgmap v111: 648 pgs: 64 creating+peering, 23 creating+activating, 1 active+clean+snaptrim, 560 active+clean; 145 MiB data, 755 MiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2024-03-21T13:29:35.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:35 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2601202727' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-03-21T13:29:35.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:35 smithi097 ceph-mon[39416]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-03-21T13:29:35.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:35 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:29:35.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:35 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:35.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:35 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/63390924' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi097-79636-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:35.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:35 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/440005378' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi097-80861-3"}]': finished 2024-03-21T13:29:35.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:35 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:29:35.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:35 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-7"}]: dispatch 2024-03-21T13:29:35.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:35 smithi097 ceph-mon[39416]: osdmap e99: 8 total, 8 up, 8 in 2024-03-21T13:29:35.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:35 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4221965477' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:35.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:35 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-7"}]: dispatch 2024-03-21T13:29:35.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:35 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/440005378' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi097-80861-3"}]: dispatch 2024-03-21T13:29:35.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:35 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:35.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:35 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/457706421' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:35.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:35 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:29:35.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:35 smithi097 ceph-mon[30040]: pgmap v111: 648 pgs: 64 creating+peering, 23 creating+activating, 1 active+clean+snaptrim, 560 active+clean; 145 MiB data, 755 MiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2024-03-21T13:29:35.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:35 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2601202727' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-03-21T13:29:35.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:35 smithi097 ceph-mon[30040]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-03-21T13:29:35.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:35 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:29:35.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:35 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:35.307 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:35 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/63390924' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi097-79636-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:35.307 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:35 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/440005378' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi097-80861-3"}]': finished 2024-03-21T13:29:35.307 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:35 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:29:35.307 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:35 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-7"}]: dispatch 2024-03-21T13:29:35.307 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:35 smithi097 ceph-mon[30040]: osdmap e99: 8 total, 8 up, 8 in 2024-03-21T13:29:35.307 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:35 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4221965477' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:35.307 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:35 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-7"}]: dispatch 2024-03-21T13:29:35.307 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:35 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/440005378' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi097-80861-3"}]: dispatch 2024-03-21T13:29:35.307 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:35 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:35.307 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:35 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/457706421' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:35.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:35 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:29:35.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:35 smithi144 ceph-mon[33633]: pgmap v111: 648 pgs: 64 creating+peering, 23 creating+activating, 1 active+clean+snaptrim, 560 active+clean; 145 MiB data, 755 MiB used, 714 GiB / 715 GiB avail; 3.3 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2024-03-21T13:29:35.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:35 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2601202727' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-03-21T13:29:35.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:35 smithi144 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-03-21T13:29:35.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:35 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:29:35.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:35 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:35.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:35 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/63390924' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi097-79636-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:35.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:35 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/440005378' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi097-80861-3"}]': finished 2024-03-21T13:29:35.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:35 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:29:35.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:35 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-7"}]: dispatch 2024-03-21T13:29:35.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:35 smithi144 ceph-mon[33633]: osdmap e99: 8 total, 8 up, 8 in 2024-03-21T13:29:35.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:35 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4221965477' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:35.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:35 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-7"}]: dispatch 2024-03-21T13:29:35.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:35 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/440005378' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi097-80861-3"}]: dispatch 2024-03-21T13:29:35.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:35 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:35.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:35 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/457706421' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:36.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:36 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:36.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:36 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-7"}]': finished 2024-03-21T13:29:36.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:36 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/440005378' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi097-80861-3"}]': finished 2024-03-21T13:29:36.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:36 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:36.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:36 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/457706421' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:36.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:36 smithi097 ceph-mon[39416]: osdmap e100: 8 total, 8 up, 8 in 2024-03-21T13:29:36.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:36 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/474508915' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi097-79618-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:36.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:36 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi097-79618-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:36.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:36 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:36.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:36 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-7"}]': finished 2024-03-21T13:29:36.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:36 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/440005378' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi097-80861-3"}]': finished 2024-03-21T13:29:36.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:36 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:36.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:36 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/457706421' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:36.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:36 smithi097 ceph-mon[30040]: osdmap e100: 8 total, 8 up, 8 in 2024-03-21T13:29:36.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:36 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/474508915' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi097-79618-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:36.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:36 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi097-79618-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:36.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:36 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:36.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:36 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-7"}]': finished 2024-03-21T13:29:36.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:36 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/440005378' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi097-80861-3"}]': finished 2024-03-21T13:29:36.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:36 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:36.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:36 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/457706421' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-80309-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:36.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:36 smithi144 ceph-mon[33633]: osdmap e100: 8 total, 8 up, 8 in 2024-03-21T13:29:36.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:36 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/474508915' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi097-79618-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:36.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:36 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi097-79618-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:37.238 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_write_operations: Running main() from gmock_main.cc 2024-03-21T13:29:37.238 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_write_operations: [==========] Running 8 tests from 2 test suites. 2024-03-21T13:29:37.238 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_write_operations: [----------] Global test environment set-up. 2024-03-21T13:29:37.238 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps 2024-03-21T13:29:37.239 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_write_operations: [ RUN ] LibradosCWriteOps.NewDelete 2024-03-21T13:29:37.239 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_write_operations: [ OK ] LibradosCWriteOps.NewDelete (0 ms) 2024-03-21T13:29:37.239 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps (0 ms total) 2024-03-21T13:29:37.239 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_write_operations: 2024-03-21T13:29:37.239 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps 2024-03-21T13:29:37.239 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.assertExists 2024-03-21T13:29:37.239 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.assertExists (15042 ms) 2024-03-21T13:29:37.239 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteOpAssertVersion 2024-03-21T13:29:37.239 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteOpAssertVersion (4424 ms) 2024-03-21T13:29:37.239 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Xattrs 2024-03-21T13:29:37.239 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Xattrs (3144 ms) 2024-03-21T13:29:37.239 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Write 2024-03-21T13:29:37.239 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Write (3805 ms) 2024-03-21T13:29:37.239 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Exec 2024-03-21T13:29:37.239 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Exec (3091 ms) 2024-03-21T13:29:37.240 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteSame 2024-03-21T13:29:37.240 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteSame (4062 ms) 2024-03-21T13:29:37.240 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.CmpExt 2024-03-21T13:29:37.240 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.CmpExt (4008 ms) 2024-03-21T13:29:37.240 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps (37577 ms total) 2024-03-21T13:29:37.240 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_write_operations: 2024-03-21T13:29:37.240 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_write_operations: [----------] Global test environment tear-down 2024-03-21T13:29:37.240 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_write_operations: [==========] 8 tests from 2 test suites ran. (37577 ms total) 2024-03-21T13:29:37.240 INFO:tasks.workunit.client.0.smithi097.stdout: api_c_write_operations: [ PASSED ] 8 tests. 2024-03-21T13:29:37.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:37 smithi097 ceph-mon[39416]: pgmap v114: 680 pgs: 160 unknown, 7 creating+activating, 1 active+clean+snaptrim, 512 active+clean; 145 MiB data, 755 MiB used, 714 GiB / 715 GiB avail; 2.5 KiB/s rd, 2.0 KiB/s wr, 4 op/s 2024-03-21T13:29:37.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:37 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:37.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:37 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi097-79618-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:37.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:37 smithi097 ceph-mon[39416]: osdmap e101: 8 total, 8 up, 8 in 2024-03-21T13:29:37.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:37 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1210104199' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi097-79636-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:37.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:37 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi097-79636-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:37.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:37 smithi097 ceph-mon[30040]: pgmap v114: 680 pgs: 160 unknown, 7 creating+activating, 1 active+clean+snaptrim, 512 active+clean; 145 MiB data, 755 MiB used, 714 GiB / 715 GiB avail; 2.5 KiB/s rd, 2.0 KiB/s wr, 4 op/s 2024-03-21T13:29:37.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:37 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:37.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:37 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi097-79618-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:37.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:37 smithi097 ceph-mon[30040]: osdmap e101: 8 total, 8 up, 8 in 2024-03-21T13:29:37.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:37 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1210104199' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi097-79636-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:37.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:37 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi097-79636-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:37.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:37 smithi144 ceph-mon[33633]: pgmap v114: 680 pgs: 160 unknown, 7 creating+activating, 1 active+clean+snaptrim, 512 active+clean; 145 MiB data, 755 MiB used, 714 GiB / 715 GiB avail; 2.5 KiB/s rd, 2.0 KiB/s wr, 4 op/s 2024-03-21T13:29:37.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:37 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:37.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:37 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi097-79618-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:37.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:37 smithi144 ceph-mon[33633]: osdmap e101: 8 total, 8 up, 8 in 2024-03-21T13:29:37.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:37 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1210104199' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi097-79636-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:37.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:37 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi097-79636-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:38.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:38 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:38.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:38 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi097-79636-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:38.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:38 smithi097 ceph-mon[30040]: osdmap e102: 8 total, 8 up, 8 in 2024-03-21T13:29:38.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:38 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:38.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:38 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:38.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:38 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/795380094' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4"}]: dispatch 2024-03-21T13:29:38.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:38 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4"}]: dispatch 2024-03-21T13:29:38.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:38 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:38.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:38 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi097-79636-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:38.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:38 smithi097 ceph-mon[39416]: osdmap e102: 8 total, 8 up, 8 in 2024-03-21T13:29:38.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:38 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:38.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:38 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:38.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:38 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/795380094' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4"}]: dispatch 2024-03-21T13:29:38.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:38 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4"}]: dispatch 2024-03-21T13:29:38.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:38 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:38.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:38 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi097-79636-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:38.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:38 smithi144 ceph-mon[33633]: osdmap e102: 8 total, 8 up, 8 in 2024-03-21T13:29:38.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:38 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:38.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:38 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:38.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:38 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/795380094' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4"}]: dispatch 2024-03-21T13:29:38.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:38 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4"}]: dispatch 2024-03-21T13:29:38.580 INFO:tasks.workunit.client.0.smithi097.stdout:5183c7753) squid (dev)", 2024-03-21T13:29:38.580 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "ceph_version_short": "19.0.0-2267-g960fbbe4", 2024-03-21T13:29:38.580 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "container_hostname": "smithi097", 2024-03-21T13:29:38.580 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "container_image": "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93", 2024-03-21T13:29:38.580 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz", 2024-03-21T13:29:38.580 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "distro": "centos", 2024-03-21T13:29:38.580 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "distro_description": "CentOS Stream 8", 2024-03-21T13:29:38.580 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "distro_version": "8", 2024-03-21T13:29:38.580 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "frontend_config#0": "beast port=80", 2024-03-21T13:29:38.580 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "frontend_type#0": "beast", 2024-03-21T13:29:38.580 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "hostname": "smithi097", 2024-03-21T13:29:38.580 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "id": "foo.a", 2024-03-21T13:29:38.580 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024", 2024-03-21T13:29:38.580 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "kernel_version": "5.14.0-430.el9.x86_64", 2024-03-21T13:29:38.580 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "mem_swap_kb": "0", 2024-03-21T13:29:38.581 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "mem_total_kb": "32488448", 2024-03-21T13:29:38.581 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "num_handles": "1", 2024-03-21T13:29:38.581 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "os": "Linux", 2024-03-21T13:29:38.581 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "pid": "7", 2024-03-21T13:29:38.581 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "realm_id": "", 2024-03-21T13:29:38.581 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "realm_name": "", 2024-03-21T13:29:38.581 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "zone_id": "05021e79-2756-4d82-a884-be9d6d267269", 2024-03-21T13:29:38.581 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "zone_name": "default", 2024-03-21T13:29:38.581 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "zonegroup_id": "ea3541c8-5570-4782-8e7a-ffb5231d1677", 2024-03-21T13:29:38.581 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "zonegroup_name": "default" 2024-03-21T13:29:38.581 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: }, 2024-03-21T13:29:38.581 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: "task_status": {} 2024-03-21T13:29:38.581 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: } 2024-03-21T13:29:38.581 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: } 2024-03-21T13:29:38.581 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: } 2024-03-21T13:29:38.582 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: } 2024-03-21T13:29:38.582 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: } 2024-03-21T13:29:38.582 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: 2024-03-21T13:29:38.582 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: attempt 1 of 20 2024-03-21T13:29:38.582 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: [ OK ] LibRadosServicePP.Close (10367 ms) 2024-03-21T13:29:38.582 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP (38910 ms total) 2024-03-21T13:29:38.582 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: 2024-03-21T13:29:38.582 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: [----------] Global test environment tear-down 2024-03-21T13:29:38.582 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: [==========] 4 tests from 1 test suite ran. (38910 ms total) 2024-03-21T13:29:38.582 INFO:tasks.workunit.client.0.smithi097.stdout: api_service_pp: [ PASSED ] 4 tests. 2024-03-21T13:29:39.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:39 smithi097 ceph-mon[39416]: pgmap v117: 648 pgs: 160 unknown, 7 creating+activating, 1 active+clean+snaptrim, 480 active+clean; 145 MiB data, 755 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:29:39.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:39 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:39.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:39 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:39.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:39 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4"}]': finished 2024-03-21T13:29:39.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:39 smithi097 ceph-mon[39416]: osdmap e103: 8 total, 8 up, 8 in 2024-03-21T13:29:39.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:39 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/795380094' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4", "tierpool": "test-rados-api-smithi097-80028-6"}]: dispatch 2024-03-21T13:29:39.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:39 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/264539440' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi097-79618-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:39.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:39 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4", "tierpool": "test-rados-api-smithi097-80028-6"}]: dispatch 2024-03-21T13:29:39.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:39 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:29:39.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:39 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1794642294' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-03-21T13:29:39.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:39 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:29:39.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:39 smithi097 ceph-mon[30040]: pgmap v117: 648 pgs: 160 unknown, 7 creating+activating, 1 active+clean+snaptrim, 480 active+clean; 145 MiB data, 755 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:29:39.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:39 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:39.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:39 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:39.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:39 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4"}]': finished 2024-03-21T13:29:39.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:39 smithi097 ceph-mon[30040]: osdmap e103: 8 total, 8 up, 8 in 2024-03-21T13:29:39.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:39 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/795380094' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4", "tierpool": "test-rados-api-smithi097-80028-6"}]: dispatch 2024-03-21T13:29:39.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:39 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/264539440' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi097-79618-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:39.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:39 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4", "tierpool": "test-rados-api-smithi097-80028-6"}]: dispatch 2024-03-21T13:29:39.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:39 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:29:39.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:39 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1794642294' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-03-21T13:29:39.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:39 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:29:39.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:39 smithi144 ceph-mon[33633]: pgmap v117: 648 pgs: 160 unknown, 7 creating+activating, 1 active+clean+snaptrim, 480 active+clean; 145 MiB data, 755 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:29:39.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:39 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:39.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:39 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:39.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:39 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4"}]': finished 2024-03-21T13:29:39.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:39 smithi144 ceph-mon[33633]: osdmap e103: 8 total, 8 up, 8 in 2024-03-21T13:29:39.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:39 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/795380094' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4", "tierpool": "test-rados-api-smithi097-80028-6"}]: dispatch 2024-03-21T13:29:39.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:39 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/264539440' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi097-79618-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:39.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:39 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4", "tierpool": "test-rados-api-smithi097-80028-6"}]: dispatch 2024-03-21T13:29:39.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:39 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:29:39.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:39 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1794642294' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-03-21T13:29:39.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:39 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:29:40.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:40 smithi097 ceph-mon[39416]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-03-21T13:29:40.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:40 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:40.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:40 smithi097 ceph-mon[39416]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:29:40.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:40 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:29:40.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:40 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:29:40.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:40 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/264539440' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi097-79618-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:40.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:40 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4", "tierpool": "test-rados-api-smithi097-80028-6"}]': finished 2024-03-21T13:29:40.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:40 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-9", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:29:40.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:40 smithi097 ceph-mon[39416]: osdmap e104: 8 total, 8 up, 8 in 2024-03-21T13:29:40.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:40 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-9"}]: dispatch 2024-03-21T13:29:40.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:40 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-9"}]: dispatch 2024-03-21T13:29:40.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:40 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3265369317' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:40.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:40 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/795380094' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi097-80028-6", "pool2": "test-rados-api-smithi097-80028-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T13:29:40.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:40 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi097-80028-6", "pool2": "test-rados-api-smithi097-80028-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T13:29:40.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:40 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3693278713' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi097-79636-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:40.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:40 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi097-79636-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:40.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:40 smithi097 ceph-mon[39416]: pgmap v120: 680 pgs: 64 creating+peering, 128 unknown, 488 active+clean; 145 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.0 KiB/s wr, 5 op/s 2024-03-21T13:29:40.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:40 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:40.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:40 smithi097 ceph-mon[30040]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-03-21T13:29:40.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:40 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:40.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:40 smithi097 ceph-mon[30040]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:29:40.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:40 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:29:40.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:40 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:29:40.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:40 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/264539440' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi097-79618-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:40.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:40 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4", "tierpool": "test-rados-api-smithi097-80028-6"}]': finished 2024-03-21T13:29:40.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:40 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-9", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:29:40.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:40 smithi097 ceph-mon[30040]: osdmap e104: 8 total, 8 up, 8 in 2024-03-21T13:29:40.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:40 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-9"}]: dispatch 2024-03-21T13:29:40.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:40 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-9"}]: dispatch 2024-03-21T13:29:40.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:40 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3265369317' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:40.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:40 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/795380094' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi097-80028-6", "pool2": "test-rados-api-smithi097-80028-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T13:29:40.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:40 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi097-80028-6", "pool2": "test-rados-api-smithi097-80028-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T13:29:40.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:40 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3693278713' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi097-79636-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:40.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:40 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi097-79636-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:40.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:40 smithi097 ceph-mon[30040]: pgmap v120: 680 pgs: 64 creating+peering, 128 unknown, 488 active+clean; 145 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.0 KiB/s wr, 5 op/s 2024-03-21T13:29:40.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:40 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:40.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:40 smithi144 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-03-21T13:29:40.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:40 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:40.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:40 smithi144 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:29:40.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:40 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:29:40.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:40 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:29:40.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:40 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/264539440' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi097-79618-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:40.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:40 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi097-80028-4", "tierpool": "test-rados-api-smithi097-80028-6"}]': finished 2024-03-21T13:29:40.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:40 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-9", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:29:40.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:40 smithi144 ceph-mon[33633]: osdmap e104: 8 total, 8 up, 8 in 2024-03-21T13:29:40.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:40 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-9"}]: dispatch 2024-03-21T13:29:40.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:40 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-9"}]: dispatch 2024-03-21T13:29:40.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:40 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3265369317' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:40.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:40 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/795380094' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi097-80028-6", "pool2": "test-rados-api-smithi097-80028-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T13:29:40.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:40 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi097-80028-6", "pool2": "test-rados-api-smithi097-80028-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T13:29:40.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:40 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3693278713' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi097-79636-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:40.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:40 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi097-79636-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:40.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:40 smithi144 ceph-mon[33633]: pgmap v120: 680 pgs: 64 creating+peering, 128 unknown, 488 active+clean; 145 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.0 KiB/s wr, 5 op/s 2024-03-21T13:29:40.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:40 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:41.265 INFO:tasks.workunit.client.0.smithi097.stdout:h_notify_pp: flushed 2024-03-21T13:29:41.265 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 (3007 ms) 2024-03-21T13:29:41.265 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 2024-03-21T13:29:41.265 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: trying... 2024-03-21T13:29:41.265 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: handle_notify cookie 94810143965712 notify_id 399431958535 notifier_gid 25153 2024-03-21T13:29:41.265 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: timed out 2024-03-21T13:29:41.265 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: flushing 2024-03-21T13:29:41.266 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: flushed 2024-03-21T13:29:41.266 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 (3047 ms) 2024-03-21T13:29:41.266 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 2024-03-21T13:29:41.266 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: List watches 2024-03-21T13:29:41.266 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: notify2 2024-03-21T13:29:41.266 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: handle_notify cookie 94810143965712 notify_id 412316860424 notifier_gid 25153 2024-03-21T13:29:41.266 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: notify2 done 2024-03-21T13:29:41.266 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: watch_check 2024-03-21T13:29:41.266 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: unwatch2 2024-03-21T13:29:41.266 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: flushing 2024-03-21T13:29:41.266 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: done 2024-03-21T13:29:41.266 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 (3038 ms) 2024-03-21T13:29:41.266 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 2024-03-21T13:29:41.266 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: List watches 2024-03-21T13:29:41.266 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: notify2 2024-03-21T13:29:41.266 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: handle_notify cookie 94810143965712 notify_id 425201762313 notifier_gid 25153 2024-03-21T13:29:41.267 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: notify2 done 2024-03-21T13:29:41.267 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: watch_check 2024-03-21T13:29:41.267 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: unwatch2 2024-03-21T13:29:41.267 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: flushing 2024-03-21T13:29:41.267 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: done 2024-03-21T13:29:41.267 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 (3006 ms) 2024-03-21T13:29:41.267 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP (16146 ms total) 2024-03-21T13:29:41.267 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: 2024-03-21T13:29:41.267 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: [----------] Global test environment tear-down 2024-03-21T13:29:41.267 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: [==========] 16 tests from 2 test suites ran. (41707 ms total) 2024-03-21T13:29:41.267 INFO:tasks.workunit.client.0.smithi097.stdout: api_watch_notify_pp: [ PASSED ] 16 tests. 2024-03-21T13:29:41.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:41 smithi097 ceph-mon[30040]: Health check update: 9 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:29:41.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:41 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-9"}]': finished 2024-03-21T13:29:41.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:41 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3265369317' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:41.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:41 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi097-80028-6", "pool2": "test-rados-api-smithi097-80028-6", "yes_i_really_really_mean_it": true}]': finished 2024-03-21T13:29:41.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:41 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi097-79636-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:41.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:41 smithi097 ceph-mon[30040]: osdmap e105: 8 total, 8 up, 8 in 2024-03-21T13:29:41.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:41 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-9", "mode": "writeback"}]: dispatch 2024-03-21T13:29:41.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:41 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-9", "mode": "writeback"}]: dispatch 2024-03-21T13:29:41.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:41 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:41.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:41 smithi097 ceph-mon[39416]: Health check update: 9 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:29:41.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:41 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-9"}]': finished 2024-03-21T13:29:41.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:41 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3265369317' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:41.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:41 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi097-80028-6", "pool2": "test-rados-api-smithi097-80028-6", "yes_i_really_really_mean_it": true}]': finished 2024-03-21T13:29:41.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:41 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi097-79636-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:41.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:41 smithi097 ceph-mon[39416]: osdmap e105: 8 total, 8 up, 8 in 2024-03-21T13:29:41.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:41 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-9", "mode": "writeback"}]: dispatch 2024-03-21T13:29:41.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:41 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-9", "mode": "writeback"}]: dispatch 2024-03-21T13:29:41.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:41 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:41.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:41 smithi144 ceph-mon[33633]: Health check update: 9 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:29:41.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:41 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-9"}]': finished 2024-03-21T13:29:41.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:41 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3265369317' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:41.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:41 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi097-80028-6", "pool2": "test-rados-api-smithi097-80028-6", "yes_i_really_really_mean_it": true}]': finished 2024-03-21T13:29:41.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:41 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi097-79636-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:41.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:41 smithi144 ceph-mon[33633]: osdmap e105: 8 total, 8 up, 8 in 2024-03-21T13:29:41.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:41 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-9", "mode": "writeback"}]: dispatch 2024-03-21T13:29:41.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:41 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-9", "mode": "writeback"}]: dispatch 2024-03-21T13:29:41.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:41 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:42.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:29:41 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:29:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:29:42.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:42 smithi097 ceph-mon[30040]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:29:42.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:42 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-9", "mode": "writeback"}]': finished 2024-03-21T13:29:42.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:42 smithi097 ceph-mon[30040]: osdmap e106: 8 total, 8 up, 8 in 2024-03-21T13:29:42.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:42 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/531408751' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi097-79618-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:42.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:42 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi097-79618-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:42.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:42 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4215771965' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi097-80000-10"}]: dispatch 2024-03-21T13:29:42.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:42 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:29:42.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:42 smithi097 ceph-mon[30040]: pgmap v123: 612 pgs: 192 unknown, 420 active+clean; 145 MiB data, 754 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:29:42.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:42 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi097-80000-10"}]: dispatch 2024-03-21T13:29:42.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:42 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4215771965' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi097-80000-10"}]: dispatch 2024-03-21T13:29:42.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:42 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:29:42.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:42 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi097-80000-10"}]: dispatch 2024-03-21T13:29:42.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:42 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4215771965' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi097-80000-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:42.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:42 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi097-80000-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:42.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:42 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:42.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:42 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi097-79618-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:42.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:42 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:29:42.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:42 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi097-80000-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:29:42.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:42 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-9"}]: dispatch 2024-03-21T13:29:42.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:42 smithi097 ceph-mon[39416]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:29:42.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:42 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-9", "mode": "writeback"}]': finished 2024-03-21T13:29:42.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:42 smithi097 ceph-mon[39416]: osdmap e106: 8 total, 8 up, 8 in 2024-03-21T13:29:42.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:42 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/531408751' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi097-79618-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:42.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:42 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi097-79618-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:42.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:42 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4215771965' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi097-80000-10"}]: dispatch 2024-03-21T13:29:42.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:42 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:29:42.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:42 smithi097 ceph-mon[39416]: pgmap v123: 612 pgs: 192 unknown, 420 active+clean; 145 MiB data, 754 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:29:42.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:42 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi097-80000-10"}]: dispatch 2024-03-21T13:29:42.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:42 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4215771965' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi097-80000-10"}]: dispatch 2024-03-21T13:29:42.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:42 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:29:42.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:42 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi097-80000-10"}]: dispatch 2024-03-21T13:29:42.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:42 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4215771965' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi097-80000-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:42.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:42 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi097-80000-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:42.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:42 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:42.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:42 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi097-79618-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:42.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:42 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:29:42.807 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:42 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi097-80000-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:29:42.807 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:42 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-9"}]: dispatch 2024-03-21T13:29:42.807 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:42 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4215771965' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi097-80000-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi097-80000-10"}]: dispatch 2024-03-21T13:29:42.807 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:42 smithi097 ceph-mon[30040]: osdmap e107: 8 total, 8 up, 8 in 2024-03-21T13:29:42.807 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:42 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-9"}]: dispatch 2024-03-21T13:29:42.807 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:42 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1360567586' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi097-79636-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:42.807 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:42 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi097-80000-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi097-80000-10"}]: dispatch 2024-03-21T13:29:42.807 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:42 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi097-79636-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:42.807 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:42 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4215771965' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi097-80000-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi097-80000-10"}]: dispatch 2024-03-21T13:29:42.808 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:42 smithi097 ceph-mon[39416]: osdmap e107: 8 total, 8 up, 8 in 2024-03-21T13:29:42.808 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:42 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-9"}]: dispatch 2024-03-21T13:29:42.808 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:42 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1360567586' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi097-79636-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:42.808 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:42 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi097-80000-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi097-80000-10"}]: dispatch 2024-03-21T13:29:42.808 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:42 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi097-79636-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:42.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:42 smithi144 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:29:42.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:42 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-9", "mode": "writeback"}]': finished 2024-03-21T13:29:42.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:42 smithi144 ceph-mon[33633]: osdmap e106: 8 total, 8 up, 8 in 2024-03-21T13:29:42.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:42 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/531408751' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi097-79618-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:42.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:42 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi097-79618-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:42.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:42 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4215771965' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi097-80000-10"}]: dispatch 2024-03-21T13:29:42.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:42 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:29:42.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:42 smithi144 ceph-mon[33633]: pgmap v123: 612 pgs: 192 unknown, 420 active+clean; 145 MiB data, 754 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:29:42.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:42 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi097-80000-10"}]: dispatch 2024-03-21T13:29:42.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:42 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4215771965' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi097-80000-10"}]: dispatch 2024-03-21T13:29:42.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:42 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:29:42.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:42 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi097-80000-10"}]: dispatch 2024-03-21T13:29:42.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:42 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4215771965' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi097-80000-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:42.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:42 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi097-80000-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:42.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:42 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:42.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:42 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi097-79618-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:42.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:42 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:29:42.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:42 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi097-80000-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:29:42.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:42 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-9"}]: dispatch 2024-03-21T13:29:42.873 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:42 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4215771965' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi097-80000-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi097-80000-10"}]: dispatch 2024-03-21T13:29:42.873 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:42 smithi144 ceph-mon[33633]: osdmap e107: 8 total, 8 up, 8 in 2024-03-21T13:29:42.873 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:42 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-9"}]: dispatch 2024-03-21T13:29:42.873 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:42 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1360567586' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi097-79636-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:42.873 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:42 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi097-80000-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi097-80000-10"}]: dispatch 2024-03-21T13:29:42.873 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:42 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi097-79636-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:43.774 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:29:43 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:29:44.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.0"}]: dispatch 2024-03-21T13:29:44.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[39416]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.0"}]: dispatch 2024-03-21T13:29:44.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.1"}]: dispatch 2024-03-21T13:29:44.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[39416]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.1"}]: dispatch 2024-03-21T13:29:44.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.2"}]: dispatch 2024-03-21T13:29:44.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[39416]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.2"}]: dispatch 2024-03-21T13:29:44.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.3"}]: dispatch 2024-03-21T13:29:44.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[39416]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.3"}]: dispatch 2024-03-21T13:29:44.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.4"}]: dispatch 2024-03-21T13:29:44.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[39416]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.4"}]: dispatch 2024-03-21T13:29:44.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.5"}]: dispatch 2024-03-21T13:29:44.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[39416]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.5"}]: dispatch 2024-03-21T13:29:44.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.6"}]: dispatch 2024-03-21T13:29:44.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[39416]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.6"}]: dispatch 2024-03-21T13:29:44.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.7"}]: dispatch 2024-03-21T13:29:44.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[39416]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.7"}]: dispatch 2024-03-21T13:29:44.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.8"}]: dispatch 2024-03-21T13:29:44.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[39416]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.8"}]: dispatch 2024-03-21T13:29:44.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.9"}]: dispatch 2024-03-21T13:29:44.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[39416]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.9"}]: dispatch 2024-03-21T13:29:44.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[39416]: 15.6 deep-scrub starts 2024-03-21T13:29:44.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[39416]: 15.9 deep-scrub starts 2024-03-21T13:29:44.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[39416]: 15.9 deep-scrub ok 2024-03-21T13:29:44.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[39416]: 15.6 deep-scrub ok 2024-03-21T13:29:44.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[39416]: 15.0 deep-scrub starts 2024-03-21T13:29:44.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[39416]: 15.0 deep-scrub ok 2024-03-21T13:29:44.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[39416]: 15.3 deep-scrub starts 2024-03-21T13:29:44.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[39416]: 15.3 deep-scrub ok 2024-03-21T13:29:44.556 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:44.556 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[39416]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:29:44.556 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-9"}]': finished 2024-03-21T13:29:44.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.0"}]: dispatch 2024-03-21T13:29:44.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[30040]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.0"}]: dispatch 2024-03-21T13:29:44.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.1"}]: dispatch 2024-03-21T13:29:44.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[30040]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.1"}]: dispatch 2024-03-21T13:29:44.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.2"}]: dispatch 2024-03-21T13:29:44.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[30040]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.2"}]: dispatch 2024-03-21T13:29:44.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.3"}]: dispatch 2024-03-21T13:29:44.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[30040]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.3"}]: dispatch 2024-03-21T13:29:44.557 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.4"}]: dispatch 2024-03-21T13:29:44.557 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[30040]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.4"}]: dispatch 2024-03-21T13:29:44.557 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.5"}]: dispatch 2024-03-21T13:29:44.557 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[30040]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.5"}]: dispatch 2024-03-21T13:29:44.557 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.6"}]: dispatch 2024-03-21T13:29:44.557 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[30040]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.6"}]: dispatch 2024-03-21T13:29:44.557 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.7"}]: dispatch 2024-03-21T13:29:44.557 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[30040]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.7"}]: dispatch 2024-03-21T13:29:44.557 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.8"}]: dispatch 2024-03-21T13:29:44.557 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[30040]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.8"}]: dispatch 2024-03-21T13:29:44.557 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.9"}]: dispatch 2024-03-21T13:29:44.557 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[30040]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.9"}]: dispatch 2024-03-21T13:29:44.557 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[30040]: 15.6 deep-scrub starts 2024-03-21T13:29:44.557 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[30040]: 15.9 deep-scrub starts 2024-03-21T13:29:44.557 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[30040]: 15.9 deep-scrub ok 2024-03-21T13:29:44.557 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[30040]: 15.6 deep-scrub ok 2024-03-21T13:29:44.558 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[30040]: 15.0 deep-scrub starts 2024-03-21T13:29:44.558 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[30040]: 15.0 deep-scrub ok 2024-03-21T13:29:44.558 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[30040]: 15.3 deep-scrub starts 2024-03-21T13:29:44.558 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[30040]: 15.3 deep-scrub ok 2024-03-21T13:29:44.558 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:44.558 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[30040]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:29:44.558 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-9"}]': finished 2024-03-21T13:29:44.558 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi097-79636-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:44.558 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[30040]: osdmap e108: 8 total, 8 up, 8 in 2024-03-21T13:29:44.558 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi097-79636-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:44.558 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:44 smithi097 ceph-mon[39416]: osdmap e108: 8 total, 8 up, 8 in 2024-03-21T13:29:44.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:44 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.0"}]: dispatch 2024-03-21T13:29:44.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:44 smithi144 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.0"}]: dispatch 2024-03-21T13:29:44.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:44 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.1"}]: dispatch 2024-03-21T13:29:44.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:44 smithi144 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.1"}]: dispatch 2024-03-21T13:29:44.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:44 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.2"}]: dispatch 2024-03-21T13:29:44.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:44 smithi144 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.2"}]: dispatch 2024-03-21T13:29:44.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:44 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.3"}]: dispatch 2024-03-21T13:29:44.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:44 smithi144 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.3"}]: dispatch 2024-03-21T13:29:44.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:44 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.4"}]: dispatch 2024-03-21T13:29:44.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:44 smithi144 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.4"}]: dispatch 2024-03-21T13:29:44.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:44 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.5"}]: dispatch 2024-03-21T13:29:44.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:44 smithi144 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.5"}]: dispatch 2024-03-21T13:29:44.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:44 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.6"}]: dispatch 2024-03-21T13:29:44.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:44 smithi144 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.6"}]: dispatch 2024-03-21T13:29:44.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:44 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.7"}]: dispatch 2024-03-21T13:29:44.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:44 smithi144 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.7"}]: dispatch 2024-03-21T13:29:44.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:44 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.8"}]: dispatch 2024-03-21T13:29:44.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:44 smithi144 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.8"}]: dispatch 2024-03-21T13:29:44.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:44 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.9"}]: dispatch 2024-03-21T13:29:44.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:44 smithi144 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.9"}]: dispatch 2024-03-21T13:29:44.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:44 smithi144 ceph-mon[33633]: 15.6 deep-scrub starts 2024-03-21T13:29:44.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:44 smithi144 ceph-mon[33633]: 15.9 deep-scrub starts 2024-03-21T13:29:44.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:44 smithi144 ceph-mon[33633]: 15.9 deep-scrub ok 2024-03-21T13:29:44.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:44 smithi144 ceph-mon[33633]: 15.6 deep-scrub ok 2024-03-21T13:29:44.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:44 smithi144 ceph-mon[33633]: 15.0 deep-scrub starts 2024-03-21T13:29:44.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:44 smithi144 ceph-mon[33633]: 15.0 deep-scrub ok 2024-03-21T13:29:44.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:44 smithi144 ceph-mon[33633]: 15.3 deep-scrub starts 2024-03-21T13:29:44.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:44 smithi144 ceph-mon[33633]: 15.3 deep-scrub ok 2024-03-21T13:29:44.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:44 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:44.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:44 smithi144 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:29:44.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:44 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-9"}]': finished 2024-03-21T13:29:44.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:44 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi097-79636-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:44.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:44 smithi144 ceph-mon[33633]: osdmap e108: 8 total, 8 up, 8 in 2024-03-21T13:29:45.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:45 smithi097 ceph-mon[39416]: 15.7 deep-scrub starts 2024-03-21T13:29:45.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:45 smithi097 ceph-mon[39416]: 15.7 deep-scrub ok 2024-03-21T13:29:45.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:45 smithi097 ceph-mon[39416]: 15.1 deep-scrub starts 2024-03-21T13:29:45.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:45 smithi097 ceph-mon[39416]: 15.1 deep-scrub ok 2024-03-21T13:29:45.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:45 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:29:45.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:45 smithi097 ceph-mon[39416]: pgmap v126: 516 pgs: 15 creating+peering, 17 unknown, 484 active+clean; 217 MiB data, 1011 MiB used, 714 GiB / 715 GiB avail; 18 MiB/s wr, 0 op/s 2024-03-21T13:29:45.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:45 smithi097 ceph-mon[39416]: 15.8 deep-scrub starts 2024-03-21T13:29:45.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:45 smithi097 ceph-mon[39416]: 15.8 deep-scrub ok 2024-03-21T13:29:45.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:45 smithi097 ceph-mon[39416]: 15.4 deep-scrub starts 2024-03-21T13:29:45.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:45 smithi097 ceph-mon[39416]: 15.4 deep-scrub ok 2024-03-21T13:29:45.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:45 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:45.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:45 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi097-80000-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi097-80000-10"}]': finished 2024-03-21T13:29:45.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:45 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2981113507' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:45.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:45 smithi097 ceph-mon[39416]: osdmap e109: 8 total, 8 up, 8 in 2024-03-21T13:29:45.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:45 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:45.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:45 smithi097 ceph-mon[30040]: 15.7 deep-scrub starts 2024-03-21T13:29:45.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:45 smithi097 ceph-mon[30040]: 15.7 deep-scrub ok 2024-03-21T13:29:45.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:45 smithi097 ceph-mon[30040]: 15.1 deep-scrub starts 2024-03-21T13:29:45.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:45 smithi097 ceph-mon[30040]: 15.1 deep-scrub ok 2024-03-21T13:29:45.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:45 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:29:45.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:45 smithi097 ceph-mon[30040]: pgmap v126: 516 pgs: 15 creating+peering, 17 unknown, 484 active+clean; 217 MiB data, 1011 MiB used, 714 GiB / 715 GiB avail; 18 MiB/s wr, 0 op/s 2024-03-21T13:29:45.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:45 smithi097 ceph-mon[30040]: 15.8 deep-scrub starts 2024-03-21T13:29:45.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:45 smithi097 ceph-mon[30040]: 15.8 deep-scrub ok 2024-03-21T13:29:45.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:45 smithi097 ceph-mon[30040]: 15.4 deep-scrub starts 2024-03-21T13:29:45.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:45 smithi097 ceph-mon[30040]: 15.4 deep-scrub ok 2024-03-21T13:29:45.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:45 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:45.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:45 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi097-80000-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi097-80000-10"}]': finished 2024-03-21T13:29:45.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:45 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2981113507' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:45.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:45 smithi097 ceph-mon[30040]: osdmap e109: 8 total, 8 up, 8 in 2024-03-21T13:29:45.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:45 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:45.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:45 smithi144 ceph-mon[33633]: 15.7 deep-scrub starts 2024-03-21T13:29:45.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:45 smithi144 ceph-mon[33633]: 15.7 deep-scrub ok 2024-03-21T13:29:45.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:45 smithi144 ceph-mon[33633]: 15.1 deep-scrub starts 2024-03-21T13:29:45.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:45 smithi144 ceph-mon[33633]: 15.1 deep-scrub ok 2024-03-21T13:29:45.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:45 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:29:45.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:45 smithi144 ceph-mon[33633]: pgmap v126: 516 pgs: 15 creating+peering, 17 unknown, 484 active+clean; 217 MiB data, 1011 MiB used, 714 GiB / 715 GiB avail; 18 MiB/s wr, 0 op/s 2024-03-21T13:29:45.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:45 smithi144 ceph-mon[33633]: 15.8 deep-scrub starts 2024-03-21T13:29:45.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:45 smithi144 ceph-mon[33633]: 15.8 deep-scrub ok 2024-03-21T13:29:45.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:45 smithi144 ceph-mon[33633]: 15.4 deep-scrub starts 2024-03-21T13:29:45.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:45 smithi144 ceph-mon[33633]: 15.4 deep-scrub ok 2024-03-21T13:29:45.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:45 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:45.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:45 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi097-80000-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi097-80000-10"}]': finished 2024-03-21T13:29:45.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:45 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2981113507' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:45.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:45 smithi144 ceph-mon[33633]: osdmap e109: 8 total, 8 up, 8 in 2024-03-21T13:29:45.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:45 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:46.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:46 smithi144 ceph-mon[33633]: 15.5 deep-scrub starts 2024-03-21T13:29:46.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:46 smithi144 ceph-mon[33633]: 15.5 deep-scrub ok 2024-03-21T13:29:46.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:46 smithi144 ceph-mon[33633]: 15.2 deep-scrub starts 2024-03-21T13:29:46.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:46 smithi144 ceph-mon[33633]: 15.2 deep-scrub ok 2024-03-21T13:29:46.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:46 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:46.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:46 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:46.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:46 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/463370506' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi097-79618-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:46.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:46 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:46.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:46 smithi144 ceph-mon[33633]: pgmap v129: 620 pgs: 232 unknown, 388 active+clean; 217 MiB data, 1011 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 18 MiB/s wr, 6 op/s 2024-03-21T13:29:46.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:46 smithi144 ceph-mon[33633]: osdmap e110: 8 total, 8 up, 8 in 2024-03-21T13:29:46.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:46 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi097-79618-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:46.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:46 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:46.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:46 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:46.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:46 smithi097 ceph-mon[39416]: 15.5 deep-scrub starts 2024-03-21T13:29:46.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:46 smithi097 ceph-mon[39416]: 15.5 deep-scrub ok 2024-03-21T13:29:46.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:46 smithi097 ceph-mon[39416]: 15.2 deep-scrub starts 2024-03-21T13:29:46.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:46 smithi097 ceph-mon[39416]: 15.2 deep-scrub ok 2024-03-21T13:29:46.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:46 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:46.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:46 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:46.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:46 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/463370506' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi097-79618-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:46.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:46 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:46.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:46 smithi097 ceph-mon[39416]: pgmap v129: 620 pgs: 232 unknown, 388 active+clean; 217 MiB data, 1011 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 18 MiB/s wr, 6 op/s 2024-03-21T13:29:46.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:46 smithi097 ceph-mon[39416]: osdmap e110: 8 total, 8 up, 8 in 2024-03-21T13:29:46.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:46 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi097-79618-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:46.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:46 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:46.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:46 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:46.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:46 smithi097 ceph-mon[30040]: 15.5 deep-scrub starts 2024-03-21T13:29:46.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:46 smithi097 ceph-mon[30040]: 15.5 deep-scrub ok 2024-03-21T13:29:46.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:46 smithi097 ceph-mon[30040]: 15.2 deep-scrub starts 2024-03-21T13:29:46.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:46 smithi097 ceph-mon[30040]: 15.2 deep-scrub ok 2024-03-21T13:29:46.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:46 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:46.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:46 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79932-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:46.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:46 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/463370506' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi097-79618-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:46.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:46 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:46.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:46 smithi097 ceph-mon[30040]: pgmap v129: 620 pgs: 232 unknown, 388 active+clean; 217 MiB data, 1011 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 18 MiB/s wr, 6 op/s 2024-03-21T13:29:46.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:46 smithi097 ceph-mon[30040]: osdmap e110: 8 total, 8 up, 8 in 2024-03-21T13:29:46.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:46 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi097-79618-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:46.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:46 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:46.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:46 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:47.243 INFO:tasks.workunit.client.0.smithi097.stdout: misc: Running main() from gmock_main.cc 2024-03-21T13:29:47.243 INFO:tasks.workunit.client.0.smithi097.stdout: misc: [==========] Running 12 tests from 1 test suite. 2024-03-21T13:29:47.243 INFO:tasks.workunit.client.0.smithi097.stdout: misc: [----------] Global test environment set-up. 2024-03-21T13:29:47.243 INFO:tasks.workunit.client.0.smithi097.stdout: misc: [----------] 12 tests from NeoRadosMisc 2024-03-21T13:29:47.243 INFO:tasks.workunit.client.0.smithi097.stdout: misc: [ RUN ] NeoRadosMisc.Version 2024-03-21T13:29:47.243 INFO:tasks.workunit.client.0.smithi097.stdout: misc: [ OK ] NeoRadosMisc.Version (9992 ms) 2024-03-21T13:29:47.243 INFO:tasks.workunit.client.0.smithi097.stdout: misc: [ RUN ] NeoRadosMisc.WaitOSDMap 2024-03-21T13:29:47.243 INFO:tasks.workunit.client.0.smithi097.stdout: misc: [ OK ] NeoRadosMisc.WaitOSDMap (6692 ms) 2024-03-21T13:29:47.243 INFO:tasks.workunit.client.0.smithi097.stdout: misc: [ RUN ] NeoRadosMisc.LongName 2024-03-21T13:29:47.243 INFO:tasks.workunit.client.0.smithi097.stdout: misc: [ OK ] NeoRadosMisc.LongName (3867 ms) 2024-03-21T13:29:47.243 INFO:tasks.workunit.client.0.smithi097.stdout: misc: [ RUN ] NeoRadosMisc.LongLocator 2024-03-21T13:29:47.243 INFO:tasks.workunit.client.0.smithi097.stdout: misc: [ OK ] NeoRadosMisc.LongLocator (2879 ms) 2024-03-21T13:29:47.244 INFO:tasks.workunit.client.0.smithi097.stdout: misc: [ RUN ] NeoRadosMisc.LongNamespace 2024-03-21T13:29:47.244 INFO:tasks.workunit.client.0.smithi097.stdout: misc: [ OK ] NeoRadosMisc.LongNamespace (2786 ms) 2024-03-21T13:29:47.244 INFO:tasks.workunit.client.0.smithi097.stdout: misc: [ RUN ] NeoRadosMisc.LongAttrName 2024-03-21T13:29:47.244 INFO:tasks.workunit.client.0.smithi097.stdout: misc: [ OK ] NeoRadosMisc.LongAttrName (3090 ms) 2024-03-21T13:29:47.244 INFO:tasks.workunit.client.0.smithi097.stdout: misc: [ RUN ] NeoRadosMisc.Exec 2024-03-21T13:29:47.244 INFO:tasks.workunit.client.0.smithi097.stdout: misc: [ OK ] NeoRadosMisc.Exec (4067 ms) 2024-03-21T13:29:47.244 INFO:tasks.workunit.client.0.smithi097.stdout: misc: [ RUN ] NeoRadosMisc.Operate1 2024-03-21T13:29:47.244 INFO:tasks.workunit.client.0.smithi097.stdout: misc: [ OK ] NeoRadosMisc.Operate1 (3018 ms) 2024-03-21T13:29:47.244 INFO:tasks.workunit.client.0.smithi097.stdout: misc: [ RUN ] NeoRadosMisc.Operate2 2024-03-21T13:29:47.244 INFO:tasks.workunit.client.0.smithi097.stdout: misc: [ OK ] NeoRadosMisc.Operate2 (3013 ms) 2024-03-21T13:29:47.244 INFO:tasks.workunit.client.0.smithi097.stdout: misc: [ RUN ] NeoRadosMisc.BigObject 2024-03-21T13:29:47.244 INFO:tasks.workunit.client.0.smithi097.stdout: misc: [ OK ] NeoRadosMisc.BigObject (2973 ms) 2024-03-21T13:29:47.244 INFO:tasks.workunit.client.0.smithi097.stdout: misc: [ RUN ] NeoRadosMisc.BigAttr 2024-03-21T13:29:47.244 INFO:tasks.workunit.client.0.smithi097.stdout: misc: [ OK ] NeoRadosMisc.BigAttr (2018 ms) 2024-03-21T13:29:47.244 INFO:tasks.workunit.client.0.smithi097.stdout: misc: [ RUN ] NeoRadosMisc.WriteSame 2024-03-21T13:29:47.245 INFO:tasks.workunit.client.0.smithi097.stdout: misc: [ OK ] NeoRadosMisc.WriteSame (2988 ms) 2024-03-21T13:29:47.245 INFO:tasks.workunit.client.0.smithi097.stdout: misc: [----------] 12 tests from NeoRadosMisc (47383 ms total) 2024-03-21T13:29:47.245 INFO:tasks.workunit.client.0.smithi097.stdout: misc: 2024-03-21T13:29:47.245 INFO:tasks.workunit.client.0.smithi097.stdout: misc: [----------] Global test environment tear-down 2024-03-21T13:29:47.245 INFO:tasks.workunit.client.0.smithi097.stdout: misc: [==========] 12 tests from 1 test suite ran. (47383 ms total) 2024-03-21T13:29:47.245 INFO:tasks.workunit.client.0.smithi097.stdout: misc: [ PASSED ] 12 tests. 2024-03-21T13:29:47.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:47 smithi097 ceph-mon[39416]: Health check update: 11 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:29:47.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:47 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi097-79618-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:47.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:47 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:47.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:47 smithi097 ceph-mon[39416]: osdmap e111: 8 total, 8 up, 8 in 2024-03-21T13:29:47.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:47 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2911933933' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi097-79636-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:47.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:47 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2981113507' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79932-10", "tierpool":"test-rados-api-smithi097-79932-10-cache", "force_nonempty":""}]: dispatch 2024-03-21T13:29:47.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:47 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79932-10", "tierpool":"test-rados-api-smithi097-79932-10-cache", "force_nonempty":""}]: dispatch 2024-03-21T13:29:47.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:47 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:47.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:47 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2911933933' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi097-79636-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:47.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:47 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79932-10", "tierpool":"test-rados-api-smithi097-79932-10-cache", "force_nonempty":""}]': finished 2024-03-21T13:29:47.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:47 smithi097 ceph-mon[39416]: osdmap e112: 8 total, 8 up, 8 in 2024-03-21T13:29:47.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:47 smithi097 ceph-mon[30040]: Health check update: 11 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:29:47.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:47 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi097-79618-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:47.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:47 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:47.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:47 smithi097 ceph-mon[30040]: osdmap e111: 8 total, 8 up, 8 in 2024-03-21T13:29:47.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:47 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2911933933' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi097-79636-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:47.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:47 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2981113507' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79932-10", "tierpool":"test-rados-api-smithi097-79932-10-cache", "force_nonempty":""}]: dispatch 2024-03-21T13:29:47.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:47 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79932-10", "tierpool":"test-rados-api-smithi097-79932-10-cache", "force_nonempty":""}]: dispatch 2024-03-21T13:29:47.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:47 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:47.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:47 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2911933933' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi097-79636-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:47.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:47 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79932-10", "tierpool":"test-rados-api-smithi097-79932-10-cache", "force_nonempty":""}]': finished 2024-03-21T13:29:47.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:47 smithi097 ceph-mon[30040]: osdmap e112: 8 total, 8 up, 8 in 2024-03-21T13:29:47.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:47 smithi144 ceph-mon[33633]: Health check update: 11 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:29:47.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:47 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi097-79618-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:47.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:47 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:47.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:47 smithi144 ceph-mon[33633]: osdmap e111: 8 total, 8 up, 8 in 2024-03-21T13:29:47.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:47 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2911933933' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi097-79636-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:47.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:47 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2981113507' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79932-10", "tierpool":"test-rados-api-smithi097-79932-10-cache", "force_nonempty":""}]: dispatch 2024-03-21T13:29:47.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:47 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79932-10", "tierpool":"test-rados-api-smithi097-79932-10-cache", "force_nonempty":""}]: dispatch 2024-03-21T13:29:47.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:47 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:47.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:47 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2911933933' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi097-79636-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:47.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:47 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79932-10", "tierpool":"test-rados-api-smithi097-79932-10-cache", "force_nonempty":""}]': finished 2024-03-21T13:29:47.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:47 smithi144 ceph-mon[33633]: osdmap e112: 8 total, 8 up, 8 in 2024-03-21T13:29:48.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:48 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2981113507' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79932-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:48.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:48 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79932-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:48.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:48 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:29:48.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:48 smithi097 ceph-mon[30040]: pgmap v132: 588 pgs: 200 unknown, 388 active+clean; 217 MiB data, 1011 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:29:48.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:48 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:29:48.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:48 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:48.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:48 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79932-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:48.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:48 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-11", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:29:48.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:48 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-11"}]: dispatch 2024-03-21T13:29:48.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:48 smithi097 ceph-mon[30040]: osdmap e113: 8 total, 8 up, 8 in 2024-03-21T13:29:48.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:48 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-11"}]: dispatch 2024-03-21T13:29:48.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:48 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2075534584' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi097-79618-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:48.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:48 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2981113507' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79932-10", "tierpool":"test-rados-api-smithi097-79932-10-cache"}]: dispatch 2024-03-21T13:29:48.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:48 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79932-10", "tierpool":"test-rados-api-smithi097-79932-10-cache"}]: dispatch 2024-03-21T13:29:48.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:48 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2981113507' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79932-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:48.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:48 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79932-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:48.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:48 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:29:48.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:48 smithi097 ceph-mon[39416]: pgmap v132: 588 pgs: 200 unknown, 388 active+clean; 217 MiB data, 1011 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:29:48.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:48 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:29:48.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:48 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:48.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:48 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79932-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:48.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:48 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-11", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:29:48.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:48 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-11"}]: dispatch 2024-03-21T13:29:48.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:48 smithi097 ceph-mon[39416]: osdmap e113: 8 total, 8 up, 8 in 2024-03-21T13:29:48.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:48 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-11"}]: dispatch 2024-03-21T13:29:48.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:48 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2075534584' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi097-79618-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:48.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:48 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2981113507' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79932-10", "tierpool":"test-rados-api-smithi097-79932-10-cache"}]: dispatch 2024-03-21T13:29:48.556 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:48 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79932-10", "tierpool":"test-rados-api-smithi097-79932-10-cache"}]: dispatch 2024-03-21T13:29:48.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:48 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2981113507' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79932-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:48.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:48 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79932-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:48.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:48 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:29:48.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:48 smithi144 ceph-mon[33633]: pgmap v132: 588 pgs: 200 unknown, 388 active+clean; 217 MiB data, 1011 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:29:48.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:48 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:29:48.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:48 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:48.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:48 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79932-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:48.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:48 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-11", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:29:48.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:48 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-11"}]: dispatch 2024-03-21T13:29:48.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:48 smithi144 ceph-mon[33633]: osdmap e113: 8 total, 8 up, 8 in 2024-03-21T13:29:48.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:48 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-11"}]: dispatch 2024-03-21T13:29:48.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:48 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2075534584' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi097-79618-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:48.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:48 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2981113507' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79932-10", "tierpool":"test-rados-api-smithi097-79932-10-cache"}]: dispatch 2024-03-21T13:29:48.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:48 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79932-10", "tierpool":"test-rados-api-smithi097-79932-10-cache"}]: dispatch 2024-03-21T13:29:49.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:49 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:49.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:49 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-11"}]': finished 2024-03-21T13:29:49.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:49 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2075534584' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi097-79618-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:49.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:49 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79932-10", "tierpool":"test-rados-api-smithi097-79932-10-cache"}]': finished 2024-03-21T13:29:49.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:49 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-11", "mode": "writeback"}]: dispatch 2024-03-21T13:29:49.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:49 smithi097 ceph-mon[39416]: osdmap e114: 8 total, 8 up, 8 in 2024-03-21T13:29:49.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:49 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-11", "mode": "writeback"}]: dispatch 2024-03-21T13:29:49.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:49 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4196093722' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi097-79636-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:49.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:49 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:49.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:49 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-11"}]': finished 2024-03-21T13:29:49.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:49 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2075534584' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi097-79618-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:49.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:49 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79932-10", "tierpool":"test-rados-api-smithi097-79932-10-cache"}]': finished 2024-03-21T13:29:49.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:49 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-11", "mode": "writeback"}]: dispatch 2024-03-21T13:29:49.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:49 smithi097 ceph-mon[30040]: osdmap e114: 8 total, 8 up, 8 in 2024-03-21T13:29:49.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:49 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-11", "mode": "writeback"}]: dispatch 2024-03-21T13:29:49.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:49 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4196093722' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi097-79636-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:49.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:49 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:49.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:49 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-11"}]': finished 2024-03-21T13:29:49.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:49 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2075534584' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi097-79618-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:49.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:49 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79932-10", "tierpool":"test-rados-api-smithi097-79932-10-cache"}]': finished 2024-03-21T13:29:49.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:49 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-11", "mode": "writeback"}]: dispatch 2024-03-21T13:29:49.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:49 smithi144 ceph-mon[33633]: osdmap e114: 8 total, 8 up, 8 in 2024-03-21T13:29:49.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:49 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-11", "mode": "writeback"}]: dispatch 2024-03-21T13:29:49.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:49 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4196093722' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi097-79636-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:50.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:50 smithi097 ceph-mon[39416]: pgmap v135: 588 pgs: 64 creating+peering, 32 unknown, 492 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 258 KiB/s wr, 4 op/s 2024-03-21T13:29:50.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:50 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:50.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:50 smithi097 ceph-mon[39416]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:29:50.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:50 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-11", "mode": "writeback"}]': finished 2024-03-21T13:29:50.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:50 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4196093722' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi097-79636-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:50.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:50 smithi097 ceph-mon[39416]: osdmap e115: 8 total, 8 up, 8 in 2024-03-21T13:29:50.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:50 smithi097 ceph-mon[30040]: pgmap v135: 588 pgs: 64 creating+peering, 32 unknown, 492 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 258 KiB/s wr, 4 op/s 2024-03-21T13:29:50.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:50 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:50.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:50 smithi097 ceph-mon[30040]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:29:50.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:50 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-11", "mode": "writeback"}]': finished 2024-03-21T13:29:50.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:50 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4196093722' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi097-79636-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:50.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:50 smithi097 ceph-mon[30040]: osdmap e115: 8 total, 8 up, 8 in 2024-03-21T13:29:50.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:50 smithi144 ceph-mon[33633]: pgmap v135: 588 pgs: 64 creating+peering, 32 unknown, 492 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 258 KiB/s wr, 4 op/s 2024-03-21T13:29:50.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:50 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:50.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:50 smithi144 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:29:50.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:50 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-11", "mode": "writeback"}]': finished 2024-03-21T13:29:50.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:50 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4196093722' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi097-79636-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:50.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:50 smithi144 ceph-mon[33633]: osdmap e115: 8 total, 8 up, 8 in 2024-03-21T13:29:51.245 INFO:tasks.workunit.client.0.smithi097.stdout: api_pool: Running main() from gmock_main.cc 2024-03-21T13:29:51.245 INFO:tasks.workunit.client.0.smithi097.stdout: api_pool: [==========] Running 9 tests from 1 test suite. 2024-03-21T13:29:51.245 INFO:tasks.workunit.client.0.smithi097.stdout: api_pool: [----------] Global test environment set-up. 2024-03-21T13:29:51.245 INFO:tasks.workunit.client.0.smithi097.stdout: api_pool: [----------] 9 tests from LibRadosPools 2024-03-21T13:29:51.245 INFO:tasks.workunit.client.0.smithi097.stdout: api_pool: [ RUN ] LibRadosPools.PoolList 2024-03-21T13:29:51.245 INFO:tasks.workunit.client.0.smithi097.stdout: api_pool: [ OK ] LibRadosPools.PoolList (12390 ms) 2024-03-21T13:29:51.245 INFO:tasks.workunit.client.0.smithi097.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup 2024-03-21T13:29:51.245 INFO:tasks.workunit.client.0.smithi097.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup (4622 ms) 2024-03-21T13:29:51.245 INFO:tasks.workunit.client.0.smithi097.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup2 2024-03-21T13:29:51.245 INFO:tasks.workunit.client.0.smithi097.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup2 (4638 ms) 2024-03-21T13:29:51.245 INFO:tasks.workunit.client.0.smithi097.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookupOtherInstance 2024-03-21T13:29:51.245 INFO:tasks.workunit.client.0.smithi097.stdout: api_pool: [ OK ] LibRadosPools.PoolLookupOtherInstance (3890 ms) 2024-03-21T13:29:51.245 INFO:tasks.workunit.client.0.smithi097.stdout: api_pool: [ RUN ] LibRadosPools.PoolReverseLookupOtherInstance 2024-03-21T13:29:51.246 INFO:tasks.workunit.client.0.smithi097.stdout: api_pool: [ OK ] LibRadosPools.PoolReverseLookupOtherInstance (3098 ms) 2024-03-21T13:29:51.246 INFO:tasks.workunit.client.0.smithi097.stdout: api_pool: [ RUN ] LibRadosPools.PoolDelete 2024-03-21T13:29:51.246 INFO:tasks.workunit.client.0.smithi097.stdout: api_pool: [ OK ] LibRadosPools.PoolDelete (5070 ms) 2024-03-21T13:29:51.246 INFO:tasks.workunit.client.0.smithi097.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateDelete 2024-03-21T13:29:51.246 INFO:tasks.workunit.client.0.smithi097.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateDelete (4990 ms) 2024-03-21T13:29:51.246 INFO:tasks.workunit.client.0.smithi097.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateWithCrushRule 2024-03-21T13:29:51.246 INFO:tasks.workunit.client.0.smithi097.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateWithCrushRule (5269 ms) 2024-03-21T13:29:51.246 INFO:tasks.workunit.client.0.smithi097.stdout: api_pool: [ RUN ] LibRadosPools.PoolGetBaseTier 2024-03-21T13:29:51.246 INFO:tasks.workunit.client.0.smithi097.stdout: api_pool: [ OK ] LibRadosPools.PoolGetBaseTier (7748 ms) 2024-03-21T13:29:51.246 INFO:tasks.workunit.client.0.smithi097.stdout: api_pool: [----------] 9 tests from LibRadosPools (51715 ms total) 2024-03-21T13:29:51.246 INFO:tasks.workunit.client.0.smithi097.stdout: api_pool: 2024-03-21T13:29:51.246 INFO:tasks.workunit.client.0.smithi097.stdout: api_pool: [----------] Global test environment tear-down 2024-03-21T13:29:51.247 INFO:tasks.workunit.client.0.smithi097.stdout: api_pool: [==========] 9 tests from 1 test suite ran. (51715 ms total) 2024-03-21T13:29:51.247 INFO:tasks.workunit.client.0.smithi097.stdout: api_pool: [ PASSED ] 9 tests. 2024-03-21T13:29:51.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:51 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:51.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:51 smithi097 ceph-mon[39416]: osdmap e116: 8 total, 8 up, 8 in 2024-03-21T13:29:51.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:51 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2524876381' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi097-79618-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:51.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:51 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi097-79618-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:51.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:51 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:51.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:51 smithi097 ceph-mon[30040]: osdmap e116: 8 total, 8 up, 8 in 2024-03-21T13:29:51.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:51 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2524876381' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi097-79618-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:51.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:51 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi097-79618-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:51.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:51 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:51.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:51 smithi144 ceph-mon[33633]: osdmap e116: 8 total, 8 up, 8 in 2024-03-21T13:29:51.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:51 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2524876381' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi097-79618-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:51.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:51 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi097-79618-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:52.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:29:51 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:29:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:29:52.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:52 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:29:52.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:52 smithi097 ceph-mon[39416]: pgmap v138: 556 pgs: 128 unknown, 428 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 258 KiB/s wr, 4 op/s 2024-03-21T13:29:52.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:52 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:29:52.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:52 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:52.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:52 smithi097 ceph-mon[39416]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:29:52.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:52 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi097-79618-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:52.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:52 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:29:52.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:52 smithi097 ceph-mon[39416]: osdmap e117: 8 total, 8 up, 8 in 2024-03-21T13:29:52.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:52 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-11"}]: dispatch 2024-03-21T13:29:52.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:52 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-11"}]: dispatch 2024-03-21T13:29:52.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:52 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3600525175' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi097-79636-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:52.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:52 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:29:52.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:52 smithi097 ceph-mon[30040]: pgmap v138: 556 pgs: 128 unknown, 428 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 258 KiB/s wr, 4 op/s 2024-03-21T13:29:52.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:52 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:29:52.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:52 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:52.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:52 smithi097 ceph-mon[30040]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:29:52.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:52 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi097-79618-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:52.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:52 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:29:52.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:52 smithi097 ceph-mon[30040]: osdmap e117: 8 total, 8 up, 8 in 2024-03-21T13:29:52.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:52 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-11"}]: dispatch 2024-03-21T13:29:52.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:52 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-11"}]: dispatch 2024-03-21T13:29:52.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:52 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3600525175' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi097-79636-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:52.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:52 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:29:52.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:52 smithi144 ceph-mon[33633]: pgmap v138: 556 pgs: 128 unknown, 428 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 258 KiB/s wr, 4 op/s 2024-03-21T13:29:52.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:52 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:29:52.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:52 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:52.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:52 smithi144 ceph-mon[33633]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:29:52.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:52 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi097-79618-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:52.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:52 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:29:52.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:52 smithi144 ceph-mon[33633]: osdmap e117: 8 total, 8 up, 8 in 2024-03-21T13:29:52.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:52 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-11"}]: dispatch 2024-03-21T13:29:52.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:52 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-11"}]: dispatch 2024-03-21T13:29:52.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:52 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3600525175' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi097-79636-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:53.773 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:29:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:29:54.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:53 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:54.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:53 smithi097 ceph-mon[39416]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:29:54.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:53 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-11"}]': finished 2024-03-21T13:29:54.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:53 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3600525175' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi097-79636-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:54.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:53 smithi097 ceph-mon[39416]: osdmap e118: 8 total, 8 up, 8 in 2024-03-21T13:29:54.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:53 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:54.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:53 smithi097 ceph-mon[30040]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:29:54.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:53 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-11"}]': finished 2024-03-21T13:29:54.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:53 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3600525175' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi097-79636-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:54.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:53 smithi097 ceph-mon[30040]: osdmap e118: 8 total, 8 up, 8 in 2024-03-21T13:29:54.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:53 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:54.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:53 smithi144 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:29:54.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:53 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-11"}]': finished 2024-03-21T13:29:54.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:53 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3600525175' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi097-79636-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:54.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:53 smithi144 ceph-mon[33633]: osdmap e118: 8 total, 8 up, 8 in 2024-03-21T13:29:55.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:54 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:29:55.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:54 smithi097 ceph-mon[39416]: pgmap v141: 524 pgs: 32 creating+peering, 492 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 3.5 KiB/s rd, 5.5 KiB/s wr, 11 op/s 2024-03-21T13:29:55.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:54 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:55.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:54 smithi097 ceph-mon[39416]: osdmap e119: 8 total, 8 up, 8 in 2024-03-21T13:29:55.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:54 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4215771965' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi097-80000-10"}]: dispatch 2024-03-21T13:29:55.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:54 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3673732186' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi097-79618-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:55.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:54 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi097-80000-10"}]: dispatch 2024-03-21T13:29:55.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:54 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi097-79618-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:55.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:54 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:29:55.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:54 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:29:55.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:54 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:29:55.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:54 smithi097 ceph-mon[30040]: pgmap v141: 524 pgs: 32 creating+peering, 492 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 3.5 KiB/s rd, 5.5 KiB/s wr, 11 op/s 2024-03-21T13:29:55.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:54 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:55.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:54 smithi097 ceph-mon[30040]: osdmap e119: 8 total, 8 up, 8 in 2024-03-21T13:29:55.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:54 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4215771965' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi097-80000-10"}]: dispatch 2024-03-21T13:29:55.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:54 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3673732186' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi097-79618-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:55.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:54 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi097-80000-10"}]: dispatch 2024-03-21T13:29:55.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:54 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi097-79618-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:55.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:54 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:29:55.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:54 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:29:55.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:54 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:29:55.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:54 smithi144 ceph-mon[33633]: pgmap v141: 524 pgs: 32 creating+peering, 492 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 3.5 KiB/s rd, 5.5 KiB/s wr, 11 op/s 2024-03-21T13:29:55.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:54 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:55.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:54 smithi144 ceph-mon[33633]: osdmap e119: 8 total, 8 up, 8 in 2024-03-21T13:29:55.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:54 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4215771965' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi097-80000-10"}]: dispatch 2024-03-21T13:29:55.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:54 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3673732186' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi097-79618-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:55.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:54 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi097-80000-10"}]: dispatch 2024-03-21T13:29:55.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:54 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi097-79618-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:55.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:54 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:29:55.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:54 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:29:55.242 INFO:tasks.workunit.client.0.smithi097.stdout: io: Running main() from gmock_main.cc 2024-03-21T13:29:55.242 INFO:tasks.workunit.client.0.smithi097.stdout: io: [==========] Running 14 tests from 1 test suite. 2024-03-21T13:29:55.242 INFO:tasks.workunit.client.0.smithi097.stdout: io: [----------] Global test environment set-up. 2024-03-21T13:29:55.242 INFO:tasks.workunit.client.0.smithi097.stdout: io: [----------] 14 tests from NeoRadosIo 2024-03-21T13:29:55.242 INFO:tasks.workunit.client.0.smithi097.stdout: io: [ RUN ] NeoRadosIo.Limits 2024-03-21T13:29:55.242 INFO:tasks.workunit.client.0.smithi097.stdout: io: [ OK ] NeoRadosIo.Limits (14913 ms) 2024-03-21T13:29:55.242 INFO:tasks.workunit.client.0.smithi097.stdout: io: [ RUN ] NeoRadosIo.SimpleWrite 2024-03-21T13:29:55.242 INFO:tasks.workunit.client.0.smithi097.stdout: io: [ OK ] NeoRadosIo.SimpleWrite (4418 ms) 2024-03-21T13:29:55.242 INFO:tasks.workunit.client.0.smithi097.stdout: io: [ RUN ] NeoRadosIo.ReadOp 2024-03-21T13:29:55.242 INFO:tasks.workunit.client.0.smithi097.stdout: io: [ OK ] NeoRadosIo.ReadOp (3141 ms) 2024-03-21T13:29:55.242 INFO:tasks.workunit.client.0.smithi097.stdout: io: [ RUN ] NeoRadosIo.SparseRead 2024-03-21T13:29:55.242 INFO:tasks.workunit.client.0.smithi097.stdout: io: [ OK ] NeoRadosIo.SparseRead (2804 ms) 2024-03-21T13:29:55.242 INFO:tasks.workunit.client.0.smithi097.stdout: io: [ RUN ] NeoRadosIo.RoundTrip 2024-03-21T13:29:55.242 INFO:tasks.workunit.client.0.smithi097.stdout: io: [ OK ] NeoRadosIo.RoundTrip (3094 ms) 2024-03-21T13:29:55.243 INFO:tasks.workunit.client.0.smithi097.stdout: io: [ RUN ] NeoRadosIo.ReadIntoBuufferlist 2024-03-21T13:29:55.243 INFO:tasks.workunit.client.0.smithi097.stdout: io: [ OK ] NeoRadosIo.ReadIntoBuufferlist (3040 ms) 2024-03-21T13:29:55.243 INFO:tasks.workunit.client.0.smithi097.stdout: io: [ RUN ] NeoRadosIo.OverlappingWriteRoundTrip 2024-03-21T13:29:55.243 INFO:tasks.workunit.client.0.smithi097.stdout: io: [ OK ] NeoRadosIo.OverlappingWriteRoundTrip (3031 ms) 2024-03-21T13:29:55.243 INFO:tasks.workunit.client.0.smithi097.stdout: io: [ RUN ] NeoRadosIo.WriteFullRoundTrip 2024-03-21T13:29:55.243 INFO:tasks.workunit.client.0.smithi097.stdout: io: [ OK ] NeoRadosIo.WriteFullRoundTrip (2996 ms) 2024-03-21T13:29:55.243 INFO:tasks.workunit.client.0.smithi097.stdout: io: [ RUN ] NeoRadosIo.AppendRoundTrip 2024-03-21T13:29:55.243 INFO:tasks.workunit.client.0.smithi097.stdout: io: [ OK ] NeoRadosIo.AppendRoundTrip (3031 ms) 2024-03-21T13:29:55.243 INFO:tasks.workunit.client.0.smithi097.stdout: io: [ RUN ] NeoRadosIo.Trunc 2024-03-21T13:29:55.243 INFO:tasks.workunit.client.0.smithi097.stdout: io: [ OK ] NeoRadosIo.Trunc (3231 ms) 2024-03-21T13:29:55.243 INFO:tasks.workunit.client.0.smithi097.stdout: io: [ RUN ] NeoRadosIo.Remove 2024-03-21T13:29:55.243 INFO:tasks.workunit.client.0.smithi097.stdout: io: [ OK ] NeoRadosIo.Remove (2778 ms) 2024-03-21T13:29:55.243 INFO:tasks.workunit.client.0.smithi097.stdout: io: [ RUN ] NeoRadosIo.XattrsRoundTrip 2024-03-21T13:29:55.243 INFO:tasks.workunit.client.0.smithi097.stdout: io: [ OK ] NeoRadosIo.XattrsRoundTrip (2974 ms) 2024-03-21T13:29:55.243 INFO:tasks.workunit.client.0.smithi097.stdout: io: [ RUN ] NeoRadosIo.RmXattr 2024-03-21T13:29:55.244 INFO:tasks.workunit.client.0.smithi097.stdout: io: [ OK ] NeoRadosIo.RmXattr (3003 ms) 2024-03-21T13:29:55.244 INFO:tasks.workunit.client.0.smithi097.stdout: io: [ RUN ] NeoRadosIo.GetXattrs 2024-03-21T13:29:55.244 INFO:tasks.workunit.client.0.smithi097.stdout: io: [ OK ] NeoRadosIo.GetXattrs (2994 ms) 2024-03-21T13:29:55.244 INFO:tasks.workunit.client.0.smithi097.stdout: io: [----------] 14 tests from NeoRadosIo (55448 ms total) 2024-03-21T13:29:55.244 INFO:tasks.workunit.client.0.smithi097.stdout: io: 2024-03-21T13:29:55.244 INFO:tasks.workunit.client.0.smithi097.stdout: io: [----------] Global test environment tear-down 2024-03-21T13:29:55.244 INFO:tasks.workunit.client.0.smithi097.stdout: io: [==========] 14 tests from 1 test suite ran. (55448 ms total) 2024-03-21T13:29:55.244 INFO:tasks.workunit.client.0.smithi097.stdout: io: [ PASSED ] 14 tests. 2024-03-21T13:29:56.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:55 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:56.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:55 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi097-80000-10"}]': finished 2024-03-21T13:29:56.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:55 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi097-79618-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:56.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:55 smithi097 ceph-mon[39416]: osdmap e120: 8 total, 8 up, 8 in 2024-03-21T13:29:56.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:55 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4215771965' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi097-80000-10"}]: dispatch 2024-03-21T13:29:56.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:55 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:56.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:55 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2836093171' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi097-79636-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:56.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:55 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi097-80000-10"}]: dispatch 2024-03-21T13:29:56.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:55 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:56.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:55 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi097-79636-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:56.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:55 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:56.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:55 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi097-80000-10"}]': finished 2024-03-21T13:29:56.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:55 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi097-79618-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:56.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:55 smithi097 ceph-mon[30040]: osdmap e120: 8 total, 8 up, 8 in 2024-03-21T13:29:56.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:55 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4215771965' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi097-80000-10"}]: dispatch 2024-03-21T13:29:56.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:55 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:56.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:55 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2836093171' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi097-79636-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:56.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:55 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi097-80000-10"}]: dispatch 2024-03-21T13:29:56.056 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:55 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:56.056 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:55 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi097-79636-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:56.073 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:55 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:56.073 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:55 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi097-80000-10"}]': finished 2024-03-21T13:29:56.074 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:55 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi097-79618-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:56.074 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:55 smithi144 ceph-mon[33633]: osdmap e120: 8 total, 8 up, 8 in 2024-03-21T13:29:56.074 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:55 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4215771965' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi097-80000-10"}]: dispatch 2024-03-21T13:29:56.074 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:55 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:56.074 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:55 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2836093171' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi097-79636-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:56.074 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:55 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi097-80000-10"}]: dispatch 2024-03-21T13:29:56.074 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:55 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:56.074 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:55 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi097-79636-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:56.254 INFO:tasks.workunit.client.0.smithi097.stdout: read_operations: Running main() from gmock_main.cc 2024-03-21T13:29:56.254 INFO:tasks.workunit.client.0.smithi097.stdout: read_operations: [==========] Running 14 tests from 1 test suite. 2024-03-21T13:29:56.254 INFO:tasks.workunit.client.0.smithi097.stdout: read_operations: [----------] Global test environment set-up. 2024-03-21T13:29:56.254 INFO:tasks.workunit.client.0.smithi097.stdout: read_operations: [----------] 14 tests from NeoRadosReadOps 2024-03-21T13:29:56.254 INFO:tasks.workunit.client.0.smithi097.stdout: read_operations: [ RUN ] NeoRadosReadOps.SetOpFlags 2024-03-21T13:29:56.254 INFO:tasks.workunit.client.0.smithi097.stdout: read_operations: [ OK ] NeoRadosReadOps.SetOpFlags (14846 ms) 2024-03-21T13:29:56.254 INFO:tasks.workunit.client.0.smithi097.stdout: read_operations: [ RUN ] NeoRadosReadOps.AssertExists 2024-03-21T13:29:56.255 INFO:tasks.workunit.client.0.smithi097.stdout: read_operations: [ OK ] NeoRadosReadOps.AssertExists (4418 ms) 2024-03-21T13:29:56.255 INFO:tasks.workunit.client.0.smithi097.stdout: read_operations: [ RUN ] NeoRadosReadOps.AssertVersion 2024-03-21T13:29:56.255 INFO:tasks.workunit.client.0.smithi097.stdout: read_operations: [ OK ] NeoRadosReadOps.AssertVersion (3139 ms) 2024-03-21T13:29:56.255 INFO:tasks.workunit.client.0.smithi097.stdout: read_operations: [ RUN ] NeoRadosReadOps.CmpXattr 2024-03-21T13:29:56.255 INFO:tasks.workunit.client.0.smithi097.stdout: read_operations: [ OK ] NeoRadosReadOps.CmpXattr (3810 ms) 2024-03-21T13:29:56.255 INFO:tasks.workunit.client.0.smithi097.stdout: read_operations: [ RUN ] NeoRadosReadOps.Read 2024-03-21T13:29:56.255 INFO:tasks.workunit.client.0.smithi097.stdout: read_operations: [ OK ] NeoRadosReadOps.Read (3095 ms) 2024-03-21T13:29:56.255 INFO:tasks.workunit.client.0.smithi097.stdout: read_operations: [ RUN ] NeoRadosReadOps.Checksum 2024-03-21T13:29:56.255 INFO:tasks.workunit.client.0.smithi097.stdout: read_operations: [ OK ] NeoRadosReadOps.Checksum (3058 ms) 2024-03-21T13:29:56.255 INFO:tasks.workunit.client.0.smithi097.stdout: read_operations: [ RUN ] NeoRadosReadOps.RWOrderedRead 2024-03-21T13:29:56.255 INFO:tasks.workunit.client.0.smithi097.stdout: read_operations: [ OK ] NeoRadosReadOps.RWOrderedRead (3011 ms) 2024-03-21T13:29:56.255 INFO:tasks.workunit.client.0.smithi097.stdout: read_operations: [ RUN ] NeoRadosReadOps.ShortRead 2024-03-21T13:29:56.255 INFO:tasks.workunit.client.0.smithi097.stdout: read_operations: [ OK ] NeoRadosReadOps.ShortRead (2995 ms) 2024-03-21T13:29:56.255 INFO:tasks.workunit.client.0.smithi097.stdout: read_operations: [ RUN ] NeoRadosReadOps.Exec 2024-03-21T13:29:56.255 INFO:tasks.workunit.client.0.smithi097.stdout: read_operations: [ OK ] NeoRadosReadOps.Exec (3038 ms) 2024-03-21T13:29:56.256 INFO:tasks.workunit.client.0.smithi097.stdout: read_operations: [ RUN ] NeoRadosReadOps.Stat 2024-03-21T13:29:56.256 INFO:tasks.workunit.client.0.smithi097.stdout: read_operations: [ OK ] NeoRadosReadOps.Stat (2975 ms) 2024-03-21T13:29:56.256 INFO:tasks.workunit.client.0.smithi097.stdout: read_operations: [ RUN ] NeoRadosReadOps.Omap 2024-03-21T13:29:56.256 INFO:tasks.workunit.client.0.smithi097.stdout: read_operations: [ OK ] NeoRadosReadOps.Omap (2991 ms) 2024-03-21T13:29:56.256 INFO:tasks.workunit.client.0.smithi097.stdout: read_operations: [ RUN ] NeoRadosReadOps.OmapNuls 2024-03-21T13:29:56.256 INFO:tasks.workunit.client.0.smithi097.stdout: read_operations: [ OK ] NeoRadosReadOps.OmapNuls (3008 ms) 2024-03-21T13:29:56.256 INFO:tasks.workunit.client.0.smithi097.stdout: read_operations: [ RUN ] NeoRadosReadOps.GetXattrs 2024-03-21T13:29:56.256 INFO:tasks.workunit.client.0.smithi097.stdout: read_operations: [ OK ] NeoRadosReadOps.GetXattrs (2996 ms) 2024-03-21T13:29:56.256 INFO:tasks.workunit.client.0.smithi097.stdout: read_operations: [ RUN ] NeoRadosReadOps.CmpExt 2024-03-21T13:29:56.256 INFO:tasks.workunit.client.0.smithi097.stdout: read_operations: [ OK ] NeoRadosReadOps.CmpExt (3012 ms) 2024-03-21T13:29:56.256 INFO:tasks.workunit.client.0.smithi097.stdout: read_operations: [----------] 14 tests from NeoRadosReadOps (56392 ms total) 2024-03-21T13:29:56.256 INFO:tasks.workunit.client.0.smithi097.stdout: read_operations: 2024-03-21T13:29:56.256 INFO:tasks.workunit.client.0.smithi097.stdout: read_operations: [----------] Global test environment tear-down 2024-03-21T13:29:56.256 INFO:tasks.workunit.client.0.smithi097.stdout: read_operations: [==========] 14 tests from 1 test suite ran. (56392 ms total) 2024-03-21T13:29:56.256 INFO:tasks.workunit.client.0.smithi097.stdout: read_operations: [ PASSED ] 14 tests. 2024-03-21T13:29:57.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:56 smithi097 ceph-mon[39416]: pgmap v144: 548 pgs: 128 unknown, 420 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.0 KiB/s wr, 6 op/s 2024-03-21T13:29:57.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:56 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:57.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:56 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi097-80000-10"}]': finished 2024-03-21T13:29:57.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:56 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:57.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:56 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi097-79636-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:57.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:56 smithi097 ceph-mon[39416]: osdmap e121: 8 total, 8 up, 8 in 2024-03-21T13:29:57.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:56 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1231934815' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi097-80000-15"}]: dispatch 2024-03-21T13:29:57.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:56 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi097-80000-15"}]: dispatch 2024-03-21T13:29:57.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:56 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1231934815' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi097-80000-15"}]: dispatch 2024-03-21T13:29:57.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:56 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi097-80000-15"}]: dispatch 2024-03-21T13:29:57.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:56 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1231934815' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi097-80000-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:57.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:56 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi097-80000-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:57.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:56 smithi097 ceph-mon[30040]: pgmap v144: 548 pgs: 128 unknown, 420 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.0 KiB/s wr, 6 op/s 2024-03-21T13:29:57.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:56 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:57.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:56 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi097-80000-10"}]': finished 2024-03-21T13:29:57.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:56 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:57.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:56 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi097-79636-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:57.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:56 smithi097 ceph-mon[30040]: osdmap e121: 8 total, 8 up, 8 in 2024-03-21T13:29:57.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:56 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1231934815' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi097-80000-15"}]: dispatch 2024-03-21T13:29:57.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:56 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi097-80000-15"}]: dispatch 2024-03-21T13:29:57.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:56 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1231934815' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi097-80000-15"}]: dispatch 2024-03-21T13:29:57.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:56 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi097-80000-15"}]: dispatch 2024-03-21T13:29:57.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:56 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1231934815' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi097-80000-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:57.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:56 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi097-80000-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:57.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:56 smithi144 ceph-mon[33633]: pgmap v144: 548 pgs: 128 unknown, 420 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.0 KiB/s wr, 6 op/s 2024-03-21T13:29:57.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:56 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:57.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:56 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi097-80000-10"}]': finished 2024-03-21T13:29:57.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:56 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:57.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:56 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi097-79636-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:57.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:56 smithi144 ceph-mon[33633]: osdmap e121: 8 total, 8 up, 8 in 2024-03-21T13:29:57.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:56 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1231934815' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi097-80000-15"}]: dispatch 2024-03-21T13:29:57.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:56 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi097-80000-15"}]: dispatch 2024-03-21T13:29:57.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:56 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1231934815' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi097-80000-15"}]: dispatch 2024-03-21T13:29:57.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:56 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi097-80000-15"}]: dispatch 2024-03-21T13:29:57.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:56 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1231934815' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi097-80000-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:57.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:56 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi097-80000-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:29:58.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:57 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:58.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:57 smithi097 ceph-mon[39416]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:29:58.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:57 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi097-80000-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:29:58.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:57 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1231934815' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi097-80000-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi097-80000-15"}]: dispatch 2024-03-21T13:29:58.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:57 smithi097 ceph-mon[39416]: osdmap e122: 8 total, 8 up, 8 in 2024-03-21T13:29:58.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:57 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/95100902' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi097-79618-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:58.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:57 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi097-80000-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi097-80000-15"}]: dispatch 2024-03-21T13:29:58.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:57 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:58.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:57 smithi097 ceph-mon[30040]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:29:58.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:57 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi097-80000-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:29:58.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:57 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1231934815' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi097-80000-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi097-80000-15"}]: dispatch 2024-03-21T13:29:58.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:57 smithi097 ceph-mon[30040]: osdmap e122: 8 total, 8 up, 8 in 2024-03-21T13:29:58.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:57 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/95100902' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi097-79618-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:58.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:57 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi097-80000-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi097-80000-15"}]: dispatch 2024-03-21T13:29:58.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:57 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:58.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:57 smithi144 ceph-mon[33633]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:29:58.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:57 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi097-80000-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:29:58.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:57 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1231934815' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi097-80000-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi097-80000-15"}]: dispatch 2024-03-21T13:29:58.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:57 smithi144 ceph-mon[33633]: osdmap e122: 8 total, 8 up, 8 in 2024-03-21T13:29:58.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:57 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/95100902' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi097-79618-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:58.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:57 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi097-80000-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi097-80000-15"}]: dispatch 2024-03-21T13:29:59.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:58 smithi097 ceph-mon[39416]: pgmap v147: 452 pgs: 64 unknown, 388 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-03-21T13:29:59.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:58 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:59.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:58 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/95100902' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi097-79618-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:59.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:58 smithi097 ceph-mon[39416]: osdmap e123: 8 total, 8 up, 8 in 2024-03-21T13:29:59.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:58 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1693131542' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi097-79636-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:59.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:58 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi097-79636-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:59.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:58 smithi097 ceph-mon[30040]: pgmap v147: 452 pgs: 64 unknown, 388 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-03-21T13:29:59.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:58 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:59.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:58 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/95100902' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi097-79618-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:59.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:58 smithi097 ceph-mon[30040]: osdmap e123: 8 total, 8 up, 8 in 2024-03-21T13:29:59.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:58 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1693131542' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi097-79636-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:59.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:58 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi097-79636-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:59.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:58 smithi144 ceph-mon[33633]: pgmap v147: 452 pgs: 64 unknown, 388 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-03-21T13:29:59.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:58 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:29:59.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:58 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/95100902' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi097-79618-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:29:59.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:58 smithi144 ceph-mon[33633]: osdmap e123: 8 total, 8 up, 8 in 2024-03-21T13:29:59.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:58 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1693131542' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi097-79636-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:29:59.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:58 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi097-79636-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:00.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:59 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:00.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:59 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi097-80000-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi097-80000-15"}]': finished 2024-03-21T13:30:00.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:59 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi097-79636-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:00.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:29:59 smithi097 ceph-mon[39416]: osdmap e124: 8 total, 8 up, 8 in 2024-03-21T13:30:00.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:59 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:00.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:59 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi097-80000-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi097-80000-15"}]': finished 2024-03-21T13:30:00.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:59 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi097-79636-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:00.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:29:59 smithi097 ceph-mon[30040]: osdmap e124: 8 total, 8 up, 8 in 2024-03-21T13:30:00.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:59 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:00.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:59 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi097-80000-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi097-80000-15"}]': finished 2024-03-21T13:30:00.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:59 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi097-79636-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:00.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:29:59 smithi144 ceph-mon[33633]: osdmap e124: 8 total, 8 up, 8 in 2024-03-21T13:30:01.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:00 smithi097 ceph-mon[30040]: pgmap v150: 492 pgs: 34 creating+peering, 38 unknown, 420 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 46 KiB/s wr, 46 op/s 2024-03-21T13:30:01.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:00 smithi097 ceph-mon[30040]: Health detail: HEALTH_WARN 1 stray daemon(s) not managed by cephadm; 5 pool(s) do not have an application enabled 2024-03-21T13:30:01.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:00 smithi097 ceph-mon[30040]: [WRN] CEPHADM_STRAY_DAEMON: 1 stray daemon(s) not managed by cephadm 2024-03-21T13:30:01.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:00 smithi097 ceph-mon[30040]: stray daemon laundry.close-test-pid80277 on host smithi097 not managed by cephadm 2024-03-21T13:30:01.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:00 smithi097 ceph-mon[30040]: [WRN] POOL_APP_NOT_ENABLED: 5 pool(s) do not have an application enabled 2024-03-21T13:30:01.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:00 smithi097 ceph-mon[30040]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-03-21T13:30:01.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:00 smithi097 ceph-mon[30040]: application not enabled on pool 'poolrace.1804289383' 2024-03-21T13:30:01.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:00 smithi097 ceph-mon[30040]: application not enabled on pool 'AssertExistssmithi097-80928-1' 2024-03-21T13:30:01.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:00 smithi097 ceph-mon[30040]: application not enabled on pool 'poolrace2.846930886' 2024-03-21T13:30:01.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:00 smithi097 ceph-mon[30040]: application not enabled on pool 'Flush_smithi097-79618-15' 2024-03-21T13:30:01.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:00 smithi097 ceph-mon[30040]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-03-21T13:30:01.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:00 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:01.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:00 smithi097 ceph-mon[30040]: osdmap e125: 8 total, 8 up, 8 in 2024-03-21T13:30:01.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:00 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1733518257' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi097-79618-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:01.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:00 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi097-79618-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:01.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:00 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:30:01.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:00 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:30:01.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:00 smithi097 ceph-mon[39416]: pgmap v150: 492 pgs: 34 creating+peering, 38 unknown, 420 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 46 KiB/s wr, 46 op/s 2024-03-21T13:30:01.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:00 smithi097 ceph-mon[39416]: Health detail: HEALTH_WARN 1 stray daemon(s) not managed by cephadm; 5 pool(s) do not have an application enabled 2024-03-21T13:30:01.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:00 smithi097 ceph-mon[39416]: [WRN] CEPHADM_STRAY_DAEMON: 1 stray daemon(s) not managed by cephadm 2024-03-21T13:30:01.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:00 smithi097 ceph-mon[39416]: stray daemon laundry.close-test-pid80277 on host smithi097 not managed by cephadm 2024-03-21T13:30:01.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:00 smithi097 ceph-mon[39416]: [WRN] POOL_APP_NOT_ENABLED: 5 pool(s) do not have an application enabled 2024-03-21T13:30:01.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:00 smithi097 ceph-mon[39416]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-03-21T13:30:01.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:00 smithi097 ceph-mon[39416]: application not enabled on pool 'poolrace.1804289383' 2024-03-21T13:30:01.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:00 smithi097 ceph-mon[39416]: application not enabled on pool 'AssertExistssmithi097-80928-1' 2024-03-21T13:30:01.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:00 smithi097 ceph-mon[39416]: application not enabled on pool 'poolrace2.846930886' 2024-03-21T13:30:01.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:00 smithi097 ceph-mon[39416]: application not enabled on pool 'Flush_smithi097-79618-15' 2024-03-21T13:30:01.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:00 smithi097 ceph-mon[39416]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-03-21T13:30:01.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:00 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:01.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:00 smithi097 ceph-mon[39416]: osdmap e125: 8 total, 8 up, 8 in 2024-03-21T13:30:01.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:00 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1733518257' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi097-79618-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:01.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:00 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi097-79618-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:01.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:00 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:30:01.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:00 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:30:01.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:00 smithi144 ceph-mon[33633]: pgmap v150: 492 pgs: 34 creating+peering, 38 unknown, 420 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 46 KiB/s wr, 46 op/s 2024-03-21T13:30:01.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:00 smithi144 ceph-mon[33633]: Health detail: HEALTH_WARN 1 stray daemon(s) not managed by cephadm; 5 pool(s) do not have an application enabled 2024-03-21T13:30:01.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:00 smithi144 ceph-mon[33633]: [WRN] CEPHADM_STRAY_DAEMON: 1 stray daemon(s) not managed by cephadm 2024-03-21T13:30:01.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:00 smithi144 ceph-mon[33633]: stray daemon laundry.close-test-pid80277 on host smithi097 not managed by cephadm 2024-03-21T13:30:01.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:00 smithi144 ceph-mon[33633]: [WRN] POOL_APP_NOT_ENABLED: 5 pool(s) do not have an application enabled 2024-03-21T13:30:01.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:00 smithi144 ceph-mon[33633]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-03-21T13:30:01.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:00 smithi144 ceph-mon[33633]: application not enabled on pool 'poolrace.1804289383' 2024-03-21T13:30:01.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:00 smithi144 ceph-mon[33633]: application not enabled on pool 'AssertExistssmithi097-80928-1' 2024-03-21T13:30:01.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:00 smithi144 ceph-mon[33633]: application not enabled on pool 'poolrace2.846930886' 2024-03-21T13:30:01.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:00 smithi144 ceph-mon[33633]: application not enabled on pool 'Flush_smithi097-79618-15' 2024-03-21T13:30:01.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:00 smithi144 ceph-mon[33633]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-03-21T13:30:01.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:00 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:01.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:00 smithi144 ceph-mon[33633]: osdmap e125: 8 total, 8 up, 8 in 2024-03-21T13:30:01.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:00 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1733518257' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi097-79618-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:01.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:00 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi097-79618-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:01.123 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:00 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:30:01.123 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:00 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:30:01.857 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: [==========] Running 12 tests from 4 test suites. 2024-03-21T13:30:01.858 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: [----------] Global test environment set-up. 2024-03-21T13:30:01.858 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion 2024-03-21T13:30:01.858 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: [ RUN ] LibRadosMiscVersion.Version 2024-03-21T13:30:01.858 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: [ OK ] LibRadosMiscVersion.Version (0 ms) 2024-03-21T13:30:01.858 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-03-21T13:30:01.858 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: 2024-03-21T13:30:01.858 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure 2024-03-21T13:30:01.858 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectFailure 2024-03-21T13:30:01.858 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: unable to get monitor info from DNS SRV with service name: ceph-mon 2024-03-21T13:30:01.858 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: 2024-03-21T13:28:59.624+0000 7f738777d780 -1 failed for service _ceph-mon._tcp 2024-03-21T13:30:01.858 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: 2024-03-21T13:28:59.624+0000 7f738777d780 -1 monclient: get_monmap_and_config cannot identify monitors to contact 2024-03-21T13:30:01.858 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectFailure (84 ms) 2024-03-21T13:30:01.858 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectTimeout 2024-03-21T13:30:01.859 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectTimeout (2009 ms) 2024-03-21T13:30:01.859 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure (2093 ms total) 2024-03-21T13:30:01.859 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: 2024-03-21T13:30:01.859 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: [----------] 1 test from LibRadosMiscPool 2024-03-21T13:30:01.859 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: [ RUN ] LibRadosMiscPool.PoolCreationRace 2024-03-21T13:30:01.859 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a26b600 2024-03-21T13:30:01.859 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5af790 2024-03-21T13:30:01.859 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b01c0 2024-03-21T13:30:01.859 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b07e0 2024-03-21T13:30:01.859 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b06b0 2024-03-21T13:30:01.859 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b0d30 2024-03-21T13:30:01.859 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b11e0 2024-03-21T13:30:01.859 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b1710 2024-03-21T13:30:01.859 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b1c20 2024-03-21T13:30:01.859 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b1dc0 2024-03-21T13:30:01.859 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b22e0 2024-03-21T13:30:01.860 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b2480 2024-03-21T13:30:01.860 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b1820 2024-03-21T13:30:01.860 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b1950 2024-03-21T13:30:01.860 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b1a60 2024-03-21T13:30:01.860 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b2590 2024-03-21T13:30:01.860 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b26a0 2024-03-21T13:30:01.860 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b2cd0 2024-03-21T13:30:01.860 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b2820 2024-03-21T13:30:01.860 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b2930 2024-03-21T13:30:01.860 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b2a40 2024-03-21T13:30:01.860 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b2b50 2024-03-21T13:30:01.860 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b3290 2024-03-21T13:30:01.860 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b2de0 2024-03-21T13:30:01.860 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b1ef0 2024-03-21T13:30:01.860 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b2020 2024-03-21T13:30:01.860 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b2150 2024-03-21T13:30:01.861 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b3870 2024-03-21T13:30:01.861 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b3760 2024-03-21T13:30:01.861 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b2f30 2024-03-21T13:30:01.861 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b3080 2024-03-21T13:30:01.861 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b39a0 2024-03-21T13:30:01.861 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b3af0 2024-03-21T13:30:01.861 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b40b0 2024-03-21T13:30:01.861 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b3c00 2024-03-21T13:30:01.861 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b3d30 2024-03-21T13:30:01.861 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b3e40 2024-03-21T13:30:01.861 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b3f50 2024-03-21T13:30:01.861 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b4670 2024-03-21T13:30:01.861 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b41c0 2024-03-21T13:30:01.861 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b33a0 2024-03-21T13:30:01.861 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b34d0 2024-03-21T13:30:01.861 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b3600 2024-03-21T13:30:01.861 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b4c50 2024-03-21T13:30:01.862 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b47a0 2024-03-21T13:30:01.862 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b48d0 2024-03-21T13:30:01.862 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b4a20 2024-03-21T13:30:01.862 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b4d60 2024-03-21T13:30:01.862 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b5210 2024-03-21T13:30:01.862 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b5320 2024-03-21T13:30:01.862 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b5920 2024-03-21T13:30:02.053 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:30:01 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:30:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:30:02.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:01 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:02.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:01 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi097-79618-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:02.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:01 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-13", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:30:02.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:01 smithi097 ceph-mon[39416]: osdmap e126: 8 total, 8 up, 8 in 2024-03-21T13:30:02.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:01 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-13"}]: dispatch 2024-03-21T13:30:02.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:01 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2010636348' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi097-79636-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:02.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:01 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-13"}]: dispatch 2024-03-21T13:30:02.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:01 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi097-79636-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:02.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:01 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:02.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:01 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi097-79618-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:02.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:01 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-13", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:30:02.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:01 smithi097 ceph-mon[30040]: osdmap e126: 8 total, 8 up, 8 in 2024-03-21T13:30:02.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:01 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-13"}]: dispatch 2024-03-21T13:30:02.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:01 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2010636348' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi097-79636-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:02.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:01 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-13"}]: dispatch 2024-03-21T13:30:02.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:01 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi097-79636-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:02.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:01 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:02.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:01 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi097-79618-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:02.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:01 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-13", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:30:02.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:01 smithi144 ceph-mon[33633]: osdmap e126: 8 total, 8 up, 8 in 2024-03-21T13:30:02.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:01 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-13"}]: dispatch 2024-03-21T13:30:02.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:01 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2010636348' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi097-79636-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:02.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:01 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-13"}]: dispatch 2024-03-21T13:30:02.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:01 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi097-79636-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:03.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:02 smithi097 ceph-mon[30040]: pgmap v153: 524 pgs: 13 creating+peering, 91 unknown, 420 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 50 KiB/s wr, 51 op/s 2024-03-21T13:30:03.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:02 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:03.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:02 smithi097 ceph-mon[30040]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:30:03.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:02 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-13"}]': finished 2024-03-21T13:30:03.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:02 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi097-79636-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:03.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:02 smithi097 ceph-mon[30040]: osdmap e127: 8 total, 8 up, 8 in 2024-03-21T13:30:03.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:02 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-13", "mode": "writeback"}]: dispatch 2024-03-21T13:30:03.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:02 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-13", "mode": "writeback"}]: dispatch 2024-03-21T13:30:03.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:02 smithi097 ceph-mon[39416]: pgmap v153: 524 pgs: 13 creating+peering, 91 unknown, 420 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 50 KiB/s wr, 51 op/s 2024-03-21T13:30:03.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:02 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:03.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:02 smithi097 ceph-mon[39416]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:30:03.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:02 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-13"}]': finished 2024-03-21T13:30:03.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:02 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi097-79636-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:03.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:02 smithi097 ceph-mon[39416]: osdmap e127: 8 total, 8 up, 8 in 2024-03-21T13:30:03.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:02 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-13", "mode": "writeback"}]: dispatch 2024-03-21T13:30:03.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:02 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-13", "mode": "writeback"}]: dispatch 2024-03-21T13:30:03.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:02 smithi144 ceph-mon[33633]: pgmap v153: 524 pgs: 13 creating+peering, 91 unknown, 420 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 50 KiB/s wr, 51 op/s 2024-03-21T13:30:03.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:02 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:03.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:02 smithi144 ceph-mon[33633]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:30:03.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:02 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-13"}]': finished 2024-03-21T13:30:03.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:02 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi097-79636-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:03.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:02 smithi144 ceph-mon[33633]: osdmap e127: 8 total, 8 up, 8 in 2024-03-21T13:30:03.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:02 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-13", "mode": "writeback"}]: dispatch 2024-03-21T13:30:03.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:02 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-13", "mode": "writeback"}]: dispatch 2024-03-21T13:30:03.817 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:30:03 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:30:04.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:03 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:04.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:03 smithi144 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:30:04.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:03 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-13", "mode": "writeback"}]': finished 2024-03-21T13:30:04.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:03 smithi144 ceph-mon[33633]: osdmap e128: 8 total, 8 up, 8 in 2024-03-21T13:30:04.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:03 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/872425909' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi097-79618-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:04.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:03 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:04.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:03 smithi097 ceph-mon[30040]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:30:04.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:03 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-13", "mode": "writeback"}]': finished 2024-03-21T13:30:04.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:03 smithi097 ceph-mon[30040]: osdmap e128: 8 total, 8 up, 8 in 2024-03-21T13:30:04.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:03 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/872425909' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi097-79618-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:04.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:03 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:04.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:03 smithi097 ceph-mon[39416]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:30:04.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:03 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-13", "mode": "writeback"}]': finished 2024-03-21T13:30:04.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:03 smithi097 ceph-mon[39416]: osdmap e128: 8 total, 8 up, 8 in 2024-03-21T13:30:04.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:03 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/872425909' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi097-79618-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:05.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:04 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:30:05.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:04 smithi144 ceph-mon[33633]: pgmap v156: 460 pgs: 32 creating+peering, 428 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.5 KiB/s rd, 79 KiB/s wr, 78 op/s 2024-03-21T13:30:05.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:04 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "167.0"}]: dispatch 2024-03-21T13:30:05.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:04 smithi144 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "167.0"}]: dispatch 2024-03-21T13:30:05.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:04 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "167.1"}]: dispatch 2024-03-21T13:30:05.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:04 smithi144 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "167.1"}]: dispatch 2024-03-21T13:30:05.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:04 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "167.2"}]: dispatch 2024-03-21T13:30:05.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:04 smithi144 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "167.2"}]: dispatch 2024-03-21T13:30:05.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:04 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "167.3"}]: dispatch 2024-03-21T13:30:05.123 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:04 smithi144 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "167.3"}]: dispatch 2024-03-21T13:30:05.123 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:04 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "167.4"}]: dispatch 2024-03-21T13:30:05.123 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:04 smithi144 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "167.4"}]: dispatch 2024-03-21T13:30:05.123 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:04 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "167.5"}]: dispatch 2024-03-21T13:30:05.123 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:04 smithi144 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "167.5"}]: dispatch 2024-03-21T13:30:05.123 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:04 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "167.6"}]: dispatch 2024-03-21T13:30:05.123 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:04 smithi144 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "167.6"}]: dispatch 2024-03-21T13:30:05.123 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:04 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "167.7"}]: dispatch 2024-03-21T13:30:05.123 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:04 smithi144 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "167.7"}]: dispatch 2024-03-21T13:30:05.123 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:04 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "167.8"}]: dispatch 2024-03-21T13:30:05.123 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:04 smithi144 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "167.8"}]: dispatch 2024-03-21T13:30:05.123 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:04 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "167.9"}]: dispatch 2024-03-21T13:30:05.124 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:04 smithi144 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "167.9"}]: dispatch 2024-03-21T13:30:05.124 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:04 smithi144 ceph-mon[33633]: 167.4 scrub starts 2024-03-21T13:30:05.124 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:04 smithi144 ceph-mon[33633]: 167.4 scrub ok 2024-03-21T13:30:05.124 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:04 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:05.124 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:04 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/872425909' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi097-79618-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:05.124 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:04 smithi144 ceph-mon[33633]: osdmap e129: 8 total, 8 up, 8 in 2024-03-21T13:30:05.124 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:04 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2916535762' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi097-79636-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:05.124 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:04 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi097-79636-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:05.124 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:04 smithi144 ceph-mon[33633]: 167.0 scrub starts 2024-03-21T13:30:05.124 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:04 smithi144 ceph-mon[33633]: 167.0 scrub ok 2024-03-21T13:30:05.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:30:05.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[39416]: pgmap v156: 460 pgs: 32 creating+peering, 428 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.5 KiB/s rd, 79 KiB/s wr, 78 op/s 2024-03-21T13:30:05.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "167.0"}]: dispatch 2024-03-21T13:30:05.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[39416]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "167.0"}]: dispatch 2024-03-21T13:30:05.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "167.1"}]: dispatch 2024-03-21T13:30:05.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[39416]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "167.1"}]: dispatch 2024-03-21T13:30:05.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "167.2"}]: dispatch 2024-03-21T13:30:05.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[39416]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "167.2"}]: dispatch 2024-03-21T13:30:05.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "167.3"}]: dispatch 2024-03-21T13:30:05.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[39416]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "167.3"}]: dispatch 2024-03-21T13:30:05.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "167.4"}]: dispatch 2024-03-21T13:30:05.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[39416]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "167.4"}]: dispatch 2024-03-21T13:30:05.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "167.5"}]: dispatch 2024-03-21T13:30:05.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[39416]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "167.5"}]: dispatch 2024-03-21T13:30:05.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "167.6"}]: dispatch 2024-03-21T13:30:05.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[39416]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "167.6"}]: dispatch 2024-03-21T13:30:05.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "167.7"}]: dispatch 2024-03-21T13:30:05.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[39416]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "167.7"}]: dispatch 2024-03-21T13:30:05.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "167.8"}]: dispatch 2024-03-21T13:30:05.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[39416]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "167.8"}]: dispatch 2024-03-21T13:30:05.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "167.9"}]: dispatch 2024-03-21T13:30:05.306 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[39416]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "167.9"}]: dispatch 2024-03-21T13:30:05.306 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[39416]: 167.4 scrub starts 2024-03-21T13:30:05.306 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[39416]: 167.4 scrub ok 2024-03-21T13:30:05.306 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:05.306 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/872425909' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi097-79618-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:05.306 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[39416]: osdmap e129: 8 total, 8 up, 8 in 2024-03-21T13:30:05.306 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2916535762' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi097-79636-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:05.306 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi097-79636-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:05.306 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[39416]: 167.0 scrub starts 2024-03-21T13:30:05.306 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[39416]: 167.0 scrub ok 2024-03-21T13:30:05.307 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:30:05.307 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[30040]: pgmap v156: 460 pgs: 32 creating+peering, 428 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.5 KiB/s rd, 79 KiB/s wr, 78 op/s 2024-03-21T13:30:05.307 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "167.0"}]: dispatch 2024-03-21T13:30:05.307 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[30040]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "167.0"}]: dispatch 2024-03-21T13:30:05.308 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "167.1"}]: dispatch 2024-03-21T13:30:05.308 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[30040]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "167.1"}]: dispatch 2024-03-21T13:30:05.308 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "167.2"}]: dispatch 2024-03-21T13:30:05.308 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[30040]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "167.2"}]: dispatch 2024-03-21T13:30:05.308 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "167.3"}]: dispatch 2024-03-21T13:30:05.308 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[30040]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "167.3"}]: dispatch 2024-03-21T13:30:05.308 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "167.4"}]: dispatch 2024-03-21T13:30:05.308 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[30040]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "167.4"}]: dispatch 2024-03-21T13:30:05.308 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "167.5"}]: dispatch 2024-03-21T13:30:05.308 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[30040]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "167.5"}]: dispatch 2024-03-21T13:30:05.308 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "167.6"}]: dispatch 2024-03-21T13:30:05.308 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[30040]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "167.6"}]: dispatch 2024-03-21T13:30:05.308 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "167.7"}]: dispatch 2024-03-21T13:30:05.309 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[30040]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "167.7"}]: dispatch 2024-03-21T13:30:05.309 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "167.8"}]: dispatch 2024-03-21T13:30:05.309 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[30040]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "167.8"}]: dispatch 2024-03-21T13:30:05.309 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "167.9"}]: dispatch 2024-03-21T13:30:05.309 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[30040]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "167.9"}]: dispatch 2024-03-21T13:30:05.309 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[30040]: 167.4 scrub starts 2024-03-21T13:30:05.309 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[30040]: 167.4 scrub ok 2024-03-21T13:30:05.309 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:05.309 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/872425909' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi097-79618-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:05.309 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[30040]: osdmap e129: 8 total, 8 up, 8 in 2024-03-21T13:30:05.309 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2916535762' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi097-79636-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:05.309 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi097-79636-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:05.309 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[30040]: 167.0 scrub starts 2024-03-21T13:30:05.309 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:04 smithi097 ceph-mon[30040]: 167.0 scrub ok 2024-03-21T13:30:06.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:05 smithi144 ceph-mon[33633]: 167.6 scrub starts 2024-03-21T13:30:06.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:05 smithi144 ceph-mon[33633]: 167.6 scrub ok 2024-03-21T13:30:06.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:05 smithi144 ceph-mon[33633]: 167.9 scrub starts 2024-03-21T13:30:06.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:05 smithi144 ceph-mon[33633]: 167.9 scrub ok 2024-03-21T13:30:06.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:05 smithi144 ceph-mon[33633]: 167.1 scrub starts 2024-03-21T13:30:06.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:05 smithi144 ceph-mon[33633]: 167.1 scrub ok 2024-03-21T13:30:06.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:05 smithi144 ceph-mon[33633]: 167.2 scrub starts 2024-03-21T13:30:06.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:05 smithi144 ceph-mon[33633]: 167.2 scrub ok 2024-03-21T13:30:06.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:05 smithi144 ceph-mon[33633]: 167.8 scrub starts 2024-03-21T13:30:06.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:05 smithi144 ceph-mon[33633]: 167.8 scrub ok 2024-03-21T13:30:06.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:05 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:06.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:05 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi097-79636-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:06.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:05 smithi144 ceph-mon[33633]: osdmap e130: 8 total, 8 up, 8 in 2024-03-21T13:30:06.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:05 smithi144 ceph-mon[33633]: 167.5 scrub starts 2024-03-21T13:30:06.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:05 smithi144 ceph-mon[33633]: 167.5 scrub ok 2024-03-21T13:30:06.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:05 smithi097 ceph-mon[39416]: 167.6 scrub starts 2024-03-21T13:30:06.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:05 smithi097 ceph-mon[39416]: 167.6 scrub ok 2024-03-21T13:30:06.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:05 smithi097 ceph-mon[39416]: 167.9 scrub starts 2024-03-21T13:30:06.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:05 smithi097 ceph-mon[39416]: 167.9 scrub ok 2024-03-21T13:30:06.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:05 smithi097 ceph-mon[39416]: 167.1 scrub starts 2024-03-21T13:30:06.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:05 smithi097 ceph-mon[39416]: 167.1 scrub ok 2024-03-21T13:30:06.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:05 smithi097 ceph-mon[39416]: 167.2 scrub starts 2024-03-21T13:30:06.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:05 smithi097 ceph-mon[39416]: 167.2 scrub ok 2024-03-21T13:30:06.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:05 smithi097 ceph-mon[39416]: 167.8 scrub starts 2024-03-21T13:30:06.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:05 smithi097 ceph-mon[39416]: 167.8 scrub ok 2024-03-21T13:30:06.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:05 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:06.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:05 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi097-79636-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:06.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:05 smithi097 ceph-mon[39416]: osdmap e130: 8 total, 8 up, 8 in 2024-03-21T13:30:06.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:05 smithi097 ceph-mon[39416]: 167.5 scrub starts 2024-03-21T13:30:06.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:05 smithi097 ceph-mon[39416]: 167.5 scrub ok 2024-03-21T13:30:06.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:05 smithi097 ceph-mon[30040]: 167.6 scrub starts 2024-03-21T13:30:06.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:05 smithi097 ceph-mon[30040]: 167.6 scrub ok 2024-03-21T13:30:06.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:05 smithi097 ceph-mon[30040]: 167.9 scrub starts 2024-03-21T13:30:06.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:05 smithi097 ceph-mon[30040]: 167.9 scrub ok 2024-03-21T13:30:06.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:05 smithi097 ceph-mon[30040]: 167.1 scrub starts 2024-03-21T13:30:06.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:05 smithi097 ceph-mon[30040]: 167.1 scrub ok 2024-03-21T13:30:06.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:05 smithi097 ceph-mon[30040]: 167.2 scrub starts 2024-03-21T13:30:06.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:05 smithi097 ceph-mon[30040]: 167.2 scrub ok 2024-03-21T13:30:06.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:05 smithi097 ceph-mon[30040]: 167.8 scrub starts 2024-03-21T13:30:06.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:05 smithi097 ceph-mon[30040]: 167.8 scrub ok 2024-03-21T13:30:06.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:05 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:06.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:05 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi097-79636-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:06.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:05 smithi097 ceph-mon[30040]: osdmap e130: 8 total, 8 up, 8 in 2024-03-21T13:30:06.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:05 smithi097 ceph-mon[30040]: 167.5 scrub starts 2024-03-21T13:30:06.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:05 smithi097 ceph-mon[30040]: 167.5 scrub ok 2024-03-21T13:30:07.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:06 smithi144 ceph-mon[33633]: pgmap v159: 492 pgs: 64 unknown, 428 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.5 KiB/s rd, 79 KiB/s wr, 78 op/s 2024-03-21T13:30:07.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:06 smithi144 ceph-mon[33633]: 167.7 scrub starts 2024-03-21T13:30:07.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:06 smithi144 ceph-mon[33633]: 167.7 scrub ok 2024-03-21T13:30:07.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:06 smithi144 ceph-mon[33633]: 167.3 scrub starts 2024-03-21T13:30:07.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:06 smithi144 ceph-mon[33633]: 167.3 scrub ok 2024-03-21T13:30:07.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:06 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:07.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:06 smithi144 ceph-mon[33633]: osdmap e131: 8 total, 8 up, 8 in 2024-03-21T13:30:07.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:06 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1882547468' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi097-79618-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:07.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:06 smithi097 ceph-mon[30040]: pgmap v159: 492 pgs: 64 unknown, 428 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.5 KiB/s rd, 79 KiB/s wr, 78 op/s 2024-03-21T13:30:07.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:06 smithi097 ceph-mon[30040]: 167.7 scrub starts 2024-03-21T13:30:07.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:06 smithi097 ceph-mon[30040]: 167.7 scrub ok 2024-03-21T13:30:07.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:06 smithi097 ceph-mon[30040]: 167.3 scrub starts 2024-03-21T13:30:07.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:06 smithi097 ceph-mon[30040]: 167.3 scrub ok 2024-03-21T13:30:07.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:06 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:07.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:06 smithi097 ceph-mon[30040]: osdmap e131: 8 total, 8 up, 8 in 2024-03-21T13:30:07.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:06 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1882547468' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi097-79618-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:07.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:06 smithi097 ceph-mon[39416]: pgmap v159: 492 pgs: 64 unknown, 428 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.5 KiB/s rd, 79 KiB/s wr, 78 op/s 2024-03-21T13:30:07.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:06 smithi097 ceph-mon[39416]: 167.7 scrub starts 2024-03-21T13:30:07.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:06 smithi097 ceph-mon[39416]: 167.7 scrub ok 2024-03-21T13:30:07.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:06 smithi097 ceph-mon[39416]: 167.3 scrub starts 2024-03-21T13:30:07.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:06 smithi097 ceph-mon[39416]: 167.3 scrub ok 2024-03-21T13:30:07.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:06 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:07.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:06 smithi097 ceph-mon[39416]: osdmap e131: 8 total, 8 up, 8 in 2024-03-21T13:30:07.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:06 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1882547468' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi097-79618-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:08.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:08 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:08.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:08 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1882547468' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi097-79618-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:08.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:08 smithi097 ceph-mon[39416]: osdmap e132: 8 total, 8 up, 8 in 2024-03-21T13:30:08.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:08 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3338056847' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi097-79636-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:08.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:08 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi097-79636-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:08.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:08 smithi097 ceph-mon[39416]: pgmap v162: 492 pgs: 64 unknown, 428 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-03-21T13:30:08.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:08 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:08.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:08 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1882547468' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi097-79618-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:08.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:08 smithi097 ceph-mon[30040]: osdmap e132: 8 total, 8 up, 8 in 2024-03-21T13:30:08.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:08 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3338056847' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi097-79636-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:08.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:08 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi097-79636-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:08.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:08 smithi097 ceph-mon[30040]: pgmap v162: 492 pgs: 64 unknown, 428 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-03-21T13:30:08.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:08 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:08.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:08 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1882547468' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi097-79618-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:08.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:08 smithi144 ceph-mon[33633]: osdmap e132: 8 total, 8 up, 8 in 2024-03-21T13:30:08.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:08 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3338056847' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi097-79636-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:08.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:08 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi097-79636-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:08.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:08 smithi144 ceph-mon[33633]: pgmap v162: 492 pgs: 64 unknown, 428 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-03-21T13:30:09.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:09 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:09.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:09 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi097-79636-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:09.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:09 smithi097 ceph-mon[30040]: osdmap e133: 8 total, 8 up, 8 in 2024-03-21T13:30:09.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:09 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1231934815' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi097-80000-15"}]: dispatch 2024-03-21T13:30:09.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:09 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi097-80000-15"}]: dispatch 2024-03-21T13:30:09.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:09 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:09.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:09 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:09.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:09 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi097-79636-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:09.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:09 smithi097 ceph-mon[39416]: osdmap e133: 8 total, 8 up, 8 in 2024-03-21T13:30:09.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:09 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1231934815' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi097-80000-15"}]: dispatch 2024-03-21T13:30:09.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:09 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi097-80000-15"}]: dispatch 2024-03-21T13:30:09.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:09 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:09.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:09 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:09.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:09 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi097-79636-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:09.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:09 smithi144 ceph-mon[33633]: osdmap e133: 8 total, 8 up, 8 in 2024-03-21T13:30:09.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:09 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1231934815' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi097-80000-15"}]: dispatch 2024-03-21T13:30:09.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:09 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi097-80000-15"}]: dispatch 2024-03-21T13:30:09.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:09 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:10.254 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started api_snapshots: Running main() from gmock_main.cc 2024-03-21T13:30:10.254 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots: [==========] Running 13 tests from 4 test suites. 2024-03-21T13:30:10.254 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots: [----------] Global test environment set-up. 2024-03-21T13:30:10.255 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots 2024-03-21T13:30:10.255 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapList 2024-03-21T13:30:10.255 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapList (6850 ms) 2024-03-21T13:30:10.255 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapRemove 2024-03-21T13:30:10.255 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapRemove (3056 ms) 2024-03-21T13:30:10.255 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.Rollback 2024-03-21T13:30:10.255 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots: [ OK ] LibRadosSnapshots.Rollback (2059 ms) 2024-03-21T13:30:10.255 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapGetName 2024-03-21T13:30:10.255 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapGetName (2103 ms) 2024-03-21T13:30:10.255 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots (14068 ms total) 2024-03-21T13:30:10.255 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots: 2024-03-21T13:30:10.255 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots: [----------] 3 tests from LibRadosSnapshotsSelfManaged 2024-03-21T13:30:10.255 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Snap 2024-03-21T13:30:10.255 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Snap (4122 ms) 2024-03-21T13:30:10.255 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Rollback 2024-03-21T13:30:10.256 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Rollback (4030 ms) 2024-03-21T13:30:10.256 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.FutureSnapRollback 2024-03-21T13:30:10.256 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.FutureSnapRollback (5035 ms) 2024-03-21T13:30:10.256 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots: [----------] 3 tests from LibRadosSnapshotsSelfManaged (13187 ms total) 2024-03-21T13:30:10.256 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots: 2024-03-21T13:30:10.256 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC 2024-03-21T13:30:10.256 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapList 2024-03-21T13:30:10.256 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapList (2987 ms) 2024-03-21T13:30:10.256 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapRemove 2024-03-21T13:30:10.256 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapRemove (2020 ms) 2024-03-21T13:30:10.256 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.Rollback 2024-03-21T13:30:10.256 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.Rollback (1988 ms) 2024-03-21T13:30:10.256 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapGetName 2024-03-21T13:30:10.256 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapGetName (1994 ms) 2024-03-21T13:30:10.256 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC (8989 ms total) 2024-03-21T13:30:10.257 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots: 2024-03-21T13:30:10.257 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC 2024-03-21T13:30:10.257 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Snap 2024-03-21T13:30:10.257 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Snap (4060 ms) 2024-03-21T13:30:10.257 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Rollback 2024-03-21T13:30:10.257 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Rollback (3970 ms) 2024-03-21T13:30:10.257 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC (8030 ms total) 2024-03-21T13:30:10.257 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots: 2024-03-21T13:30:10.257 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots: [----------] Global test environment tear-down 2024-03-21T13:30:10.257 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots: [==========] 13 tests from 4 test suites ran. (70706 ms total) 2024-03-21T13:30:10.257 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots: [ PASSED ] 13 tests. 2024-03-21T13:30:10.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:10 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi097-80000-15"}]': finished 2024-03-21T13:30:10.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:10 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1231934815' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi097-80000-15"}]: dispatch 2024-03-21T13:30:10.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:10 smithi097 ceph-mon[39416]: osdmap e134: 8 total, 8 up, 8 in 2024-03-21T13:30:10.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:10 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi097-80000-15"}]: dispatch 2024-03-21T13:30:10.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:10 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3228809502' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi097-79618-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:10.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:10 smithi097 ceph-mon[39416]: pgmap v165: 452 pgs: 32 creating+peering, 32 unknown, 388 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 53 KiB/s rd, 72 op/s 2024-03-21T13:30:10.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:10 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:30:10.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:10 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:30:10.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:10 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:10.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:10 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi097-80000-15"}]': finished 2024-03-21T13:30:10.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:10 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3228809502' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi097-79618-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:10.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:10 smithi097 ceph-mon[39416]: osdmap e135: 8 total, 8 up, 8 in 2024-03-21T13:30:10.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:10 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2811822211' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79992-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:10.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:10 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79992-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:10.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:10 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi097-79636-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:10.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:10 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi097-80000-15"}]': finished 2024-03-21T13:30:10.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:10 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1231934815' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi097-80000-15"}]: dispatch 2024-03-21T13:30:10.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:10 smithi097 ceph-mon[30040]: osdmap e134: 8 total, 8 up, 8 in 2024-03-21T13:30:10.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:10 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi097-80000-15"}]: dispatch 2024-03-21T13:30:10.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:10 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3228809502' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi097-79618-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:10.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:10 smithi097 ceph-mon[30040]: pgmap v165: 452 pgs: 32 creating+peering, 32 unknown, 388 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 53 KiB/s rd, 72 op/s 2024-03-21T13:30:10.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:10 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:30:10.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:10 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:30:10.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:10 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:10.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:10 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi097-80000-15"}]': finished 2024-03-21T13:30:10.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:10 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3228809502' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi097-79618-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:10.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:10 smithi097 ceph-mon[30040]: osdmap e135: 8 total, 8 up, 8 in 2024-03-21T13:30:10.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:10 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2811822211' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79992-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:10.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:10 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79992-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:10.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:10 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi097-79636-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:10.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:10 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi097-80000-15"}]': finished 2024-03-21T13:30:10.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:10 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1231934815' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi097-80000-15"}]: dispatch 2024-03-21T13:30:10.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:10 smithi144 ceph-mon[33633]: osdmap e134: 8 total, 8 up, 8 in 2024-03-21T13:30:10.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:10 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi097-80000-15"}]: dispatch 2024-03-21T13:30:10.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:10 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3228809502' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi097-79618-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:10.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:10 smithi144 ceph-mon[33633]: pgmap v165: 452 pgs: 32 creating+peering, 32 unknown, 388 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 53 KiB/s rd, 72 op/s 2024-03-21T13:30:10.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:10 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:30:10.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:10 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:30:10.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:10 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:10.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:10 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi097-80000-15"}]': finished 2024-03-21T13:30:10.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:10 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3228809502' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi097-79618-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:10.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:10 smithi144 ceph-mon[33633]: osdmap e135: 8 total, 8 up, 8 in 2024-03-21T13:30:10.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:10 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2811822211' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79992-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:10.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:10 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79992-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:10.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:10 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi097-79636-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:11.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:11 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/627954975' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi097-79636-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:11.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:11 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:11.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:11 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79992-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:11.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:11 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi097-79636-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:11.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:11 smithi097 ceph-mon[39416]: osdmap e136: 8 total, 8 up, 8 in 2024-03-21T13:30:11.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:11 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/627954975' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi097-79636-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:11.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:11 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:11.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:11 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79992-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:11.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:11 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi097-79636-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:11.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:11 smithi097 ceph-mon[30040]: osdmap e136: 8 total, 8 up, 8 in 2024-03-21T13:30:11.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:11 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/627954975' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi097-79636-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:11.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:11 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:11.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:11 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79992-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:11.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:11 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi097-79636-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:11.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:11 smithi144 ceph-mon[33633]: osdmap e136: 8 total, 8 up, 8 in 2024-03-21T13:30:12.279 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:30:11 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:30:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:30:12.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:12 smithi097 ceph-mon[39416]: pgmap v168: 484 pgs: 32 creating+peering, 64 unknown, 388 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 53 KiB/s rd, 72 op/s 2024-03-21T13:30:12.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:12 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:12.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:12 smithi097 ceph-mon[30040]: pgmap v168: 484 pgs: 32 creating+peering, 64 unknown, 388 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 53 KiB/s rd, 72 op/s 2024-03-21T13:30:12.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:12 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:12.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:12 smithi144 ceph-mon[33633]: pgmap v168: 484 pgs: 32 creating+peering, 64 unknown, 388 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 53 KiB/s rd, 72 op/s 2024-03-21T13:30:12.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:12 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:13.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:13 smithi097 ceph-mon[30040]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:30:13.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:13 smithi097 ceph-mon[30040]: osdmap e137: 8 total, 8 up, 8 in 2024-03-21T13:30:13.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:13 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2056807357' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi097-79618-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:13.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:13 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-21T13:30:13.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:13 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi097-79910-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:13.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:13 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi097-79910-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:13.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:13 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:13.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:13 smithi097 ceph-mon[39416]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:30:13.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:13 smithi097 ceph-mon[39416]: osdmap e137: 8 total, 8 up, 8 in 2024-03-21T13:30:13.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:13 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2056807357' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi097-79618-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:13.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:13 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-21T13:30:13.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:13 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi097-79910-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:13.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:13 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi097-79910-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:13.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:13 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:13.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:30:13 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:30:13.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:13 smithi144 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:30:13.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:13 smithi144 ceph-mon[33633]: osdmap e137: 8 total, 8 up, 8 in 2024-03-21T13:30:13.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:13 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2056807357' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi097-79618-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:13.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:13 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-21T13:30:13.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:13 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi097-79910-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:13.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:13 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi097-79910-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:13.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:13 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:14.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:14 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2056807357' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi097-79618-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:14.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:14 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi097-79910-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:14.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:14 smithi144 ceph-mon[33633]: osdmap e138: 8 total, 8 up, 8 in 2024-03-21T13:30:14.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:14 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi097-79910-1","app": "app2"}]: dispatch 2024-03-21T13:30:14.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:14 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2486178619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi097-79636-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:14.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:14 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi097-79910-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:14.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:14 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi097-79910-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:14.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:14 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi097-79636-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:14.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:14 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:30:14.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:14 smithi144 ceph-mon[33633]: pgmap v171: 516 pgs: 32 creating+peering, 3 active, 19 creating+activating, 462 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:30:14.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:14 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:14.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:14 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi097-79910-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:14.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:14 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi097-79636-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:14.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:14 smithi144 ceph-mon[33633]: osdmap e139: 8 total, 8 up, 8 in 2024-03-21T13:30:14.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:14 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi097-79910-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-03-21T13:30:14.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:14 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi097-79910-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-21T13:30:14.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:14 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi097-79910-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-21T13:30:14.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:14 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2056807357' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi097-79618-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:14.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:14 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi097-79910-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:14.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:14 smithi097 ceph-mon[30040]: osdmap e138: 8 total, 8 up, 8 in 2024-03-21T13:30:14.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:14 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi097-79910-1","app": "app2"}]: dispatch 2024-03-21T13:30:14.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:14 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2486178619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi097-79636-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:14.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:14 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi097-79910-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:14.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:14 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi097-79910-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:14.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:14 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi097-79636-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:14.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:14 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:30:14.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:14 smithi097 ceph-mon[30040]: pgmap v171: 516 pgs: 32 creating+peering, 3 active, 19 creating+activating, 462 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:30:14.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:14 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:14.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:14 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi097-79910-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:14.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:14 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi097-79636-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:14.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:14 smithi097 ceph-mon[30040]: osdmap e139: 8 total, 8 up, 8 in 2024-03-21T13:30:14.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:14 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi097-79910-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-03-21T13:30:14.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:14 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi097-79910-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-21T13:30:14.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:14 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi097-79910-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-21T13:30:14.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:14 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2056807357' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi097-79618-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:14.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:14 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi097-79910-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:14.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:14 smithi097 ceph-mon[39416]: osdmap e138: 8 total, 8 up, 8 in 2024-03-21T13:30:14.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:14 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi097-79910-1","app": "app2"}]: dispatch 2024-03-21T13:30:14.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:14 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2486178619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi097-79636-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:14.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:14 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi097-79910-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:14.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:14 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi097-79910-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:14.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:14 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi097-79636-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:14.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:14 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:30:14.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:14 smithi097 ceph-mon[39416]: pgmap v171: 516 pgs: 32 creating+peering, 3 active, 19 creating+activating, 462 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:30:14.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:14 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:14.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:14 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi097-79910-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:14.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:14 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi097-79636-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:14.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:14 smithi097 ceph-mon[39416]: osdmap e139: 8 total, 8 up, 8 in 2024-03-21T13:30:14.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:14 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi097-79910-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-03-21T13:30:14.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:14 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi097-79910-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-21T13:30:14.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:14 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi097-79910-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-21T13:30:15.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:15 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1382685012' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi097-79992-16"}]: dispatch 2024-03-21T13:30:15.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:15 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi097-79992-16"}]: dispatch 2024-03-21T13:30:15.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:15 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1382685012' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi097-79992-16"}]: dispatch 2024-03-21T13:30:15.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:15 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi097-79992-16"}]: dispatch 2024-03-21T13:30:15.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:15 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1382685012' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi097-79992-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:30:15.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:15 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi097-79992-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:30:15.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:15 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:15.624 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:15 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi097-79910-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-03-21T13:30:15.624 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:15 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi097-79992-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:30:15.624 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:15 smithi144 ceph-mon[33633]: osdmap e140: 8 total, 8 up, 8 in 2024-03-21T13:30:15.624 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:15 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1382685012' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi097-79992-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi097-79992-16"}]: dispatch 2024-03-21T13:30:15.624 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:15 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi097-79910-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-21T13:30:15.624 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:15 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi097-79992-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi097-79992-16"}]: dispatch 2024-03-21T13:30:15.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:15 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1382685012' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi097-79992-16"}]: dispatch 2024-03-21T13:30:15.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:15 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi097-79992-16"}]: dispatch 2024-03-21T13:30:15.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:15 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1382685012' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi097-79992-16"}]: dispatch 2024-03-21T13:30:15.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:15 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi097-79992-16"}]: dispatch 2024-03-21T13:30:15.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:15 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1382685012' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi097-79992-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:30:15.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:15 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi097-79992-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:30:15.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:15 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:15.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:15 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi097-79910-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-03-21T13:30:15.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:15 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi097-79992-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:30:15.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:15 smithi097 ceph-mon[30040]: osdmap e140: 8 total, 8 up, 8 in 2024-03-21T13:30:15.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:15 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1382685012' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi097-79992-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi097-79992-16"}]: dispatch 2024-03-21T13:30:15.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:15 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi097-79910-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-21T13:30:15.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:15 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi097-79992-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi097-79992-16"}]: dispatch 2024-03-21T13:30:15.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:15 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1382685012' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi097-79992-16"}]: dispatch 2024-03-21T13:30:15.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:15 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi097-79992-16"}]: dispatch 2024-03-21T13:30:15.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:15 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1382685012' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi097-79992-16"}]: dispatch 2024-03-21T13:30:15.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:15 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi097-79992-16"}]: dispatch 2024-03-21T13:30:15.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:15 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1382685012' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi097-79992-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:30:15.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:15 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi097-79992-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:30:15.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:15 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:15.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:15 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi097-79910-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-03-21T13:30:15.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:15 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi097-79992-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:30:15.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:15 smithi097 ceph-mon[39416]: osdmap e140: 8 total, 8 up, 8 in 2024-03-21T13:30:15.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:15 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1382685012' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi097-79992-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi097-79992-16"}]: dispatch 2024-03-21T13:30:15.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:15 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi097-79910-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-21T13:30:15.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:15 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi097-79992-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi097-79992-16"}]: dispatch 2024-03-21T13:30:16.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:16 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi097-79910-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-21T13:30:16.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:16 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1962368401' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi097-79618-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:16.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:16 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi097-79618-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:16.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:16 smithi097 ceph-mon[39416]: pgmap v174: 452 pgs: 32 unknown, 420 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:30:16.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:16 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:16.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:16 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi097-79910-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-03-21T13:30:16.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:16 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi097-79910-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-21T13:30:16.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:16 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1962368401' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi097-79618-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:16.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:16 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi097-79618-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:16.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:16 smithi097 ceph-mon[30040]: pgmap v174: 452 pgs: 32 unknown, 420 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:30:16.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:16 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:16.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:16 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi097-79910-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-03-21T13:30:16.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:16 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi097-79910-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-21T13:30:16.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:16 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1962368401' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi097-79618-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:16.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:16 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi097-79618-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:16.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:16 smithi144 ceph-mon[33633]: pgmap v174: 452 pgs: 32 unknown, 420 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:30:16.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:16 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:16.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:16 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi097-79910-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-03-21T13:30:17.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:17 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi097-79618-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:17.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:17 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi097-79910-1","app":"app1","key":"key1"}]: dispatch 2024-03-21T13:30:17.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:17 smithi097 ceph-mon[39416]: osdmap e141: 8 total, 8 up, 8 in 2024-03-21T13:30:17.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:17 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3043213549' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi097-79636-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:17.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:17 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi097-79910-1","app":"app1","key":"key1"}]: dispatch 2024-03-21T13:30:17.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:17 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi097-79636-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:17.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:17 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi097-79992-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi097-79992-16"}]': finished 2024-03-21T13:30:17.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:17 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi097-79910-1","app":"app1","key":"key1"}]': finished 2024-03-21T13:30:17.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:17 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi097-79636-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:17.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:17 smithi097 ceph-mon[39416]: osdmap e142: 8 total, 8 up, 8 in 2024-03-21T13:30:17.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:17 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:17.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:17 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi097-79618-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:17.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:17 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi097-79910-1","app":"app1","key":"key1"}]: dispatch 2024-03-21T13:30:17.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:17 smithi097 ceph-mon[30040]: osdmap e141: 8 total, 8 up, 8 in 2024-03-21T13:30:17.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:17 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3043213549' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi097-79636-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:17.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:17 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi097-79910-1","app":"app1","key":"key1"}]: dispatch 2024-03-21T13:30:17.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:17 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi097-79636-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:17.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:17 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi097-79992-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi097-79992-16"}]': finished 2024-03-21T13:30:17.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:17 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi097-79910-1","app":"app1","key":"key1"}]': finished 2024-03-21T13:30:17.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:17 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi097-79636-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:17.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:17 smithi097 ceph-mon[30040]: osdmap e142: 8 total, 8 up, 8 in 2024-03-21T13:30:17.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:17 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:17.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:17 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi097-79618-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:17.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:17 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2876587590' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi097-79910-1","app":"app1","key":"key1"}]: dispatch 2024-03-21T13:30:17.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:17 smithi144 ceph-mon[33633]: osdmap e141: 8 total, 8 up, 8 in 2024-03-21T13:30:17.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:17 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3043213549' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi097-79636-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:17.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:17 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi097-79910-1","app":"app1","key":"key1"}]: dispatch 2024-03-21T13:30:17.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:17 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi097-79636-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:17.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:17 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi097-79992-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi097-79992-16"}]': finished 2024-03-21T13:30:17.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:17 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi097-79910-1","app":"app1","key":"key1"}]': finished 2024-03-21T13:30:17.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:17 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi097-79636-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:17.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:17 smithi144 ceph-mon[33633]: osdmap e142: 8 total, 8 up, 8 in 2024-03-21T13:30:17.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:17 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:18.243 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [==========] Running 31 tests from 7 test suites. 2024-03-21T13:30:18.243 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [----------] Global test environment set-up. 2024-03-21T13:30:18.243 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion 2024-03-21T13:30:18.244 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ RUN ] LibRadosMiscVersion.VersionPP 2024-03-21T13:30:18.244 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ OK ] LibRadosMiscVersion.VersionPP (0 ms) 2024-03-21T13:30:18.244 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-03-21T13:30:18.244 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: 2024-03-21T13:30:18.244 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP 2024-03-21T13:30:18.244 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: seed 79910 2024-03-21T13:30:18.244 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WaitOSDMapPP 2024-03-21T13:30:18.244 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WaitOSDMapPP (1 ms) 2024-03-21T13:30:18.244 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNamePP 2024-03-21T13:30:18.244 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNamePP (391 ms) 2024-03-21T13:30:18.244 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongLocatorPP 2024-03-21T13:30:18.244 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongLocatorPP (21 ms) 2024-03-21T13:30:18.244 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNSpacePP 2024-03-21T13:30:18.244 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNSpacePP (13 ms) 2024-03-21T13:30:18.245 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongAttrNamePP 2024-03-21T13:30:18.245 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongAttrNamePP (19 ms) 2024-03-21T13:30:18.245 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.ExecPP 2024-03-21T13:30:18.245 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.ExecPP (3 ms) 2024-03-21T13:30:18.245 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BadFlagsPP 2024-03-21T13:30:18.245 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BadFlagsPP (2 ms) 2024-03-21T13:30:18.245 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate1PP 2024-03-21T13:30:18.245 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate1PP (4 ms) 2024-03-21T13:30:18.245 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate2PP 2024-03-21T13:30:18.245 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate2PP (3 ms) 2024-03-21T13:30:18.245 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigObjectPP 2024-03-21T13:30:18.245 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigObjectPP (9 ms) 2024-03-21T13:30:18.245 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AioOperatePP 2024-03-21T13:30:18.245 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AioOperatePP (2 ms) 2024-03-21T13:30:18.245 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertExistsPP 2024-03-21T13:30:18.246 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertExistsPP (5 ms) 2024-03-21T13:30:18.246 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertVersionPP 2024-03-21T13:30:18.246 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertVersionPP (8 ms) 2024-03-21T13:30:18.246 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigAttrPP 2024-03-21T13:30:18.246 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: osd_max_attr_size = 0 2024-03-21T13:30:18.246 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: osd_max_attr_size == 0; skipping test 2024-03-21T13:30:18.246 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigAttrPP (1550 ms) 2024-03-21T13:30:18.246 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyPP 2024-03-21T13:30:18.246 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyPP (496 ms) 2024-03-21T13:30:18.246 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyScrubPP 2024-03-21T13:30:18.246 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: waiting for initial deep scrubs... 2024-03-21T13:30:18.246 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: done waiting, doing copies 2024-03-21T13:30:18.246 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: waiting for final deep scrubs... 2024-03-21T13:30:18.246 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: done waiting 2024-03-21T13:30:18.247 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyScrubPP (60619 ms) 2024-03-21T13:30:18.247 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WriteSamePP 2024-03-21T13:30:18.247 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WriteSamePP (3 ms) 2024-03-21T13:30:18.247 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CmpExtPP 2024-03-21T13:30:18.247 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CmpExtPP (3 ms) 2024-03-21T13:30:18.247 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Applications 2024-03-21T13:30:18.247 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Applications (4864 ms) 2024-03-21T13:30:18.247 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatOSD 2024-03-21T13:30:18.247 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatOSD (0 ms) 2024-03-21T13:30:18.247 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatClient 2024-03-21T13:30:18.247 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatClient (0 ms) 2024-03-21T13:30:18.247 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Conf 2024-03-21T13:30:18.247 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Conf (0 ms) 2024-03-21T13:30:18.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:18 smithi097 ceph-mon[39416]: pgmap v177: 460 pgs: 40 unknown, 420 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-03-21T13:30:18.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:18 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:18.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:18 smithi097 ceph-mon[39416]: osdmap e143: 8 total, 8 up, 8 in 2024-03-21T13:30:18.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:18 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1507615476' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi097-79618-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:18.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:18 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi097-79618-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:18.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:18 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1630115454' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi097-79910-24"}]: dispatch 2024-03-21T13:30:18.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:18 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi097-79910-24"}]: dispatch 2024-03-21T13:30:18.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:18 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1630115454' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi097-79910-24"}]: dispatch 2024-03-21T13:30:18.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:18 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi097-79910-24"}]: dispatch 2024-03-21T13:30:18.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:18 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1630115454' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi097-79910-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:30:18.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:18 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi097-79910-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:30:18.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:18 smithi097 ceph-mon[30040]: pgmap v177: 460 pgs: 40 unknown, 420 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-03-21T13:30:18.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:18 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:18.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:18 smithi097 ceph-mon[30040]: osdmap e143: 8 total, 8 up, 8 in 2024-03-21T13:30:18.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:18 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1507615476' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi097-79618-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:18.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:18 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi097-79618-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:18.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:18 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1630115454' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi097-79910-24"}]: dispatch 2024-03-21T13:30:18.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:18 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi097-79910-24"}]: dispatch 2024-03-21T13:30:18.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:18 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1630115454' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi097-79910-24"}]: dispatch 2024-03-21T13:30:18.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:18 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi097-79910-24"}]: dispatch 2024-03-21T13:30:18.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:18 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1630115454' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi097-79910-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:30:18.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:18 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi097-79910-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:30:18.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:18 smithi144 ceph-mon[33633]: pgmap v177: 460 pgs: 40 unknown, 420 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-03-21T13:30:18.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:18 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:18.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:18 smithi144 ceph-mon[33633]: osdmap e143: 8 total, 8 up, 8 in 2024-03-21T13:30:18.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:18 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1507615476' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi097-79618-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:18.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:18 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi097-79618-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:18.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:18 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1630115454' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi097-79910-24"}]: dispatch 2024-03-21T13:30:18.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:18 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi097-79910-24"}]: dispatch 2024-03-21T13:30:18.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:18 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1630115454' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi097-79910-24"}]: dispatch 2024-03-21T13:30:18.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:18 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi097-79910-24"}]: dispatch 2024-03-21T13:30:18.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:18 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1630115454' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi097-79910-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:30:18.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:18 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi097-79910-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:30:20.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:20 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi097-79618-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:20.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:20 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi097-79910-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:30:20.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:20 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:20.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:20 smithi097 ceph-mon[39416]: osdmap e144: 8 total, 8 up, 8 in 2024-03-21T13:30:20.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:20 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1630115454' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi097-79910-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi097-79910-24"}]: dispatch 2024-03-21T13:30:20.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:20 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi097-79910-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi097-79910-24"}]: dispatch 2024-03-21T13:30:20.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:20 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3724262284' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi097-79636-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:20.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:20 smithi097 ceph-mon[39416]: pgmap v180: 460 pgs: 32 unknown, 32 creating+peering, 396 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-03-21T13:30:20.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:20 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi097-79618-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:20.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:20 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi097-79910-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:30:20.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:20 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:20.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:20 smithi097 ceph-mon[30040]: osdmap e144: 8 total, 8 up, 8 in 2024-03-21T13:30:20.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:20 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1630115454' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi097-79910-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi097-79910-24"}]: dispatch 2024-03-21T13:30:20.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:20 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi097-79910-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi097-79910-24"}]: dispatch 2024-03-21T13:30:20.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:20 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3724262284' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi097-79636-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:20.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:20 smithi097 ceph-mon[30040]: pgmap v180: 460 pgs: 32 unknown, 32 creating+peering, 396 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-03-21T13:30:20.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:20 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi097-79618-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:20.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:20 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi097-79910-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:30:20.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:20 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:20.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:20 smithi144 ceph-mon[33633]: osdmap e144: 8 total, 8 up, 8 in 2024-03-21T13:30:20.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:20 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1630115454' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi097-79910-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi097-79910-24"}]: dispatch 2024-03-21T13:30:20.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:20 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi097-79910-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi097-79910-24"}]: dispatch 2024-03-21T13:30:20.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:20 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3724262284' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi097-79636-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:20.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:20 smithi144 ceph-mon[33633]: pgmap v180: 460 pgs: 32 unknown, 32 creating+peering, 396 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-03-21T13:30:21.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:21 smithi097 ceph-mon[39416]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:30:21.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:21 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3724262284' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi097-79636-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:21.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:21 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:21.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:21 smithi097 ceph-mon[39416]: osdmap e145: 8 total, 8 up, 8 in 2024-03-21T13:30:21.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:21 smithi097 ceph-mon[30040]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:30:21.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:21 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3724262284' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi097-79636-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:21.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:21 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:21.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:21 smithi097 ceph-mon[30040]: osdmap e145: 8 total, 8 up, 8 in 2024-03-21T13:30:21.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:21 smithi144 ceph-mon[33633]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:30:21.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:21 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3724262284' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi097-79636-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:21.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:21 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:21.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:21 smithi144 ceph-mon[33633]: osdmap e145: 8 total, 8 up, 8 in 2024-03-21T13:30:22.257 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:30:21 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:30:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:30:22.267 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP api_snapshots_pp: Running main() from gmock_main.cc 2024-03-21T13:30:22.267 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [==========] Running 21 tests from 5 test suites. 2024-03-21T13:30:22.267 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [----------] Global test environment set-up. 2024-03-21T13:30:22.267 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP 2024-03-21T13:30:22.267 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: seed 79992 2024-03-21T13:30:22.267 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapListPP 2024-03-21T13:30:22.267 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapListPP (3726 ms) 2024-03-21T13:30:22.267 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapRemovePP 2024-03-21T13:30:22.267 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapRemovePP (2859 ms) 2024-03-21T13:30:22.267 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.RollbackPP 2024-03-21T13:30:22.267 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.RollbackPP (3040 ms) 2024-03-21T13:30:22.267 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapGetNamePP 2024-03-21T13:30:22.267 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapGetNamePP (2336 ms) 2024-03-21T13:30:22.267 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapCreateRemovePP 2024-03-21T13:30:22.267 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapCreateRemovePP (3871 ms) 2024-03-21T13:30:22.268 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP (15832 ms total) 2024-03-21T13:30:22.268 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: 2024-03-21T13:30:22.268 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [----------] 7 tests from LibRadosSnapshotsSelfManagedPP 2024-03-21T13:30:22.268 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapPP 2024-03-21T13:30:22.268 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapPP (5133 ms) 2024-03-21T13:30:22.268 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.RollbackPP 2024-03-21T13:30:22.268 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.RollbackPP (3930 ms) 2024-03-21T13:30:22.268 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP 2024-03-21T13:30:22.268 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP (6017 ms) 2024-03-21T13:30:22.268 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.Bug11677 2024-03-21T13:30:22.268 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.Bug11677 (3989 ms) 2024-03-21T13:30:22.268 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.OrderSnap 2024-03-21T13:30:22.268 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.OrderSnap (2064 ms) 2024-03-21T13:30:22.268 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2024-03-21T13:30:22.268 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2267-g960fbbe4/rpm/el9/BUILD/ceph-19.0.0-2267-g960fbbe4/src/test/librados/snapshots_cxx.cc:463: Skipped 2024-03-21T13:30:22.269 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: 2024-03-21T13:30:22.269 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback (0 ms) 2024-03-21T13:30:22.269 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap 2024-03-21T13:30:22.269 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: deleting snap 14 in pool LibRadosSnapshotsSelfManagedPP_smithi097-79992-7 2024-03-21T13:30:22.269 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: waiting for snaps to purge 2024-03-21T13:30:22.269 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap (17943 ms) 2024-03-21T13:30:22.269 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [----------] 7 tests from LibRadosSnapshotsSelfManagedPP (39076 ms total) 2024-03-21T13:30:22.269 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: 2024-03-21T13:30:22.269 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode 2024-03-21T13:30:22.269 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected 2024-03-21T13:30:22.269 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected (9 ms) 2024-03-21T13:30:22.269 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance 2024-03-21T13:30:22.269 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance (5100 ms) 2024-03-21T13:30:22.269 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode (5109 ms total) 2024-03-21T13:30:22.269 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: 2024-03-21T13:30:22.270 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP 2024-03-21T13:30:22.270 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapListPP 2024-03-21T13:30:22.270 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapListPP (3259 ms) 2024-03-21T13:30:22.270 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapRemovePP 2024-03-21T13:30:22.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:22 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:22.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:22 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi097-79910-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi097-79910-24"}]': finished 2024-03-21T13:30:22.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:22 smithi097 ceph-mon[39416]: osdmap e146: 8 total, 8 up, 8 in 2024-03-21T13:30:22.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:22 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2040032049' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi097-79618-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:22.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:22 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi097-79618-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:22.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:22 smithi097 ceph-mon[39416]: pgmap v183: 436 pgs: 40 unknown, 396 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-03-21T13:30:22.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:22 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:22.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:22 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi097-79910-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi097-79910-24"}]': finished 2024-03-21T13:30:22.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:22 smithi097 ceph-mon[30040]: osdmap e146: 8 total, 8 up, 8 in 2024-03-21T13:30:22.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:22 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2040032049' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi097-79618-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:22.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:22 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi097-79618-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:22.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:22 smithi097 ceph-mon[30040]: pgmap v183: 436 pgs: 40 unknown, 396 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-03-21T13:30:22.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:22 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:22.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:22 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi097-79910-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi097-79910-24"}]': finished 2024-03-21T13:30:22.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:22 smithi144 ceph-mon[33633]: osdmap e146: 8 total, 8 up, 8 in 2024-03-21T13:30:22.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:22 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2040032049' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi097-79618-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:22.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:22 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi097-79618-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:22.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:22 smithi144 ceph-mon[33633]: pgmap v183: 436 pgs: 40 unknown, 396 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-03-21T13:30:23.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:23 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:23.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:23 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi097-79618-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:23.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:23 smithi097 ceph-mon[39416]: osdmap e147: 8 total, 8 up, 8 in 2024-03-21T13:30:23.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:23 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3228756653' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi097-79636-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:23.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:23 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi097-79636-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:23.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:23 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1630115454' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79910-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:23.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:23 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79910-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:23.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:23 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:23.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:23 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:23.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:23 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi097-79618-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:23.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:23 smithi097 ceph-mon[30040]: osdmap e147: 8 total, 8 up, 8 in 2024-03-21T13:30:23.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:23 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3228756653' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi097-79636-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:23.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:23 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi097-79636-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:23.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:23 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1630115454' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79910-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:23.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:23 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79910-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:23.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:23 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:23.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:30:23 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:30:23.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:23 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:23.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:23 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi097-79618-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:23.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:23 smithi144 ceph-mon[33633]: osdmap e147: 8 total, 8 up, 8 in 2024-03-21T13:30:23.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:23 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3228756653' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi097-79636-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:23.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:23 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi097-79636-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:23.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:23 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1630115454' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79910-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:23.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:23 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79910-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:23.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:23 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:24.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:24 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi097-79636-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:24.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:24 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79910-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:24.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:24 smithi097 ceph-mon[39416]: osdmap e148: 8 total, 8 up, 8 in 2024-03-21T13:30:24.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:24 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1630115454' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79910-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:24.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:24 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79910-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:24.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:24 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:30:24.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:24 smithi097 ceph-mon[39416]: pgmap v186: 468 pgs: 1 active, 10 creating+activating, 457 active+clean; 593 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-21T13:30:24.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:24 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:24.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:24 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi097-79636-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:24.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:24 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79910-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:24.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:24 smithi097 ceph-mon[30040]: osdmap e148: 8 total, 8 up, 8 in 2024-03-21T13:30:24.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:24 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1630115454' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79910-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:24.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:24 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79910-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:24.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:24 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:30:24.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:24 smithi097 ceph-mon[30040]: pgmap v186: 468 pgs: 1 active, 10 creating+activating, 457 active+clean; 593 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-21T13:30:24.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:24 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:24.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:24 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi097-79636-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:24.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:24 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79910-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:24.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:24 smithi144 ceph-mon[33633]: osdmap e148: 8 total, 8 up, 8 in 2024-03-21T13:30:24.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:24 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1630115454' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79910-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:24.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:24 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79910-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:24.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:24 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:30:24.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:24 smithi144 ceph-mon[33633]: pgmap v186: 468 pgs: 1 active, 10 creating+activating, 457 active+clean; 593 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-21T13:30:24.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:24 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:25.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:25 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79910-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:25.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:25 smithi097 ceph-mon[39416]: osdmap e149: 8 total, 8 up, 8 in 2024-03-21T13:30:25.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:25 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/735889136' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi097-79618-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:25.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:25 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi097-79618-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:25.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:25 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:30:25.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:25 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:25.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:25 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79910-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:25.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:25 smithi097 ceph-mon[30040]: osdmap e149: 8 total, 8 up, 8 in 2024-03-21T13:30:25.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:25 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/735889136' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi097-79618-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:25.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:25 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi097-79618-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:25.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:25 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:30:25.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:25 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:25.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:25 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79910-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:25.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:25 smithi144 ceph-mon[33633]: osdmap e149: 8 total, 8 up, 8 in 2024-03-21T13:30:25.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:25 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/735889136' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi097-79618-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:25.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:25 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi097-79618-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:25.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:25 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:30:25.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:25 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:26.295 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [ OK ] LibRadosS snapshots: Running main() from gmock_main.cc 2024-03-21T13:30:26.295 INFO:tasks.workunit.client.0.smithi097.stdout: snapshots: [==========] Running 11 tests from 2 test suites. 2024-03-21T13:30:26.295 INFO:tasks.workunit.client.0.smithi097.stdout: snapshots: [----------] Global test environment set-up. 2024-03-21T13:30:26.295 INFO:tasks.workunit.client.0.smithi097.stdout: snapshots: [----------] 5 tests from NeoRadosSnapshots 2024-03-21T13:30:26.295 INFO:tasks.workunit.client.0.smithi097.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapList 2024-03-21T13:30:26.295 INFO:tasks.workunit.client.0.smithi097.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapList (14833 ms) 2024-03-21T13:30:26.295 INFO:tasks.workunit.client.0.smithi097.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapRemove 2024-03-21T13:30:26.295 INFO:tasks.workunit.client.0.smithi097.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapRemove (6480 ms) 2024-03-21T13:30:26.295 INFO:tasks.workunit.client.0.smithi097.stdout: snapshots: [ RUN ] NeoRadosSnapshots.Rollback 2024-03-21T13:30:26.295 INFO:tasks.workunit.client.0.smithi097.stdout: snapshots: [ OK ] NeoRadosSnapshots.Rollback (4896 ms) 2024-03-21T13:30:26.295 INFO:tasks.workunit.client.0.smithi097.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapGetName 2024-03-21T13:30:26.295 INFO:tasks.workunit.client.0.smithi097.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapGetName (5123 ms) 2024-03-21T13:30:26.296 INFO:tasks.workunit.client.0.smithi097.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapCreateRemove 2024-03-21T13:30:26.296 INFO:tasks.workunit.client.0.smithi097.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapCreateRemove (7028 ms) 2024-03-21T13:30:26.296 INFO:tasks.workunit.client.0.smithi097.stdout: snapshots: [----------] 5 tests from NeoRadosSnapshots (38360 ms total) 2024-03-21T13:30:26.296 INFO:tasks.workunit.client.0.smithi097.stdout: snapshots: 2024-03-21T13:30:26.296 INFO:tasks.workunit.client.0.smithi097.stdout: snapshots: [----------] 6 tests from NeoRadosSelfManagedSnaps 2024-03-21T13:30:26.296 INFO:tasks.workunit.client.0.smithi097.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Snap 2024-03-21T13:30:26.296 INFO:tasks.workunit.client.0.smithi097.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Snap (5264 ms) 2024-03-21T13:30:26.296 INFO:tasks.workunit.client.0.smithi097.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Rollback 2024-03-21T13:30:26.296 INFO:tasks.workunit.client.0.smithi097.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Rollback (5774 ms) 2024-03-21T13:30:26.296 INFO:tasks.workunit.client.0.smithi097.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.SnapOverlap 2024-03-21T13:30:26.296 INFO:tasks.workunit.client.0.smithi097.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.SnapOverlap (7985 ms) 2024-03-21T13:30:26.296 INFO:tasks.workunit.client.0.smithi097.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Bug11677 2024-03-21T13:30:26.296 INFO:tasks.workunit.client.0.smithi097.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Bug11677 (6015 ms) 2024-03-21T13:30:26.296 INFO:tasks.workunit.client.0.smithi097.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.OrderSnap 2024-03-21T13:30:26.296 INFO:tasks.workunit.client.0.smithi097.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.OrderSnap (3981 ms) 2024-03-21T13:30:26.297 INFO:tasks.workunit.client.0.smithi097.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.ReusePurgedSnap 2024-03-21T13:30:26.297 INFO:tasks.workunit.client.0.smithi097.stdout: snapshots: Deleting snap 3 in pool ReusePurgedSnapsmithi097-80912-11. 2024-03-21T13:30:26.297 INFO:tasks.workunit.client.0.smithi097.stdout: snapshots: Waiting for snaps to purge. 2024-03-21T13:30:26.297 INFO:tasks.workunit.client.0.smithi097.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.ReusePurgedSnap (19043 ms) 2024-03-21T13:30:26.297 INFO:tasks.workunit.client.0.smithi097.stdout: snapshots: [----------] 6 tests from NeoRadosSelfManagedSnaps (48062 ms total) 2024-03-21T13:30:26.297 INFO:tasks.workunit.client.0.smithi097.stdout: snapshots: 2024-03-21T13:30:26.297 INFO:tasks.workunit.client.0.smithi097.stdout: snapshots: [----------] Global test environment tear-down 2024-03-21T13:30:26.297 INFO:tasks.workunit.client.0.smithi097.stdout: snapshots: [==========] 11 tests from 2 test suites ran. (86422 ms total) 2024-03-21T13:30:26.297 INFO:tasks.workunit.client.0.smithi097.stdout: snapshots: [ PASSED ] 11 tests. 2024-03-21T13:30:26.311 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:26 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi097-79618-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:26.311 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:26 smithi144 ceph-mon[33633]: osdmap e150: 8 total, 8 up, 8 in 2024-03-21T13:30:26.311 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:26 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2008779399' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi097-79636-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:26.311 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:26 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi097-79636-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:26.311 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:26 smithi144 ceph-mon[33633]: pgmap v189: 468 pgs: 64 unknown, 404 active+clean; 592 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-03-21T13:30:26.311 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:26 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:26.311 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:26 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi097-79636-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:26.312 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:26 smithi144 ceph-mon[33633]: osdmap e151: 8 total, 8 up, 8 in 2024-03-21T13:30:26.312 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:26 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1630115454' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi097-79910-24"}]: dispatch 2024-03-21T13:30:26.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:26 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi097-79910-24"}]: dispatch 2024-03-21T13:30:26.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:26 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi097-79618-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:26.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:26 smithi097 ceph-mon[39416]: osdmap e150: 8 total, 8 up, 8 in 2024-03-21T13:30:26.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:26 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2008779399' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi097-79636-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:26.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:26 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi097-79636-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:26.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:26 smithi097 ceph-mon[39416]: pgmap v189: 468 pgs: 64 unknown, 404 active+clean; 592 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-03-21T13:30:26.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:26 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:26.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:26 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi097-79636-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:26.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:26 smithi097 ceph-mon[39416]: osdmap e151: 8 total, 8 up, 8 in 2024-03-21T13:30:26.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:26 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1630115454' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi097-79910-24"}]: dispatch 2024-03-21T13:30:26.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:26 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi097-79910-24"}]: dispatch 2024-03-21T13:30:26.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:26 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi097-79618-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:26.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:26 smithi097 ceph-mon[30040]: osdmap e150: 8 total, 8 up, 8 in 2024-03-21T13:30:26.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:26 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2008779399' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi097-79636-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:26.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:26 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi097-79636-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:26.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:26 smithi097 ceph-mon[30040]: pgmap v189: 468 pgs: 64 unknown, 404 active+clean; 592 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-03-21T13:30:26.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:26 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:26.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:26 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi097-79636-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:26.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:26 smithi097 ceph-mon[30040]: osdmap e151: 8 total, 8 up, 8 in 2024-03-21T13:30:26.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:26 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1630115454' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi097-79910-24"}]: dispatch 2024-03-21T13:30:26.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:26 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi097-79910-24"}]: dispatch 2024-03-21T13:30:28.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:28 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi097-79910-24"}]': finished 2024-03-21T13:30:28.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:28 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1630115454' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi097-79910-24"}]: dispatch 2024-03-21T13:30:28.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:28 smithi097 ceph-mon[39416]: osdmap e152: 8 total, 8 up, 8 in 2024-03-21T13:30:28.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:28 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1382685012' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi097-79992-16"}]: dispatch 2024-03-21T13:30:28.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:28 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1288000562' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi097-79618-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:28.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:28 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi097-79910-24"}]: dispatch 2024-03-21T13:30:28.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:28 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi097-79992-16"}]: dispatch 2024-03-21T13:30:28.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:28 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi097-79618-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:28.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:28 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:28.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:28 smithi097 ceph-mon[39416]: pgmap v192: 388 pgs: 32 unknown, 356 active+clean; 592 KiB data, 609 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:30:28.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:28 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi097-79910-24"}]': finished 2024-03-21T13:30:28.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:28 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1630115454' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi097-79910-24"}]: dispatch 2024-03-21T13:30:28.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:28 smithi097 ceph-mon[30040]: osdmap e152: 8 total, 8 up, 8 in 2024-03-21T13:30:28.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:28 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1382685012' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi097-79992-16"}]: dispatch 2024-03-21T13:30:28.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:28 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1288000562' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi097-79618-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:28.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:28 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi097-79910-24"}]: dispatch 2024-03-21T13:30:28.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:28 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi097-79992-16"}]: dispatch 2024-03-21T13:30:28.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:28 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi097-79618-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:28.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:28 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:28.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:28 smithi097 ceph-mon[30040]: pgmap v192: 388 pgs: 32 unknown, 356 active+clean; 592 KiB data, 609 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:30:28.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:28 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi097-79910-24"}]': finished 2024-03-21T13:30:28.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:28 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1630115454' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi097-79910-24"}]: dispatch 2024-03-21T13:30:28.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:28 smithi144 ceph-mon[33633]: osdmap e152: 8 total, 8 up, 8 in 2024-03-21T13:30:28.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:28 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1382685012' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi097-79992-16"}]: dispatch 2024-03-21T13:30:28.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:28 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1288000562' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi097-79618-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:28.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:28 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi097-79910-24"}]: dispatch 2024-03-21T13:30:28.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:28 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi097-79992-16"}]: dispatch 2024-03-21T13:30:28.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:28 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi097-79618-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:28.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:28 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:28.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:28 smithi144 ceph-mon[33633]: pgmap v192: 388 pgs: 32 unknown, 356 active+clean; 592 KiB data, 609 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:30:29.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:29 smithi097 ceph-mon[30040]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:30:29.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:29 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi097-79910-24"}]': finished 2024-03-21T13:30:29.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:29 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi097-79992-16"}]': finished 2024-03-21T13:30:29.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:29 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi097-79618-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:29.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:29 smithi097 ceph-mon[30040]: osdmap e153: 8 total, 8 up, 8 in 2024-03-21T13:30:29.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:29 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/930467209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi097-79636-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:29.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:29 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi097-79636-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:29.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:29 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1382685012' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi097-79992-16"}]: dispatch 2024-03-21T13:30:29.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:29 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi097-79992-16"}]: dispatch 2024-03-21T13:30:29.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:29 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:29.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:29 smithi097 ceph-mon[39416]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:30:29.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:29 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi097-79910-24"}]': finished 2024-03-21T13:30:29.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:29 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi097-79992-16"}]': finished 2024-03-21T13:30:29.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:29 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi097-79618-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:29.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:29 smithi097 ceph-mon[39416]: osdmap e153: 8 total, 8 up, 8 in 2024-03-21T13:30:29.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:29 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/930467209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi097-79636-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:29.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:29 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi097-79636-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:29.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:29 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1382685012' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi097-79992-16"}]: dispatch 2024-03-21T13:30:29.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:29 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi097-79992-16"}]: dispatch 2024-03-21T13:30:29.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:29 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:29.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:29 smithi144 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:30:29.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:29 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi097-79910-24"}]': finished 2024-03-21T13:30:29.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:29 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi097-79992-16"}]': finished 2024-03-21T13:30:29.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:29 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi097-79618-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:29.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:29 smithi144 ceph-mon[33633]: osdmap e153: 8 total, 8 up, 8 in 2024-03-21T13:30:29.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:29 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/930467209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi097-79636-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:29.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:29 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi097-79636-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:29.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:29 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1382685012' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi097-79992-16"}]: dispatch 2024-03-21T13:30:29.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:29 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi097-79992-16"}]: dispatch 2024-03-21T13:30:29.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:29 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:30.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:30 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi097-79636-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:30.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:30 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi097-79992-16"}]': finished 2024-03-21T13:30:30.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:30 smithi097 ceph-mon[30040]: osdmap e154: 8 total, 8 up, 8 in 2024-03-21T13:30:30.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:30 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:30.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:30 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2985707049' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi097-79910-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:30.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:30 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi097-79910-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:30.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:30 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1113874615' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21"}]: dispatch 2024-03-21T13:30:30.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:30 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21"}]: dispatch 2024-03-21T13:30:30.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:30 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1113874615' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21"}]: dispatch 2024-03-21T13:30:30.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:30 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21"}]: dispatch 2024-03-21T13:30:30.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:30 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1113874615' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:30:30.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:30 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:30:30.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:30 smithi097 ceph-mon[30040]: pgmap v195: 420 pgs: 32 creating+peering, 32 unknown, 356 active+clean; 592 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:30:30.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:30 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi097-79636-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:30.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:30 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi097-79992-16"}]': finished 2024-03-21T13:30:30.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:30 smithi097 ceph-mon[39416]: osdmap e154: 8 total, 8 up, 8 in 2024-03-21T13:30:30.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:30 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:30.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:30 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2985707049' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi097-79910-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:30.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:30 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi097-79910-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:30.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:30 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1113874615' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21"}]: dispatch 2024-03-21T13:30:30.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:30 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21"}]: dispatch 2024-03-21T13:30:30.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:30 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1113874615' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21"}]: dispatch 2024-03-21T13:30:30.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:30 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21"}]: dispatch 2024-03-21T13:30:30.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:30 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1113874615' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:30:30.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:30 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:30:30.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:30 smithi097 ceph-mon[39416]: pgmap v195: 420 pgs: 32 creating+peering, 32 unknown, 356 active+clean; 592 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:30:30.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:30 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi097-79636-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:30.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:30 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi097-79992-16"}]': finished 2024-03-21T13:30:30.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:30 smithi144 ceph-mon[33633]: osdmap e154: 8 total, 8 up, 8 in 2024-03-21T13:30:30.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:30 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:30.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:30 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2985707049' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi097-79910-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:30.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:30 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi097-79910-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:30.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:30 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1113874615' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21"}]: dispatch 2024-03-21T13:30:30.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:30 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21"}]: dispatch 2024-03-21T13:30:30.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:30 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1113874615' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21"}]: dispatch 2024-03-21T13:30:30.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:30 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21"}]: dispatch 2024-03-21T13:30:30.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:30 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1113874615' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:30:30.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:30 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:30:30.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:30 smithi144 ceph-mon[33633]: pgmap v195: 420 pgs: 32 creating+peering, 32 unknown, 356 active+clean; 592 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:30:31.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:31 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi097-79910-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:31.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:31 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:30:31.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:31 smithi097 ceph-mon[30040]: osdmap e155: 8 total, 8 up, 8 in 2024-03-21T13:30:31.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:31 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2354313536' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi097-79618-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:31.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:31 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1113874615' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21"}]: dispatch 2024-03-21T13:30:31.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:31 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21"}]: dispatch 2024-03-21T13:30:31.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:31 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi097-79618-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:31.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:31 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:31.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:31 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi097-79910-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:31.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:31 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:30:31.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:31 smithi097 ceph-mon[39416]: osdmap e155: 8 total, 8 up, 8 in 2024-03-21T13:30:31.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:31 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2354313536' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi097-79618-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:31.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:31 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1113874615' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21"}]: dispatch 2024-03-21T13:30:31.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:31 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21"}]: dispatch 2024-03-21T13:30:31.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:31 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi097-79618-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:31.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:31 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:31.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:31 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi097-79910-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:31.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:31 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:30:31.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:31 smithi144 ceph-mon[33633]: osdmap e155: 8 total, 8 up, 8 in 2024-03-21T13:30:31.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:31 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2354313536' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi097-79618-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:31.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:31 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1113874615' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21"}]: dispatch 2024-03-21T13:30:31.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:31 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21"}]: dispatch 2024-03-21T13:30:31.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:31 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi097-79618-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:31.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:31 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:32.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:30:31 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:30:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:30:32.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:32 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi097-79618-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:32.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:32 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:32.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:32 smithi097 ceph-mon[30040]: pgmap v198: 420 pgs: 64 unknown, 356 active+clean; 592 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:30:32.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:32 smithi097 ceph-mon[30040]: osdmap e156: 8 total, 8 up, 8 in 2024-03-21T13:30:32.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:32 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2303152085' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi097-79636-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:32.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:32 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi097-79636-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:32.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:32 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:30:32.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:32 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21"}]': finished 2024-03-21T13:30:32.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:32 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi097-79636-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:32.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:32 smithi097 ceph-mon[30040]: osdmap e157: 8 total, 8 up, 8 in 2024-03-21T13:30:32.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:32 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3019804878' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi097-79910-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:32.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:32 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi097-79910-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:32.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:32 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:32.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:32 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1174440965' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi097-79618-27"}]: dispatch 2024-03-21T13:30:32.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:32 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi097-79618-27"}]: dispatch 2024-03-21T13:30:32.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:32 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1174440965' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi097-79618-27"}]: dispatch 2024-03-21T13:30:32.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:32 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi097-79618-27"}]: dispatch 2024-03-21T13:30:32.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:32 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1174440965' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi097-79618-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:30:32.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:32 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi097-79618-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:30:32.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:32 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:30:32.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:32 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:30:32.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:32 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:30:32.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:32 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi097-79618-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:32.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:32 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:32.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:32 smithi097 ceph-mon[39416]: pgmap v198: 420 pgs: 64 unknown, 356 active+clean; 592 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:30:32.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:32 smithi097 ceph-mon[39416]: osdmap e156: 8 total, 8 up, 8 in 2024-03-21T13:30:32.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:32 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2303152085' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi097-79636-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:32.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:32 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi097-79636-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:32.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:32 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:30:32.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:32 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21"}]': finished 2024-03-21T13:30:32.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:32 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi097-79636-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:32.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:32 smithi097 ceph-mon[39416]: osdmap e157: 8 total, 8 up, 8 in 2024-03-21T13:30:32.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:32 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3019804878' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi097-79910-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:32.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:32 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi097-79910-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:32.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:32 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:32.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:32 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1174440965' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi097-79618-27"}]: dispatch 2024-03-21T13:30:32.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:32 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi097-79618-27"}]: dispatch 2024-03-21T13:30:32.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:32 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1174440965' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi097-79618-27"}]: dispatch 2024-03-21T13:30:32.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:32 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi097-79618-27"}]: dispatch 2024-03-21T13:30:32.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:32 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1174440965' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi097-79618-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:30:32.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:32 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi097-79618-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:30:32.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:32 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:30:32.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:32 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:30:32.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:32 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:30:32.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:32 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi097-79618-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:32.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:32 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:32.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:32 smithi144 ceph-mon[33633]: pgmap v198: 420 pgs: 64 unknown, 356 active+clean; 592 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:30:32.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:32 smithi144 ceph-mon[33633]: osdmap e156: 8 total, 8 up, 8 in 2024-03-21T13:30:32.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:32 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2303152085' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi097-79636-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:32.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:32 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi097-79636-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:32.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:32 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:30:32.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:32 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21"}]': finished 2024-03-21T13:30:32.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:32 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi097-79636-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:32.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:32 smithi144 ceph-mon[33633]: osdmap e157: 8 total, 8 up, 8 in 2024-03-21T13:30:32.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:32 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3019804878' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi097-79910-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:32.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:32 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi097-79910-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:32.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:32 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:32.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:32 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1174440965' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi097-79618-27"}]: dispatch 2024-03-21T13:30:32.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:32 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi097-79618-27"}]: dispatch 2024-03-21T13:30:32.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:32 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1174440965' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi097-79618-27"}]: dispatch 2024-03-21T13:30:32.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:32 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi097-79618-27"}]: dispatch 2024-03-21T13:30:32.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:32 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1174440965' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi097-79618-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:30:32.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:32 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi097-79618-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:30:32.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:32 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:30:32.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:32 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:30:32.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:32 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:30:33.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:33 smithi097 ceph-mon[39416]: pgmap v200: 428 pgs: 72 unknown, 356 active+clean; 592 KiB data, 614 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:30:33.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:33 smithi097 ceph-mon[39416]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-03-21T13:30:33.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:33 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi097-79910-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:33.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:33 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi097-79618-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:30:33.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:33 smithi097 ceph-mon[39416]: osdmap e158: 8 total, 8 up, 8 in 2024-03-21T13:30:33.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:33 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1174440965' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi097-79618-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi097-79618-27"}]: dispatch 2024-03-21T13:30:33.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:33 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi097-79618-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi097-79618-27"}]: dispatch 2024-03-21T13:30:33.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:33 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:33.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:33 smithi097 ceph-mon[30040]: pgmap v200: 428 pgs: 72 unknown, 356 active+clean; 592 KiB data, 614 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:30:33.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:33 smithi097 ceph-mon[30040]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-03-21T13:30:33.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:33 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi097-79910-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:33.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:33 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi097-79618-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:30:33.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:33 smithi097 ceph-mon[30040]: osdmap e158: 8 total, 8 up, 8 in 2024-03-21T13:30:33.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:33 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1174440965' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi097-79618-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi097-79618-27"}]: dispatch 2024-03-21T13:30:33.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:33 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi097-79618-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi097-79618-27"}]: dispatch 2024-03-21T13:30:33.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:33 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:33.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:33 smithi144 ceph-mon[33633]: pgmap v200: 428 pgs: 72 unknown, 356 active+clean; 592 KiB data, 614 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:30:33.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:33 smithi144 ceph-mon[33633]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-03-21T13:30:33.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:33 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi097-79910-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:33.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:33 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi097-79618-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:30:33.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:33 smithi144 ceph-mon[33633]: osdmap e158: 8 total, 8 up, 8 in 2024-03-21T13:30:33.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:33 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1174440965' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi097-79618-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi097-79618-27"}]: dispatch 2024-03-21T13:30:33.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:33 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi097-79618-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi097-79618-27"}]: dispatch 2024-03-21T13:30:33.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:33 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:33.872 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:30:33 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:30:35.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:34 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:30:35.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:34 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:35.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:34 smithi097 ceph-mon[39416]: osdmap e159: 8 total, 8 up, 8 in 2024-03-21T13:30:35.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:34 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1568681038' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi097-79636-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:35.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:34 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:30:35.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:34 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:35.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:34 smithi097 ceph-mon[30040]: osdmap e159: 8 total, 8 up, 8 in 2024-03-21T13:30:35.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:34 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1568681038' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi097-79636-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:35.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:34 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:30:35.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:34 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:35.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:34 smithi144 ceph-mon[33633]: osdmap e159: 8 total, 8 up, 8 in 2024-03-21T13:30:35.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:34 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1568681038' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi097-79636-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:36.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:35 smithi097 ceph-mon[39416]: pgmap v203: 396 pgs: 32 unknown, 364 active+clean; 600 KiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 2.5 KiB/s wr, 1 op/s 2024-03-21T13:30:36.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:35 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi097-79618-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi097-79618-27"}]': finished 2024-03-21T13:30:36.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:35 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1568681038' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi097-79636-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:36.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:35 smithi097 ceph-mon[39416]: osdmap e160: 8 total, 8 up, 8 in 2024-03-21T13:30:36.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:35 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4060997953' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi097-79910-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:36.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:35 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi097-79910-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:36.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:35 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:36.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:35 smithi097 ceph-mon[30040]: pgmap v203: 396 pgs: 32 unknown, 364 active+clean; 600 KiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 2.5 KiB/s wr, 1 op/s 2024-03-21T13:30:36.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:35 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi097-79618-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi097-79618-27"}]': finished 2024-03-21T13:30:36.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:35 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1568681038' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi097-79636-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:36.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:35 smithi097 ceph-mon[30040]: osdmap e160: 8 total, 8 up, 8 in 2024-03-21T13:30:36.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:35 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4060997953' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi097-79910-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:36.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:35 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi097-79910-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:36.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:35 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:36.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:35 smithi144 ceph-mon[33633]: pgmap v203: 396 pgs: 32 unknown, 364 active+clean; 600 KiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 2.5 KiB/s wr, 1 op/s 2024-03-21T13:30:36.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:35 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi097-79618-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi097-79618-27"}]': finished 2024-03-21T13:30:36.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:35 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1568681038' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi097-79636-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:36.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:35 smithi144 ceph-mon[33633]: osdmap e160: 8 total, 8 up, 8 in 2024-03-21T13:30:36.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:35 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4060997953' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi097-79910-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:36.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:35 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi097-79910-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:36.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:35 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:37.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:37 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi097-79910-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:37.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:37 smithi097 ceph-mon[39416]: osdmap e161: 8 total, 8 up, 8 in 2024-03-21T13:30:37.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:37 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:37.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:37 smithi097 ceph-mon[39416]: pgmap v206: 404 pgs: 40 unknown, 364 active+clean; 600 KiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-03-21T13:30:37.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:37 smithi097 ceph-mon[39416]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:30:37.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:37 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi097-79910-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:37.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:37 smithi097 ceph-mon[30040]: osdmap e161: 8 total, 8 up, 8 in 2024-03-21T13:30:37.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:37 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:37.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:37 smithi097 ceph-mon[30040]: pgmap v206: 404 pgs: 40 unknown, 364 active+clean; 600 KiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-03-21T13:30:37.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:37 smithi097 ceph-mon[30040]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:30:37.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:37 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi097-79910-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:37.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:37 smithi144 ceph-mon[33633]: osdmap e161: 8 total, 8 up, 8 in 2024-03-21T13:30:37.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:37 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:37.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:37 smithi144 ceph-mon[33633]: pgmap v206: 404 pgs: 40 unknown, 364 active+clean; 600 KiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-03-21T13:30:37.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:37 smithi144 ceph-mon[33633]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:30:38.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:38 smithi097 ceph-mon[39416]: osdmap e162: 8 total, 8 up, 8 in 2024-03-21T13:30:38.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:38 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:38.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:38 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3452135990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi097-79636-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:38.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:38 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi097-79636-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:38.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:38 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1174440965' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi097-79618-27"}]: dispatch 2024-03-21T13:30:38.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:38 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi097-79618-27"}]: dispatch 2024-03-21T13:30:38.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:38 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1743848932' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi097-79910-36"}]: dispatch 2024-03-21T13:30:38.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:38 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi097-79910-36"}]: dispatch 2024-03-21T13:30:38.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:38 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1743848932' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi097-79910-36"}]: dispatch 2024-03-21T13:30:38.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:38 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi097-79910-36"}]: dispatch 2024-03-21T13:30:38.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:38 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1743848932' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi097-79910-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:30:38.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:38 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi097-79910-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:30:38.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:38 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:30:38.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:38 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:30:38.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:38 smithi097 ceph-mon[30040]: osdmap e162: 8 total, 8 up, 8 in 2024-03-21T13:30:38.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:38 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:38.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:38 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3452135990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi097-79636-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:38.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:38 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi097-79636-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:38.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:38 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1174440965' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi097-79618-27"}]: dispatch 2024-03-21T13:30:38.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:38 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi097-79618-27"}]: dispatch 2024-03-21T13:30:38.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:38 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1743848932' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi097-79910-36"}]: dispatch 2024-03-21T13:30:38.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:38 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi097-79910-36"}]: dispatch 2024-03-21T13:30:38.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:38 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1743848932' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi097-79910-36"}]: dispatch 2024-03-21T13:30:38.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:38 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi097-79910-36"}]: dispatch 2024-03-21T13:30:38.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:38 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1743848932' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi097-79910-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:30:38.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:38 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi097-79910-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:30:38.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:38 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:30:38.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:38 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:30:38.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:38 smithi144 ceph-mon[33633]: osdmap e162: 8 total, 8 up, 8 in 2024-03-21T13:30:38.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:38 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:38.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:38 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3452135990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi097-79636-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:38.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:38 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi097-79636-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:38.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:38 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1174440965' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi097-79618-27"}]: dispatch 2024-03-21T13:30:38.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:38 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi097-79618-27"}]: dispatch 2024-03-21T13:30:38.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:38 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1743848932' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi097-79910-36"}]: dispatch 2024-03-21T13:30:38.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:38 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi097-79910-36"}]: dispatch 2024-03-21T13:30:38.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:38 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1743848932' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi097-79910-36"}]: dispatch 2024-03-21T13:30:38.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:38 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi097-79910-36"}]: dispatch 2024-03-21T13:30:38.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:38 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1743848932' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi097-79910-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:30:38.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:38 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi097-79910-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:30:38.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:38 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:30:38.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:38 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:30:39.289 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: Running main() from gmock_main.cc 2024-03-21T13:30:39.289 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [==========] Running 42 tests from 2 test suites. 2024-03-21T13:30:39.289 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [----------] Global test environment set-up. 2024-03-21T13:30:39.289 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [----------] 26 tests from LibRadosAio 2024-03-21T13:30:39.289 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ RUN ] LibRadosAio.TooBig 2024-03-21T13:30:39.289 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ OK ] LibRadosAio.TooBig (10418 ms) 2024-03-21T13:30:39.289 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ RUN ] LibRadosAio.SimpleWrite 2024-03-21T13:30:39.289 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ OK ] LibRadosAio.SimpleWrite (6946 ms) 2024-03-21T13:30:39.290 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ RUN ] LibRadosAio.WaitForSafe 2024-03-21T13:30:39.290 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ OK ] LibRadosAio.WaitForSafe (3614 ms) 2024-03-21T13:30:39.290 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip 2024-03-21T13:30:39.290 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip (2879 ms) 2024-03-21T13:30:39.290 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip2 2024-03-21T13:30:39.290 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip2 (2799 ms) 2024-03-21T13:30:39.290 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip3 2024-03-21T13:30:39.290 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip3 (5119 ms) 2024-03-21T13:30:39.290 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripAppend 2024-03-21T13:30:39.290 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ OK ] LibRadosAio.RoundTripAppend (3030 ms) 2024-03-21T13:30:39.290 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ RUN ] LibRadosAio.RemoveTest 2024-03-21T13:30:39.290 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ OK ] LibRadosAio.RemoveTest (2995 ms) 2024-03-21T13:30:39.291 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ RUN ] LibRadosAio.XattrsRoundTrip 2024-03-21T13:30:39.291 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ OK ] LibRadosAio.XattrsRoundTrip (3036 ms) 2024-03-21T13:30:39.291 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ RUN ] LibRadosAio.RmXattr 2024-03-21T13:30:39.291 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ OK ] LibRadosAio.RmXattr (3232 ms) 2024-03-21T13:30:39.291 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ RUN ] LibRadosAio.XattrIter 2024-03-21T13:30:39.291 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ OK ] LibRadosAio.XattrIter (3741 ms) 2024-03-21T13:30:39.291 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ RUN ] LibRadosAio.IsComplete 2024-03-21T13:30:39.291 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ OK ] LibRadosAio.IsComplete (3007 ms) 2024-03-21T13:30:39.291 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ RUN ] LibRadosAio.IsSafe 2024-03-21T13:30:39.291 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ OK ] LibRadosAio.IsSafe (2996 ms) 2024-03-21T13:30:39.291 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ RUN ] LibRadosAio.ReturnValue 2024-03-21T13:30:39.291 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ OK ] LibRadosAio.ReturnValue (3016 ms) 2024-03-21T13:30:39.291 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ RUN ] LibRadosAio.Flush 2024-03-21T13:30:39.291 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ OK ] LibRadosAio.Flush (3005 ms) 2024-03-21T13:30:39.292 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ RUN ] LibRadosAio.FlushAsync 2024-03-21T13:30:39.292 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ OK ] LibRadosAio.FlushAsync (3003 ms) 2024-03-21T13:30:39.292 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteFull 2024-03-21T13:30:39.292 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteFull (3015 ms) 2024-03-21T13:30:39.292 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteSame 2024-03-21T13:30:39.292 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteSame (2990 ms) 2024-03-21T13:30:39.292 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStat 2024-03-21T13:30:39.292 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ OK ] LibRadosAio.SimpleStat (2987 ms) 2024-03-21T13:30:39.292 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ RUN ] LibRadosAio.OperateMtime 2024-03-21T13:30:39.292 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ OK ] LibRadosAio.OperateMtime (3037 ms) 2024-03-21T13:30:39.292 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ RUN ] LibRadosAio.Operate2Mtime 2024-03-21T13:30:39.292 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ OK ] LibRadosAio.Operate2Mtime (2943 ms) 2024-03-21T13:30:39.292 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStatNS 2024-03-21T13:30:39.293 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ OK ] LibRadosAio.SimpleStatNS (3259 ms) 2024-03-21T13:30:39.293 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ RUN ] LibRadosAio.StatRemove 2024-03-21T13:30:39.293 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ OK ] LibRadosAio.StatRemove (2781 ms) 2024-03-21T13:30:39.293 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ RUN ] LibRadosAio.ExecuteClass 2024-03-21T13:30:39.293 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ OK ] LibRadosAio.ExecuteClass (3013 ms) 2024-03-21T13:30:39.293 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ RUN ] LibRadosAio.MultiWrite 2024-03-21T13:30:39.293 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ OK ] LibRadosAio.MultiWrite (2974 ms) 2024-03-21T13:30:39.293 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ RUN ] LibRadosAio.AioUnlock 2024-03-21T13:30:39.293 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ OK ] LibRadosAio.AioUnlock (3008 ms) 2024-03-21T13:30:39.293 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [----------] 26 tests from LibRadosAio (92843 ms total) 2024-03-21T13:30:39.293 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: 2024-03-21T13:30:39.293 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [----------] 16 tests from LibRadosAioEC 2024-03-21T13:30:39.294 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleWrite 2024-03-21T13:30:39.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:39 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi097-79636-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:39.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:39 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi097-79618-27"}]': finished 2024-03-21T13:30:39.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:39 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi097-79910-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:30:39.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:39 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:30:39.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:39 smithi097 ceph-mon[30040]: osdmap e163: 8 total, 8 up, 8 in 2024-03-21T13:30:39.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:39 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1743848932' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi097-79910-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi097-79910-36"}]: dispatch 2024-03-21T13:30:39.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:39 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-13"}]: dispatch 2024-03-21T13:30:39.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:39 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi097-79910-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi097-79910-36"}]: dispatch 2024-03-21T13:30:39.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:39 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1174440965' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi097-79618-27"}]: dispatch 2024-03-21T13:30:39.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:39 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-13"}]: dispatch 2024-03-21T13:30:39.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:39 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:39.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:39 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi097-79618-27"}]: dispatch 2024-03-21T13:30:39.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:39 smithi097 ceph-mon[30040]: pgmap v209: 396 pgs: 2 creating+activating, 394 active+clean; 610 KiB data, 628 MiB used, 715 GiB / 715 GiB avail; 11 KiB/s rd, 10 KiB/s wr, 27 op/s 2024-03-21T13:30:39.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:39 smithi097 ceph-mon[30040]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:30:39.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:39 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-13"}]': finished 2024-03-21T13:30:39.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:39 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi097-79636-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:39.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:39 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi097-79618-27"}]': finished 2024-03-21T13:30:39.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:39 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi097-79910-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:30:39.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:39 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:30:39.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:39 smithi097 ceph-mon[39416]: osdmap e163: 8 total, 8 up, 8 in 2024-03-21T13:30:39.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:39 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1743848932' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi097-79910-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi097-79910-36"}]: dispatch 2024-03-21T13:30:39.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:39 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-13"}]: dispatch 2024-03-21T13:30:39.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:39 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi097-79910-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi097-79910-36"}]: dispatch 2024-03-21T13:30:39.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:39 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1174440965' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi097-79618-27"}]: dispatch 2024-03-21T13:30:39.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:39 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-13"}]: dispatch 2024-03-21T13:30:39.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:39 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:39.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:39 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi097-79618-27"}]: dispatch 2024-03-21T13:30:39.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:39 smithi097 ceph-mon[39416]: pgmap v209: 396 pgs: 2 creating+activating, 394 active+clean; 610 KiB data, 628 MiB used, 715 GiB / 715 GiB avail; 11 KiB/s rd, 10 KiB/s wr, 27 op/s 2024-03-21T13:30:39.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:39 smithi097 ceph-mon[39416]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:30:39.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:39 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-13"}]': finished 2024-03-21T13:30:39.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:39 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi097-79636-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:39.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:39 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi097-79618-27"}]': finished 2024-03-21T13:30:39.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:39 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi097-79910-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:30:39.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:39 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:30:39.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:39 smithi144 ceph-mon[33633]: osdmap e163: 8 total, 8 up, 8 in 2024-03-21T13:30:39.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:39 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1743848932' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi097-79910-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi097-79910-36"}]: dispatch 2024-03-21T13:30:39.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:39 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-13"}]: dispatch 2024-03-21T13:30:39.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:39 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi097-79910-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi097-79910-36"}]: dispatch 2024-03-21T13:30:39.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:39 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1174440965' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi097-79618-27"}]: dispatch 2024-03-21T13:30:39.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:39 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-13"}]: dispatch 2024-03-21T13:30:39.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:39 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:39.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:39 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi097-79618-27"}]: dispatch 2024-03-21T13:30:39.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:39 smithi144 ceph-mon[33633]: pgmap v209: 396 pgs: 2 creating+activating, 394 active+clean; 610 KiB data, 628 MiB used, 715 GiB / 715 GiB avail; 11 KiB/s rd, 10 KiB/s wr, 27 op/s 2024-03-21T13:30:39.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:39 smithi144 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:30:39.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:39 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-13"}]': finished 2024-03-21T13:30:40.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:40 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi097-79618-27"}]': finished 2024-03-21T13:30:40.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:40 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:40.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:40 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2193329488' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi097-79618-28"}]: dispatch 2024-03-21T13:30:40.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:40 smithi097 ceph-mon[39416]: osdmap e164: 8 total, 8 up, 8 in 2024-03-21T13:30:40.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:40 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi097-79618-28"}]: dispatch 2024-03-21T13:30:40.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:40 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2193329488' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi097-79618-28"}]: dispatch 2024-03-21T13:30:40.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:40 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi097-79618-28"}]: dispatch 2024-03-21T13:30:40.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:40 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2193329488' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi097-79618-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:30:40.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:40 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi097-79618-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:30:40.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:40 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:30:40.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:40 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:30:40.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:40 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi097-79910-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi097-79910-36"}]': finished 2024-03-21T13:30:40.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:40 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi097-79618-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:30:40.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:40 smithi097 ceph-mon[39416]: osdmap e165: 8 total, 8 up, 8 in 2024-03-21T13:30:40.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:40 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2193329488' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi097-79618-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi097-79618-28"}]: dispatch 2024-03-21T13:30:40.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:40 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi097-79618-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi097-79618-28"}]: dispatch 2024-03-21T13:30:40.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:40 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2000600795' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi097-79636-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:40.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:40 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi097-79618-27"}]': finished 2024-03-21T13:30:40.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:40 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:40.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:40 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2193329488' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi097-79618-28"}]: dispatch 2024-03-21T13:30:40.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:40 smithi097 ceph-mon[30040]: osdmap e164: 8 total, 8 up, 8 in 2024-03-21T13:30:40.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:40 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi097-79618-28"}]: dispatch 2024-03-21T13:30:40.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:40 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2193329488' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi097-79618-28"}]: dispatch 2024-03-21T13:30:40.806 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:40 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi097-79618-28"}]: dispatch 2024-03-21T13:30:40.806 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:40 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2193329488' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi097-79618-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:30:40.806 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:40 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi097-79618-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:30:40.806 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:40 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:30:40.806 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:40 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:30:40.806 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:40 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi097-79910-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi097-79910-36"}]': finished 2024-03-21T13:30:40.806 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:40 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi097-79618-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:30:40.806 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:40 smithi097 ceph-mon[30040]: osdmap e165: 8 total, 8 up, 8 in 2024-03-21T13:30:40.806 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:40 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2193329488' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi097-79618-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi097-79618-28"}]: dispatch 2024-03-21T13:30:40.806 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:40 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi097-79618-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi097-79618-28"}]: dispatch 2024-03-21T13:30:40.806 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:40 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2000600795' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi097-79636-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:40.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:40 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi097-79618-27"}]': finished 2024-03-21T13:30:40.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:40 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:40.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:40 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2193329488' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi097-79618-28"}]: dispatch 2024-03-21T13:30:40.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:40 smithi144 ceph-mon[33633]: osdmap e164: 8 total, 8 up, 8 in 2024-03-21T13:30:40.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:40 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi097-79618-28"}]: dispatch 2024-03-21T13:30:40.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:40 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2193329488' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi097-79618-28"}]: dispatch 2024-03-21T13:30:40.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:40 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi097-79618-28"}]: dispatch 2024-03-21T13:30:40.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:40 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2193329488' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi097-79618-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:30:40.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:40 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi097-79618-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:30:40.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:40 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:30:40.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:40 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:30:40.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:40 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi097-79910-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi097-79910-36"}]': finished 2024-03-21T13:30:40.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:40 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi097-79618-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:30:40.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:40 smithi144 ceph-mon[33633]: osdmap e165: 8 total, 8 up, 8 in 2024-03-21T13:30:40.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:40 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2193329488' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi097-79618-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi097-79618-28"}]: dispatch 2024-03-21T13:30:40.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:40 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi097-79618-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi097-79618-28"}]: dispatch 2024-03-21T13:30:40.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:40 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2000600795' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi097-79636-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:41.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:41 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:41.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:41 smithi097 ceph-mon[39416]: pgmap v212: 372 pgs: 40 unknown, 332 active+clean; 610 KiB data, 628 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s wr, 1 op/s 2024-03-21T13:30:41.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:41 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi097-79636-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:41.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:41 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi097-79636-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:41.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:41 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:41.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:41 smithi097 ceph-mon[30040]: pgmap v212: 372 pgs: 40 unknown, 332 active+clean; 610 KiB data, 628 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s wr, 1 op/s 2024-03-21T13:30:41.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:41 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi097-79636-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:41.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:41 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi097-79636-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:41.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:41 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:41.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:41 smithi144 ceph-mon[33633]: pgmap v212: 372 pgs: 40 unknown, 332 active+clean; 610 KiB data, 628 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s wr, 1 op/s 2024-03-21T13:30:41.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:41 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi097-79636-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:41.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:41 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi097-79636-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:42.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:30:41 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:30:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:30:42.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:42 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:42.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:42 smithi144 ceph-mon[33633]: osdmap e166: 8 total, 8 up, 8 in 2024-03-21T13:30:42.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:42 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:42.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:42 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:42.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:42 smithi144 ceph-mon[33633]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:30:42.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:42 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi097-79618-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi097-79618-28"}]': finished 2024-03-21T13:30:42.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:42 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:42.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:42 smithi144 ceph-mon[33633]: osdmap e167: 8 total, 8 up, 8 in 2024-03-21T13:30:42.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:42 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1743848932' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi097-79910-36"}]: dispatch 2024-03-21T13:30:42.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:42 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi097-79910-36"}]: dispatch 2024-03-21T13:30:42.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:42 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:43.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:42 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:43.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:42 smithi097 ceph-mon[30040]: osdmap e166: 8 total, 8 up, 8 in 2024-03-21T13:30:43.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:42 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:43.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:42 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:43.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:42 smithi097 ceph-mon[30040]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:30:43.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:42 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi097-79618-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi097-79618-28"}]': finished 2024-03-21T13:30:43.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:42 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:43.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:42 smithi097 ceph-mon[30040]: osdmap e167: 8 total, 8 up, 8 in 2024-03-21T13:30:43.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:42 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1743848932' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi097-79910-36"}]: dispatch 2024-03-21T13:30:43.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:42 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi097-79910-36"}]: dispatch 2024-03-21T13:30:43.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:42 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:43.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:42 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:43.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:42 smithi097 ceph-mon[39416]: osdmap e166: 8 total, 8 up, 8 in 2024-03-21T13:30:43.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:42 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:43.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:42 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:43.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:42 smithi097 ceph-mon[39416]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:30:43.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:42 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi097-79618-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi097-79618-28"}]': finished 2024-03-21T13:30:43.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:42 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:43.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:42 smithi097 ceph-mon[39416]: osdmap e167: 8 total, 8 up, 8 in 2024-03-21T13:30:43.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:42 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1743848932' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi097-79910-36"}]: dispatch 2024-03-21T13:30:43.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:42 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi097-79910-36"}]: dispatch 2024-03-21T13:30:43.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:42 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:43.772 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:30:43 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:30:43.772 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:43 smithi144 ceph-mon[33633]: pgmap v215: 372 pgs: 40 unknown, 332 active+clean; 610 KiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:30:43.773 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:43 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi097-79910-36"}]': finished 2024-03-21T13:30:43.773 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:43 smithi144 ceph-mon[33633]: osdmap e168: 8 total, 8 up, 8 in 2024-03-21T13:30:43.773 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:43 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1743848932' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi097-79910-36"}]: dispatch 2024-03-21T13:30:43.773 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:43 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi097-79910-36"}]: dispatch 2024-03-21T13:30:43.773 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:43 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1632308273' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi097-79636-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:43.773 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:43 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi097-79636-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:43.773 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:43 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:30:43.773 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:43 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:30:43.773 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:43 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:44.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:43 smithi097 ceph-mon[39416]: pgmap v215: 372 pgs: 40 unknown, 332 active+clean; 610 KiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:30:44.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:43 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi097-79910-36"}]': finished 2024-03-21T13:30:44.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:43 smithi097 ceph-mon[39416]: osdmap e168: 8 total, 8 up, 8 in 2024-03-21T13:30:44.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:43 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1743848932' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi097-79910-36"}]: dispatch 2024-03-21T13:30:44.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:43 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi097-79910-36"}]: dispatch 2024-03-21T13:30:44.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:43 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1632308273' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi097-79636-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:44.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:43 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi097-79636-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:44.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:43 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:30:44.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:43 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:30:44.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:43 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:44.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:43 smithi097 ceph-mon[30040]: pgmap v215: 372 pgs: 40 unknown, 332 active+clean; 610 KiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:30:44.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:43 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi097-79910-36"}]': finished 2024-03-21T13:30:44.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:43 smithi097 ceph-mon[30040]: osdmap e168: 8 total, 8 up, 8 in 2024-03-21T13:30:44.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:43 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1743848932' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi097-79910-36"}]: dispatch 2024-03-21T13:30:44.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:43 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi097-79910-36"}]: dispatch 2024-03-21T13:30:44.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:43 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1632308273' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi097-79636-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:44.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:43 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi097-79636-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:44.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:43 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:30:44.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:43 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:30:44.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:43 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:44.256 INFO:tasks.workunit.client.0.smithi097.stdout:(68017 ms total) 2024-03-21T13:30:44.256 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: 2024-03-21T13:30:44.256 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP 2024-03-21T13:30:44.256 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ RUN ] LibRadosTwoPoolsECPP.CopyFrom 2024-03-21T13:30:44.256 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ OK ] LibRadosTwoPoolsECPP.CopyFrom (103 ms) 2024-03-21T13:30:44.256 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP (103 ms total) 2024-03-21T13:30:44.256 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: 2024-03-21T13:30:44.256 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)0, Checksummer::xxhash32, ceph_le > 2024-03-21T13:30:44.256 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Subset 2024-03-21T13:30:44.256 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Subset (39 ms) 2024-03-21T13:30:44.257 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Chunked 2024-03-21T13:30:44.257 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Chunked (3 ms) 2024-03-21T13:30:44.257 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0 (42 ms total) 2024-03-21T13:30:44.257 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: 2024-03-21T13:30:44.257 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)1, Checksummer::xxhash64, ceph_le > 2024-03-21T13:30:44.257 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Subset 2024-03-21T13:30:44.257 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Subset (38 ms) 2024-03-21T13:30:44.257 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Chunked 2024-03-21T13:30:44.257 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Chunked (3 ms) 2024-03-21T13:30:44.257 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1 (41 ms total) 2024-03-21T13:30:44.257 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: 2024-03-21T13:30:44.257 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)2, Checksummer::crc32c, ceph_le > 2024-03-21T13:30:44.257 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Subset 2024-03-21T13:30:44.257 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Subset (50 ms) 2024-03-21T13:30:44.257 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Chunked 2024-03-21T13:30:44.258 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Chunked (12 ms) 2024-03-21T13:30:44.258 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2 (63 ms total) 2024-03-21T13:30:44.258 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: 2024-03-21T13:30:44.258 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP 2024-03-21T13:30:44.258 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ RUN ] LibRadosMiscECPP.CompareExtentRange 2024-03-21T13:30:44.258 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ OK ] LibRadosMiscECPP.CompareExtentRange (1024 ms) 2024-03-21T13:30:44.258 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP (1024 ms total) 2024-03-21T13:30:44.258 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: 2024-03-21T13:30:44.258 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [----------] Global test environment tear-down 2024-03-21T13:30:44.258 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [==========] 31 tests from 7 test suites ran. (104685 ms total) 2024-03-21T13:30:44.258 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc_pp: [ PASSED ] 31 tests. 2024-03-21T13:30:44.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:44 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:30:44.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:44 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi097-79910-36"}]': finished 2024-03-21T13:30:44.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:44 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi097-79636-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:44.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:44 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-15", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:30:44.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:44 smithi144 ceph-mon[33633]: osdmap e169: 8 total, 8 up, 8 in 2024-03-21T13:30:44.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:44 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-15"}]: dispatch 2024-03-21T13:30:44.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:44 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-15"}]: dispatch 2024-03-21T13:30:44.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:44 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2193329488' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi097-79618-28"}]: dispatch 2024-03-21T13:30:44.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:44 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi097-79618-28"}]: dispatch 2024-03-21T13:30:44.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:44 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:45.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:44 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:30:45.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:44 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi097-79910-36"}]': finished 2024-03-21T13:30:45.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:44 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi097-79636-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:45.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:44 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-15", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:30:45.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:44 smithi097 ceph-mon[39416]: osdmap e169: 8 total, 8 up, 8 in 2024-03-21T13:30:45.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:44 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-15"}]: dispatch 2024-03-21T13:30:45.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:44 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-15"}]: dispatch 2024-03-21T13:30:45.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:44 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2193329488' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi097-79618-28"}]: dispatch 2024-03-21T13:30:45.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:44 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi097-79618-28"}]: dispatch 2024-03-21T13:30:45.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:44 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:45.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:44 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:30:45.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:44 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi097-79910-36"}]': finished 2024-03-21T13:30:45.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:44 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi097-79636-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:45.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:44 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-15", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:30:45.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:44 smithi097 ceph-mon[30040]: osdmap e169: 8 total, 8 up, 8 in 2024-03-21T13:30:45.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:44 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-15"}]: dispatch 2024-03-21T13:30:45.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:44 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-15"}]: dispatch 2024-03-21T13:30:45.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:44 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2193329488' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi097-79618-28"}]: dispatch 2024-03-21T13:30:45.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:44 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi097-79618-28"}]: dispatch 2024-03-21T13:30:45.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:44 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:45.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:45 smithi144 ceph-mon[33633]: pgmap v218: 396 pgs: 32 creating+peering, 364 active+clean; 610 KiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.5 KiB/s wr, 2 op/s 2024-03-21T13:30:45.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:45 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-15"}]': finished 2024-03-21T13:30:45.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:45 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi097-79618-28"}]': finished 2024-03-21T13:30:45.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:45 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-15", "mode": "writeback"}]: dispatch 2024-03-21T13:30:45.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:45 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2193329488' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi097-79618-28"}]: dispatch 2024-03-21T13:30:45.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:45 smithi144 ceph-mon[33633]: osdmap e170: 8 total, 8 up, 8 in 2024-03-21T13:30:45.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:45 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-15", "mode": "writeback"}]: dispatch 2024-03-21T13:30:45.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:45 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi097-79618-28"}]: dispatch 2024-03-21T13:30:45.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:45 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1113874615' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21"}]: dispatch 2024-03-21T13:30:45.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:45 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21"}]: dispatch 2024-03-21T13:30:45.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:45 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:46.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:45 smithi097 ceph-mon[39416]: pgmap v218: 396 pgs: 32 creating+peering, 364 active+clean; 610 KiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.5 KiB/s wr, 2 op/s 2024-03-21T13:30:46.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:45 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-15"}]': finished 2024-03-21T13:30:46.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:45 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi097-79618-28"}]': finished 2024-03-21T13:30:46.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:45 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-15", "mode": "writeback"}]: dispatch 2024-03-21T13:30:46.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:45 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2193329488' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi097-79618-28"}]: dispatch 2024-03-21T13:30:46.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:45 smithi097 ceph-mon[39416]: osdmap e170: 8 total, 8 up, 8 in 2024-03-21T13:30:46.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:45 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-15", "mode": "writeback"}]: dispatch 2024-03-21T13:30:46.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:45 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi097-79618-28"}]: dispatch 2024-03-21T13:30:46.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:45 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1113874615' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21"}]: dispatch 2024-03-21T13:30:46.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:45 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21"}]: dispatch 2024-03-21T13:30:46.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:45 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:46.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:45 smithi097 ceph-mon[30040]: pgmap v218: 396 pgs: 32 creating+peering, 364 active+clean; 610 KiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.5 KiB/s wr, 2 op/s 2024-03-21T13:30:46.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:45 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-15"}]': finished 2024-03-21T13:30:46.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:45 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi097-79618-28"}]': finished 2024-03-21T13:30:46.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:45 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-15", "mode": "writeback"}]: dispatch 2024-03-21T13:30:46.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:45 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2193329488' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi097-79618-28"}]: dispatch 2024-03-21T13:30:46.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:45 smithi097 ceph-mon[30040]: osdmap e170: 8 total, 8 up, 8 in 2024-03-21T13:30:46.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:45 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-15", "mode": "writeback"}]: dispatch 2024-03-21T13:30:46.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:45 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi097-79618-28"}]: dispatch 2024-03-21T13:30:46.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:45 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1113874615' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21"}]: dispatch 2024-03-21T13:30:46.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:45 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21"}]: dispatch 2024-03-21T13:30:46.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:45 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:46.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:46 smithi144 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:30:46.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:46 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-15", "mode": "writeback"}]': finished 2024-03-21T13:30:46.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:46 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi097-79618-28"}]': finished 2024-03-21T13:30:46.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:46 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21"}]': finished 2024-03-21T13:30:46.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:46 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1113874615' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21"}]: dispatch 2024-03-21T13:30:46.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:46 smithi144 ceph-mon[33633]: osdmap e171: 8 total, 8 up, 8 in 2024-03-21T13:30:46.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:46 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21"}]: dispatch 2024-03-21T13:30:46.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:46 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3182951365' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi097-79636-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:46.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:46 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi097-79636-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:46.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:46 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4244412087' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi097-79618-29"}]: dispatch 2024-03-21T13:30:46.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:46 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi097-79618-29"}]: dispatch 2024-03-21T13:30:46.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:46 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4244412087' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi097-79618-29"}]: dispatch 2024-03-21T13:30:46.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:46 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi097-79618-29"}]: dispatch 2024-03-21T13:30:46.873 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:46 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4244412087' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi097-79618-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:30:46.873 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:46 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi097-79618-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:30:46.873 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:46 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:47.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:46 smithi097 ceph-mon[39416]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:30:47.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:46 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-15", "mode": "writeback"}]': finished 2024-03-21T13:30:47.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:46 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi097-79618-28"}]': finished 2024-03-21T13:30:47.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:46 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21"}]': finished 2024-03-21T13:30:47.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:46 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1113874615' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21"}]: dispatch 2024-03-21T13:30:47.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:46 smithi097 ceph-mon[39416]: osdmap e171: 8 total, 8 up, 8 in 2024-03-21T13:30:47.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:46 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21"}]: dispatch 2024-03-21T13:30:47.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:46 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3182951365' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi097-79636-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:47.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:46 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi097-79636-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:47.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:46 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4244412087' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi097-79618-29"}]: dispatch 2024-03-21T13:30:47.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:46 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi097-79618-29"}]: dispatch 2024-03-21T13:30:47.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:46 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4244412087' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi097-79618-29"}]: dispatch 2024-03-21T13:30:47.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:46 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi097-79618-29"}]: dispatch 2024-03-21T13:30:47.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:46 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4244412087' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi097-79618-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:30:47.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:46 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi097-79618-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:30:47.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:46 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:47.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:46 smithi097 ceph-mon[30040]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:30:47.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:46 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-15", "mode": "writeback"}]': finished 2024-03-21T13:30:47.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:46 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi097-79618-28"}]': finished 2024-03-21T13:30:47.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:46 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21"}]': finished 2024-03-21T13:30:47.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:46 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1113874615' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21"}]: dispatch 2024-03-21T13:30:47.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:46 smithi097 ceph-mon[30040]: osdmap e171: 8 total, 8 up, 8 in 2024-03-21T13:30:47.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:46 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21"}]: dispatch 2024-03-21T13:30:47.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:46 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3182951365' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi097-79636-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:47.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:46 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi097-79636-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:47.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:46 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4244412087' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi097-79618-29"}]: dispatch 2024-03-21T13:30:47.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:46 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi097-79618-29"}]: dispatch 2024-03-21T13:30:47.056 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:46 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4244412087' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi097-79618-29"}]: dispatch 2024-03-21T13:30:47.056 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:46 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi097-79618-29"}]: dispatch 2024-03-21T13:30:47.056 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:46 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4244412087' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi097-79618-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:30:47.056 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:46 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi097-79618-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:30:47.056 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:46 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:47.277 INFO:tasks.workunit.client.0.smithi097.stdout:napshotsECPP.SnapRemovePP (1775 ms) 2024-03-21T13:30:47.278 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.RollbackPP 2024-03-21T13:30:47.278 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.RollbackPP (2013 ms) 2024-03-21T13:30:47.278 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapGetNamePP 2024-03-21T13:30:47.278 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapGetNamePP (2004 ms) 2024-03-21T13:30:47.278 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP (9051 ms total) 2024-03-21T13:30:47.278 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: 2024-03-21T13:30:47.278 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP 2024-03-21T13:30:47.278 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.SnapPP 2024-03-21T13:30:47.278 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.SnapPP (4059 ms) 2024-03-21T13:30:47.278 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.RollbackPP 2024-03-21T13:30:47.279 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.RollbackPP (3968 ms) 2024-03-21T13:30:47.279 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.Bug11677 2024-03-21T13:30:47.279 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.Bug11677 (3953 ms) 2024-03-21T13:30:47.279 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP (11980 ms total) 2024-03-21T13:30:47.279 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: 2024-03-21T13:30:47.279 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [----------] Global test environment tear-down 2024-03-21T13:30:47.279 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [==========] 21 tests from 5 test suites ran. (107721 ms total) 2024-03-21T13:30:47.279 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [ PASSED ] 20 tests. 2024-03-21T13:30:47.279 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [ SKIPPED ] 1 test, listed below: 2024-03-21T13:30:47.279 INFO:tasks.workunit.client.0.smithi097.stdout: api_snapshots_pp: [ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2024-03-21T13:30:47.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:47 smithi144 ceph-mon[33633]: pgmap v221: 388 pgs: 32 unknown, 356 active+clean; 586 KiB data, 633 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:30:47.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:47 smithi144 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:30:47.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:47 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21"}]': finished 2024-03-21T13:30:47.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:47 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi097-79636-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:47.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:47 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi097-79618-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:30:47.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:47 smithi144 ceph-mon[33633]: osdmap e172: 8 total, 8 up, 8 in 2024-03-21T13:30:47.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:47 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4244412087' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi097-79618-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi097-79618-29"}]: dispatch 2024-03-21T13:30:47.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:47 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi097-79618-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi097-79618-29"}]: dispatch 2024-03-21T13:30:47.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:47 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:47.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:47 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:30:47.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:47 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:30:48.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:47 smithi097 ceph-mon[39416]: pgmap v221: 388 pgs: 32 unknown, 356 active+clean; 586 KiB data, 633 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:30:48.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:47 smithi097 ceph-mon[39416]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:30:48.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:47 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21"}]': finished 2024-03-21T13:30:48.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:47 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi097-79636-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:48.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:47 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi097-79618-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:30:48.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:47 smithi097 ceph-mon[39416]: osdmap e172: 8 total, 8 up, 8 in 2024-03-21T13:30:48.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:47 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4244412087' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi097-79618-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi097-79618-29"}]: dispatch 2024-03-21T13:30:48.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:47 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi097-79618-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi097-79618-29"}]: dispatch 2024-03-21T13:30:48.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:47 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:48.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:47 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:30:48.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:47 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:30:48.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:47 smithi097 ceph-mon[30040]: pgmap v221: 388 pgs: 32 unknown, 356 active+clean; 586 KiB data, 633 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:30:48.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:47 smithi097 ceph-mon[30040]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:30:48.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:47 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi097-79992-21"}]': finished 2024-03-21T13:30:48.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:47 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi097-79636-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:48.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:47 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi097-79618-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:30:48.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:47 smithi097 ceph-mon[30040]: osdmap e172: 8 total, 8 up, 8 in 2024-03-21T13:30:48.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:47 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4244412087' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi097-79618-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi097-79618-29"}]: dispatch 2024-03-21T13:30:48.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:47 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi097-79618-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi097-79618-29"}]: dispatch 2024-03-21T13:30:48.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:47 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:48.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:47 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:30:48.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:47 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:30:48.508 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: Running main() from gmock_main.cc 2024-03-21T13:30:48.508 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [==========] Running 57 tests from 4 test suites. 2024-03-21T13:30:48.508 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [----------] Global test environment set-up. 2024-03-21T13:30:48.508 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio 2024-03-21T13:30:48.508 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAio.TooBigPP 2024-03-21T13:30:48.508 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAio.TooBigPP (10418 ms) 2024-03-21T13:30:48.508 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolQuotaPP 2024-03-21T13:30:48.508 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolQuotaPP (19321 ms) 2024-03-21T13:30:48.508 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleWritePP 2024-03-21T13:30:48.508 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleWritePP (6063 ms) 2024-03-21T13:30:48.508 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAio.WaitForSafePP 2024-03-21T13:30:48.508 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAio.WaitForSafePP (2996 ms) 2024-03-21T13:30:48.508 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP 2024-03-21T13:30:48.509 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP (3035 ms) 2024-03-21T13:30:48.509 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP2 2024-03-21T13:30:48.509 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP2 (3017 ms) 2024-03-21T13:30:48.509 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP3 2024-03-21T13:30:48.509 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP3 (3948 ms) 2024-03-21T13:30:48.509 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripSparseReadPP 2024-03-21T13:30:48.509 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripSparseReadPP (3015 ms) 2024-03-21T13:30:48.509 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsCompletePP 2024-03-21T13:30:48.509 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAio.IsCompletePP (3000 ms) 2024-03-21T13:30:48.509 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsSafePP 2024-03-21T13:30:48.509 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAio.IsSafePP (2990 ms) 2024-03-21T13:30:48.509 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAio.ReturnValuePP 2024-03-21T13:30:48.509 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAio.ReturnValuePP (3019 ms) 2024-03-21T13:30:48.509 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushPP 2024-03-21T13:30:48.509 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushPP (3009 ms) 2024-03-21T13:30:48.509 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushAsyncPP 2024-03-21T13:30:48.510 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushAsyncPP (3017 ms) 2024-03-21T13:30:48.510 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP 2024-03-21T13:30:48.510 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP (2961 ms) 2024-03-21T13:30:48.510 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP2 2024-03-21T13:30:48.510 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP2 (3049 ms) 2024-03-21T13:30:48.510 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP 2024-03-21T13:30:48.510 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP (3009 ms) 2024-03-21T13:30:48.510 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP2 2024-03-21T13:30:48.510 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP2 (2949 ms) 2024-03-21T13:30:48.510 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPPNS 2024-03-21T13:30:48.510 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPPNS (3011 ms) 2024-03-21T13:30:48.510 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPP 2024-03-21T13:30:48.510 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPP (3021 ms) 2024-03-21T13:30:48.511 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime 2024-03-21T13:30:48.511 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime (2961 ms) 2024-03-21T13:30:48.511 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime2 2024-03-21T13:30:48.511 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime2 (3021 ms) 2024-03-21T13:30:48.511 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAio.StatRemovePP 2024-03-21T13:30:48.511 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAio.StatRemovePP (3009 ms) 2024-03-21T13:30:48.511 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAio.ExecuteClassPP 2024-03-21T13:30:48.511 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAio.ExecuteClassPP (3013 ms) 2024-03-21T13:30:48.511 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAio.OmapPP 2024-03-21T13:30:48.511 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAio.OmapPP (3248 ms) 2024-03-21T13:30:48.511 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiWritePP 2024-03-21T13:30:48.511 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiWritePP (2720 ms) 2024-03-21T13:30:48.511 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAio.AioUnlockPP 2024-03-21T13:30:48.511 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAio.AioUnlockPP (3009 ms) 2024-03-21T13:30:48.511 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripAppendPP 2024-03-21T13:30:49.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:49 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:30:49.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:49 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-15"}]: dispatch 2024-03-21T13:30:49.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:49 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:49.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:49 smithi097 ceph-mon[39416]: pgmap v224: 356 pgs: 356 active+clean; 586 KiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:30:49.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:49 smithi097 ceph-mon[39416]: osdmap e173: 8 total, 8 up, 8 in 2024-03-21T13:30:49.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:49 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-15"}]: dispatch 2024-03-21T13:30:49.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:49 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:30:49.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:49 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-15"}]: dispatch 2024-03-21T13:30:49.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:49 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:49.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:49 smithi097 ceph-mon[30040]: pgmap v224: 356 pgs: 356 active+clean; 586 KiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:30:49.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:49 smithi097 ceph-mon[30040]: osdmap e173: 8 total, 8 up, 8 in 2024-03-21T13:30:49.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:49 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-15"}]: dispatch 2024-03-21T13:30:49.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:49 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:30:49.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:49 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-15"}]: dispatch 2024-03-21T13:30:49.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:49 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:49.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:49 smithi144 ceph-mon[33633]: pgmap v224: 356 pgs: 356 active+clean; 586 KiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:30:49.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:49 smithi144 ceph-mon[33633]: osdmap e173: 8 total, 8 up, 8 in 2024-03-21T13:30:49.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:49 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-15"}]: dispatch 2024-03-21T13:30:50.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:50 smithi097 ceph-mon[30040]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:30:50.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:50 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi097-79618-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi097-79618-29"}]': finished 2024-03-21T13:30:50.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:50 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-15"}]': finished 2024-03-21T13:30:50.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:50 smithi097 ceph-mon[30040]: osdmap e174: 8 total, 8 up, 8 in 2024-03-21T13:30:50.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:50 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3689155111' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi097-79636-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:50.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:50 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi097-79636-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:50.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:50 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:50.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:50 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi097-79710-1","var":"pg_num","val":"11"}]: dispatch 2024-03-21T13:30:50.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:50 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi097-79710-1","var":"pg_num","val":"11"}]: dispatch 2024-03-21T13:30:50.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:50 smithi097 ceph-mon[39416]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:30:50.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:50 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi097-79618-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi097-79618-29"}]': finished 2024-03-21T13:30:50.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:50 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-15"}]': finished 2024-03-21T13:30:50.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:50 smithi097 ceph-mon[39416]: osdmap e174: 8 total, 8 up, 8 in 2024-03-21T13:30:50.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:50 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3689155111' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi097-79636-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:50.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:50 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi097-79636-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:50.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:50 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:50.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:50 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi097-79710-1","var":"pg_num","val":"11"}]: dispatch 2024-03-21T13:30:50.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:50 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi097-79710-1","var":"pg_num","val":"11"}]: dispatch 2024-03-21T13:30:50.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:50 smithi144 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:30:50.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:50 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi097-79618-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi097-79618-29"}]': finished 2024-03-21T13:30:50.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:50 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-15"}]': finished 2024-03-21T13:30:50.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:50 smithi144 ceph-mon[33633]: osdmap e174: 8 total, 8 up, 8 in 2024-03-21T13:30:50.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:50 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3689155111' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi097-79636-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:50.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:50 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi097-79636-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:50.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:50 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:50.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:50 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi097-79710-1","var":"pg_num","val":"11"}]: dispatch 2024-03-21T13:30:50.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:50 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi097-79710-1","var":"pg_num","val":"11"}]: dispatch 2024-03-21T13:30:51.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:51 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi097-79636-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:51.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:51 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi097-79710-1","var":"pg_num","val":"11"}]': finished 2024-03-21T13:30:51.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:51 smithi097 ceph-mon[39416]: osdmap e175: 8 total, 8 up, 8 in 2024-03-21T13:30:51.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:51 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:51.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:51 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:51.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:51 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi097-79710-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-21T13:30:51.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:51 smithi097 ceph-mon[39416]: pgmap v227: 364 pgs: 40 unknown, 324 active+clean; 586 KiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:30:51.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:51 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi097-79710-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-21T13:30:51.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:51 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi097-79636-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:51.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:51 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi097-79710-1","var":"pg_num","val":"11"}]': finished 2024-03-21T13:30:51.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:51 smithi097 ceph-mon[30040]: osdmap e175: 8 total, 8 up, 8 in 2024-03-21T13:30:51.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:51 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:51.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:51 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:51.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:51 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi097-79710-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-21T13:30:51.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:51 smithi097 ceph-mon[30040]: pgmap v227: 364 pgs: 40 unknown, 324 active+clean; 586 KiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:30:51.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:51 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi097-79710-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-21T13:30:51.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:51 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi097-79636-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:51.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:51 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi097-79710-1","var":"pg_num","val":"11"}]': finished 2024-03-21T13:30:51.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:51 smithi144 ceph-mon[33633]: osdmap e175: 8 total, 8 up, 8 in 2024-03-21T13:30:51.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:51 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:51.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:51 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:51.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:51 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi097-79710-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-21T13:30:51.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:51 smithi144 ceph-mon[33633]: pgmap v227: 364 pgs: 40 unknown, 324 active+clean; 586 KiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:30:51.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:51 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi097-79710-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-21T13:30:52.287 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:30:51 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:30:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:30:52.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:52 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi097-79710-1","var":"pgp_num","val":"11"}]': finished 2024-03-21T13:30:52.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:52 smithi097 ceph-mon[39416]: osdmap e176: 8 total, 8 up, 8 in 2024-03-21T13:30:52.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:52 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:52.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:52 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:52.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:52 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4244412087' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi097-79618-29"}]: dispatch 2024-03-21T13:30:52.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:52 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:52.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:52 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi097-79618-29"}]: dispatch 2024-03-21T13:30:52.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:52 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:52.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:52 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi097-79618-29"}]': finished 2024-03-21T13:30:52.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:52 smithi097 ceph-mon[39416]: osdmap e177: 8 total, 8 up, 8 in 2024-03-21T13:30:52.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:52 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4244412087' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi097-79618-29"}]: dispatch 2024-03-21T13:30:52.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:52 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi097-79618-29"}]: dispatch 2024-03-21T13:30:52.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:52 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3620224239' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi097-79636-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:52.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:52 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:30:52.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:52 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:30:52.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:52 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi097-79710-1","var":"pgp_num","val":"11"}]': finished 2024-03-21T13:30:52.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:52 smithi097 ceph-mon[30040]: osdmap e176: 8 total, 8 up, 8 in 2024-03-21T13:30:52.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:52 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:52.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:52 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:52.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:52 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4244412087' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi097-79618-29"}]: dispatch 2024-03-21T13:30:52.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:52 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:52.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:52 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi097-79618-29"}]: dispatch 2024-03-21T13:30:52.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:52 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:52.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:52 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi097-79618-29"}]': finished 2024-03-21T13:30:52.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:52 smithi097 ceph-mon[30040]: osdmap e177: 8 total, 8 up, 8 in 2024-03-21T13:30:52.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:52 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4244412087' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi097-79618-29"}]: dispatch 2024-03-21T13:30:52.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:52 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi097-79618-29"}]: dispatch 2024-03-21T13:30:52.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:52 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3620224239' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi097-79636-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:52.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:52 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:30:52.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:52 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:30:52.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:52 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi097-79710-1","var":"pgp_num","val":"11"}]': finished 2024-03-21T13:30:52.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:52 smithi144 ceph-mon[33633]: osdmap e176: 8 total, 8 up, 8 in 2024-03-21T13:30:52.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:52 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:52.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:52 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:52.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:52 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4244412087' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi097-79618-29"}]: dispatch 2024-03-21T13:30:52.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:52 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:52.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:52 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi097-79618-29"}]: dispatch 2024-03-21T13:30:52.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:52 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:52.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:52 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi097-79618-29"}]': finished 2024-03-21T13:30:52.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:52 smithi144 ceph-mon[33633]: osdmap e177: 8 total, 8 up, 8 in 2024-03-21T13:30:52.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:52 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4244412087' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi097-79618-29"}]: dispatch 2024-03-21T13:30:52.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:52 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi097-79618-29"}]: dispatch 2024-03-21T13:30:52.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:52 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3620224239' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi097-79636-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:52.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:52 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:30:52.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:52 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:30:53.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:53 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:53.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:53 smithi097 ceph-mon[39416]: pgmap v230: 388 pgs: 64 unknown, 324 active+clean; 586 KiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:30:53.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:53 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi097-79618-29"}]': finished 2024-03-21T13:30:53.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:53 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3620224239' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi097-79636-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:53.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:53 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-17", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:30:53.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:53 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-17"}]: dispatch 2024-03-21T13:30:53.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:53 smithi097 ceph-mon[39416]: osdmap e178: 8 total, 8 up, 8 in 2024-03-21T13:30:53.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:53 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-17"}]: dispatch 2024-03-21T13:30:53.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:53 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/913776673' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi097-79618-30"}]: dispatch 2024-03-21T13:30:53.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:53 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi097-79618-30"}]: dispatch 2024-03-21T13:30:53.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:53 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/913776673' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi097-79618-30"}]: dispatch 2024-03-21T13:30:53.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:53 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi097-79618-30"}]: dispatch 2024-03-21T13:30:53.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:53 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/913776673' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi097-79618-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:30:53.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:53 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi097-79618-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:30:53.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:53 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:53.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:53 smithi097 ceph-mon[30040]: pgmap v230: 388 pgs: 64 unknown, 324 active+clean; 586 KiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:30:53.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:53 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi097-79618-29"}]': finished 2024-03-21T13:30:53.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:53 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3620224239' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi097-79636-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:53.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:53 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-17", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:30:53.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:53 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-17"}]: dispatch 2024-03-21T13:30:53.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:53 smithi097 ceph-mon[30040]: osdmap e178: 8 total, 8 up, 8 in 2024-03-21T13:30:53.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:53 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-17"}]: dispatch 2024-03-21T13:30:53.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:53 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/913776673' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi097-79618-30"}]: dispatch 2024-03-21T13:30:53.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:53 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi097-79618-30"}]: dispatch 2024-03-21T13:30:53.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:53 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/913776673' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi097-79618-30"}]: dispatch 2024-03-21T13:30:53.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:53 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi097-79618-30"}]: dispatch 2024-03-21T13:30:53.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:53 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/913776673' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi097-79618-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:30:53.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:53 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi097-79618-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:30:53.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:30:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:30:53.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:53 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:53.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:53 smithi144 ceph-mon[33633]: pgmap v230: 388 pgs: 64 unknown, 324 active+clean; 586 KiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:30:53.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:53 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi097-79618-29"}]': finished 2024-03-21T13:30:53.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:53 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3620224239' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi097-79636-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:53.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:53 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-17", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:30:53.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:53 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-17"}]: dispatch 2024-03-21T13:30:53.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:53 smithi144 ceph-mon[33633]: osdmap e178: 8 total, 8 up, 8 in 2024-03-21T13:30:53.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:53 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-17"}]: dispatch 2024-03-21T13:30:53.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:53 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/913776673' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi097-79618-30"}]: dispatch 2024-03-21T13:30:53.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:53 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi097-79618-30"}]: dispatch 2024-03-21T13:30:53.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:53 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/913776673' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi097-79618-30"}]: dispatch 2024-03-21T13:30:53.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:53 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi097-79618-30"}]: dispatch 2024-03-21T13:30:53.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:53 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/913776673' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi097-79618-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:30:53.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:53 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi097-79618-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:30:54.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:54 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:54.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:54 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:30:54.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:54 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-17"}]': finished 2024-03-21T13:30:54.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:54 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi097-79618-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:30:54.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:54 smithi097 ceph-mon[39416]: osdmap e179: 8 total, 8 up, 8 in 2024-03-21T13:30:54.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:54 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-17", "mode": "writeback"}]: dispatch 2024-03-21T13:30:54.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:54 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-17", "mode": "writeback"}]: dispatch 2024-03-21T13:30:54.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:54 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/913776673' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi097-79618-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi097-79618-30"}]: dispatch 2024-03-21T13:30:54.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:54 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi097-79618-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi097-79618-30"}]: dispatch 2024-03-21T13:30:54.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:54 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:54.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:54 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:30:54.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:54 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-17"}]': finished 2024-03-21T13:30:54.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:54 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi097-79618-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:30:54.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:54 smithi097 ceph-mon[30040]: osdmap e179: 8 total, 8 up, 8 in 2024-03-21T13:30:54.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:54 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-17", "mode": "writeback"}]: dispatch 2024-03-21T13:30:54.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:54 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-17", "mode": "writeback"}]: dispatch 2024-03-21T13:30:54.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:54 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/913776673' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi097-79618-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi097-79618-30"}]: dispatch 2024-03-21T13:30:54.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:54 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi097-79618-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi097-79618-30"}]: dispatch 2024-03-21T13:30:54.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:54 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:54.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:54 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:30:54.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:54 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-17"}]': finished 2024-03-21T13:30:54.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:54 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi097-79618-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:30:54.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:54 smithi144 ceph-mon[33633]: osdmap e179: 8 total, 8 up, 8 in 2024-03-21T13:30:54.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:54 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-17", "mode": "writeback"}]: dispatch 2024-03-21T13:30:54.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:54 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-17", "mode": "writeback"}]: dispatch 2024-03-21T13:30:54.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:54 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/913776673' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi097-79618-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi097-79618-30"}]: dispatch 2024-03-21T13:30:54.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:54 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi097-79618-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi097-79618-30"}]: dispatch 2024-03-21T13:30:55.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:55 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:55.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:55 smithi097 ceph-mon[39416]: pgmap v233: 356 pgs: 356 active+clean; 586 KiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:30:55.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:55 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi097-79710-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-03-21T13:30:55.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:55 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi097-79710-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-03-21T13:30:55.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:55 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:30:55.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:55 smithi097 ceph-mon[39416]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:30:55.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:55 smithi097 ceph-mon[39416]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:30:55.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:55 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-17", "mode": "writeback"}]': finished 2024-03-21T13:30:55.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:55 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi097-79710-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-03-21T13:30:55.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:55 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:55.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:55 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:30:55.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:55 smithi097 ceph-mon[39416]: osdmap e180: 8 total, 8 up, 8 in 2024-03-21T13:30:55.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:55 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/415158219' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi097-79636-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:55.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:55 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:30:55.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:55 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi097-79636-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:55.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:55 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:55.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:55 smithi097 ceph-mon[30040]: pgmap v233: 356 pgs: 356 active+clean; 586 KiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:30:55.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:55 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi097-79710-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-03-21T13:30:55.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:55 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi097-79710-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-03-21T13:30:55.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:55 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:30:55.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:55 smithi097 ceph-mon[30040]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:30:55.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:55 smithi097 ceph-mon[30040]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:30:55.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:55 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-17", "mode": "writeback"}]': finished 2024-03-21T13:30:55.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:55 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi097-79710-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-03-21T13:30:55.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:55 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:55.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:55 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:30:55.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:55 smithi097 ceph-mon[30040]: osdmap e180: 8 total, 8 up, 8 in 2024-03-21T13:30:55.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:55 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/415158219' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi097-79636-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:55.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:55 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:30:55.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:55 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi097-79636-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:55.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:55 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:55.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:55 smithi144 ceph-mon[33633]: pgmap v233: 356 pgs: 356 active+clean; 586 KiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:30:55.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:55 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi097-79710-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-03-21T13:30:55.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:55 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi097-79710-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-03-21T13:30:55.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:55 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:30:55.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:55 smithi144 ceph-mon[33633]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:30:55.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:55 smithi144 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:30:55.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:55 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-17", "mode": "writeback"}]': finished 2024-03-21T13:30:55.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:55 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi097-79710-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-03-21T13:30:55.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:55 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:55.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:55 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:30:55.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:55 smithi144 ceph-mon[33633]: osdmap e180: 8 total, 8 up, 8 in 2024-03-21T13:30:55.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:55 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/415158219' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi097-79636-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:55.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:55 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:30:55.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:55 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi097-79636-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:57.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:57 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi097-79618-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi097-79618-30"}]': finished 2024-03-21T13:30:57.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:57 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:30:57.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:57 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi097-79636-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:57.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:57 smithi097 ceph-mon[39416]: osdmap e181: 8 total, 8 up, 8 in 2024-03-21T13:30:57.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:57 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-17"}]: dispatch 2024-03-21T13:30:57.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:57 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-17"}]: dispatch 2024-03-21T13:30:57.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:57 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:57.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:57 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi097-79710-1","var":"pg_num","val":"11"}]: dispatch 2024-03-21T13:30:57.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:57 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi097-79710-1","var":"pg_num","val":"11"}]: dispatch 2024-03-21T13:30:57.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:57 smithi097 ceph-mon[39416]: pgmap v236: 396 pgs: 40 unknown, 356 active+clean; 586 KiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:30:57.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:57 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi097-79618-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi097-79618-30"}]': finished 2024-03-21T13:30:57.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:57 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:30:57.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:57 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi097-79636-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:57.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:57 smithi097 ceph-mon[30040]: osdmap e181: 8 total, 8 up, 8 in 2024-03-21T13:30:57.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:57 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-17"}]: dispatch 2024-03-21T13:30:57.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:57 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-17"}]: dispatch 2024-03-21T13:30:57.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:57 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:57.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:57 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi097-79710-1","var":"pg_num","val":"11"}]: dispatch 2024-03-21T13:30:57.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:57 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi097-79710-1","var":"pg_num","val":"11"}]: dispatch 2024-03-21T13:30:57.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:57 smithi097 ceph-mon[30040]: pgmap v236: 396 pgs: 40 unknown, 356 active+clean; 586 KiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:30:57.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:57 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi097-79618-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi097-79618-30"}]': finished 2024-03-21T13:30:57.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:57 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:30:57.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:57 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi097-79636-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:30:57.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:57 smithi144 ceph-mon[33633]: osdmap e181: 8 total, 8 up, 8 in 2024-03-21T13:30:57.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:57 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-17"}]: dispatch 2024-03-21T13:30:57.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:57 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-17"}]: dispatch 2024-03-21T13:30:57.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:57 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:57.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:57 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi097-79710-1","var":"pg_num","val":"11"}]: dispatch 2024-03-21T13:30:57.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:57 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi097-79710-1","var":"pg_num","val":"11"}]: dispatch 2024-03-21T13:30:57.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:57 smithi144 ceph-mon[33633]: pgmap v236: 396 pgs: 40 unknown, 356 active+clean; 586 KiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:30:58.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:58 smithi097 ceph-mon[39416]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:30:58.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:58 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-17"}]': finished 2024-03-21T13:30:58.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:58 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi097-79710-1","var":"pg_num","val":"11"}]': finished 2024-03-21T13:30:58.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:58 smithi097 ceph-mon[39416]: osdmap e182: 8 total, 8 up, 8 in 2024-03-21T13:30:58.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:58 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:58.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:58 smithi097 ceph-mon[30040]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:30:58.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:58 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-17"}]': finished 2024-03-21T13:30:58.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:58 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi097-79710-1","var":"pg_num","val":"11"}]': finished 2024-03-21T13:30:58.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:58 smithi097 ceph-mon[30040]: osdmap e182: 8 total, 8 up, 8 in 2024-03-21T13:30:58.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:58 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:58.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:58 smithi144 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:30:58.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:58 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-17"}]': finished 2024-03-21T13:30:58.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:58 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi097-79710-1","var":"pg_num","val":"11"}]': finished 2024-03-21T13:30:58.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:58 smithi144 ceph-mon[33633]: osdmap e182: 8 total, 8 up, 8 in 2024-03-21T13:30:58.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:58 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:59.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:59 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:59.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:59 smithi097 ceph-mon[39416]: osdmap e183: 8 total, 8 up, 8 in 2024-03-21T13:30:59.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:59 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4258867489' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi097-79636-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:59.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:59 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/913776673' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi097-79618-30"}]: dispatch 2024-03-21T13:30:59.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:59 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi097-79618-30"}]: dispatch 2024-03-21T13:30:59.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:59 smithi097 ceph-mon[39416]: pgmap v239: 356 pgs: 32 creating+peering, 324 active+clean; 586 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-03-21T13:30:59.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:59 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi097-79710-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-03-21T13:30:59.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:30:59 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi097-79710-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-03-21T13:30:59.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:59 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:59.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:59 smithi097 ceph-mon[30040]: osdmap e183: 8 total, 8 up, 8 in 2024-03-21T13:30:59.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:59 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4258867489' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi097-79636-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:59.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:59 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/913776673' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi097-79618-30"}]: dispatch 2024-03-21T13:30:59.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:59 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi097-79618-30"}]: dispatch 2024-03-21T13:30:59.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:59 smithi097 ceph-mon[30040]: pgmap v239: 356 pgs: 32 creating+peering, 324 active+clean; 586 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-03-21T13:30:59.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:59 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi097-79710-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-03-21T13:30:59.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:30:59 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi097-79710-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-03-21T13:30:59.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:59 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:30:59.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:59 smithi144 ceph-mon[33633]: osdmap e183: 8 total, 8 up, 8 in 2024-03-21T13:30:59.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:59 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4258867489' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi097-79636-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:30:59.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:59 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/913776673' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi097-79618-30"}]: dispatch 2024-03-21T13:30:59.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:59 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi097-79618-30"}]: dispatch 2024-03-21T13:30:59.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:59 smithi144 ceph-mon[33633]: pgmap v239: 356 pgs: 32 creating+peering, 324 active+clean; 586 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-03-21T13:30:59.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:59 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi097-79710-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-03-21T13:30:59.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:30:59 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi097-79710-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-03-21T13:31:00.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:00 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:00.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:00 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4258867489' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi097-79636-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:00.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:00 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi097-79618-30"}]': finished 2024-03-21T13:31:00.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:00 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi097-79710-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-03-21T13:31:00.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:00 smithi097 ceph-mon[39416]: osdmap e184: 8 total, 8 up, 8 in 2024-03-21T13:31:00.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:00 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:00.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:00 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/913776673' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi097-79618-30"}]: dispatch 2024-03-21T13:31:00.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:00 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi097-79618-30"}]: dispatch 2024-03-21T13:31:00.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:00 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:00.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:00 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4258867489' entity='client.admin' cmd=[{ 2024-03-21T13:31:00.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:00 smithi097 ceph-mon[39416]: "prefix": "osd pool set", 2024-03-21T13:31:00.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:00 smithi097 ceph-mon[39416]: "pool": "PoolEIOFlag_smithi097-79636-33", 2024-03-21T13:31:00.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:00 smithi097 ceph-mon[39416]: "var": "eio", 2024-03-21T13:31:00.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:00 smithi097 ceph-mon[39416]: "val": "true" 2024-03-21T13:31:00.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:00 smithi097 ceph-mon[39416]: }]: dispatch 2024-03-21T13:31:00.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:00 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:00.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:00 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:00.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:00 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4258867489' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi097-79636-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:00.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:00 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi097-79618-30"}]': finished 2024-03-21T13:31:00.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:00 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi097-79710-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-03-21T13:31:00.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:00 smithi097 ceph-mon[30040]: osdmap e184: 8 total, 8 up, 8 in 2024-03-21T13:31:00.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:00 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:00.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:00 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/913776673' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi097-79618-30"}]: dispatch 2024-03-21T13:31:00.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:00 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi097-79618-30"}]: dispatch 2024-03-21T13:31:00.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:00 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:00.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:00 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4258867489' entity='client.admin' cmd=[{ 2024-03-21T13:31:00.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:00 smithi097 ceph-mon[30040]: "prefix": "osd pool set", 2024-03-21T13:31:00.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:00 smithi097 ceph-mon[30040]: "pool": "PoolEIOFlag_smithi097-79636-33", 2024-03-21T13:31:00.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:00 smithi097 ceph-mon[30040]: "var": "eio", 2024-03-21T13:31:00.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:00 smithi097 ceph-mon[30040]: "val": "true" 2024-03-21T13:31:00.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:00 smithi097 ceph-mon[30040]: }]: dispatch 2024-03-21T13:31:00.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:00 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:00.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:00 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:00.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:00 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4258867489' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi097-79636-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:00.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:00 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi097-79618-30"}]': finished 2024-03-21T13:31:00.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:00 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi097-79710-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-03-21T13:31:00.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:00 smithi144 ceph-mon[33633]: osdmap e184: 8 total, 8 up, 8 in 2024-03-21T13:31:00.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:00 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:00.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:00 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/913776673' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi097-79618-30"}]: dispatch 2024-03-21T13:31:00.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:00 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi097-79618-30"}]: dispatch 2024-03-21T13:31:00.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:00 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:00.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:00 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4258867489' entity='client.admin' cmd=[{ 2024-03-21T13:31:00.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:00 smithi144 ceph-mon[33633]: "prefix": "osd pool set", 2024-03-21T13:31:00.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:00 smithi144 ceph-mon[33633]: "pool": "PoolEIOFlag_smithi097-79636-33", 2024-03-21T13:31:00.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:00 smithi144 ceph-mon[33633]: "var": "eio", 2024-03-21T13:31:00.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:00 smithi144 ceph-mon[33633]: "val": "true" 2024-03-21T13:31:00.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:00 smithi144 ceph-mon[33633]: }]: dispatch 2024-03-21T13:31:00.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:00 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:01.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:01 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi097-79618-30"}]': finished 2024-03-21T13:31:01.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:01 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:01.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:01 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4258867489' entity='client.admin' cmd='[{ 2024-03-21T13:31:01.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:01 smithi097 ceph-mon[39416]: "prefix": "osd pool set", 2024-03-21T13:31:01.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:01 smithi097 ceph-mon[39416]: "pool": "PoolEIOFlag_smithi097-79636-33", 2024-03-21T13:31:01.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:01 smithi097 ceph-mon[39416]: "var": "eio", 2024-03-21T13:31:01.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:01 smithi097 ceph-mon[39416]: "val": "true" 2024-03-21T13:31:01.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:01 smithi097 ceph-mon[39416]: }]': finished 2024-03-21T13:31:01.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:01 smithi097 ceph-mon[39416]: osdmap e185: 8 total, 8 up, 8 in 2024-03-21T13:31:01.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:01 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:31:01.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:01 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:31:01.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:01 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2059866463' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi097-79618-31"}]: dispatch 2024-03-21T13:31:01.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:01 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi097-79618-31"}]: dispatch 2024-03-21T13:31:01.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:01 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2059866463' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi097-79618-31"}]: dispatch 2024-03-21T13:31:01.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:01 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi097-79618-31"}]: dispatch 2024-03-21T13:31:01.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:01 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2059866463' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi097-79618-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:01.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:01 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi097-79618-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:01.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:01 smithi097 ceph-mon[39416]: pgmap v242: 388 pgs: 32 unknown, 32 creating+peering, 324 active+clean; 586 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-03-21T13:31:01.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:01 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:01.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:01 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi097-79618-30"}]': finished 2024-03-21T13:31:01.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:01 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:01.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:01 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4258867489' entity='client.admin' cmd='[{ 2024-03-21T13:31:01.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:01 smithi097 ceph-mon[30040]: "prefix": "osd pool set", 2024-03-21T13:31:01.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:01 smithi097 ceph-mon[30040]: "pool": "PoolEIOFlag_smithi097-79636-33", 2024-03-21T13:31:01.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:01 smithi097 ceph-mon[30040]: "var": "eio", 2024-03-21T13:31:01.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:01 smithi097 ceph-mon[30040]: "val": "true" 2024-03-21T13:31:01.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:01 smithi097 ceph-mon[30040]: }]': finished 2024-03-21T13:31:01.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:01 smithi097 ceph-mon[30040]: osdmap e185: 8 total, 8 up, 8 in 2024-03-21T13:31:01.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:01 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:31:01.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:01 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:31:01.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:01 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2059866463' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi097-79618-31"}]: dispatch 2024-03-21T13:31:01.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:01 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi097-79618-31"}]: dispatch 2024-03-21T13:31:01.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:01 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2059866463' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi097-79618-31"}]: dispatch 2024-03-21T13:31:01.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:01 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi097-79618-31"}]: dispatch 2024-03-21T13:31:01.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:01 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2059866463' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi097-79618-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:01.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:01 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi097-79618-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:01.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:01 smithi097 ceph-mon[30040]: pgmap v242: 388 pgs: 32 unknown, 32 creating+peering, 324 active+clean; 586 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-03-21T13:31:01.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:01 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:01.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:01 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi097-79618-30"}]': finished 2024-03-21T13:31:01.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:01 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:01.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:01 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4258867489' entity='client.admin' cmd='[{ 2024-03-21T13:31:01.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:01 smithi144 ceph-mon[33633]: "prefix": "osd pool set", 2024-03-21T13:31:01.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:01 smithi144 ceph-mon[33633]: "pool": "PoolEIOFlag_smithi097-79636-33", 2024-03-21T13:31:01.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:01 smithi144 ceph-mon[33633]: "var": "eio", 2024-03-21T13:31:01.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:01 smithi144 ceph-mon[33633]: "val": "true" 2024-03-21T13:31:01.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:01 smithi144 ceph-mon[33633]: }]': finished 2024-03-21T13:31:01.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:01 smithi144 ceph-mon[33633]: osdmap e185: 8 total, 8 up, 8 in 2024-03-21T13:31:01.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:01 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:31:01.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:01 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:31:01.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:01 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2059866463' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi097-79618-31"}]: dispatch 2024-03-21T13:31:01.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:01 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi097-79618-31"}]: dispatch 2024-03-21T13:31:01.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:01 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2059866463' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi097-79618-31"}]: dispatch 2024-03-21T13:31:01.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:01 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi097-79618-31"}]: dispatch 2024-03-21T13:31:01.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:01 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2059866463' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi097-79618-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:01.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:01 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi097-79618-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:01.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:01 smithi144 ceph-mon[33633]: pgmap v242: 388 pgs: 32 unknown, 32 creating+peering, 324 active+clean; 586 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-03-21T13:31:01.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:01 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:02.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:31:01 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:31:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:31:02.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:02 smithi097 ceph-mon[30040]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:31:02.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:02 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-19", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:31:02.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:02 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi097-79618-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:31:02.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:02 smithi097 ceph-mon[30040]: osdmap e186: 8 total, 8 up, 8 in 2024-03-21T13:31:02.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:02 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2059866463' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi097-79618-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi097-79618-31"}]: dispatch 2024-03-21T13:31:02.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:02 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi097-79618-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi097-79618-31"}]: dispatch 2024-03-21T13:31:02.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:02 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-19"}]: dispatch 2024-03-21T13:31:02.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:02 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-19"}]: dispatch 2024-03-21T13:31:02.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:02 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:02.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:02 smithi097 ceph-mon[39416]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:31:02.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:02 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-19", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:31:02.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:02 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi097-79618-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:31:02.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:02 smithi097 ceph-mon[39416]: osdmap e186: 8 total, 8 up, 8 in 2024-03-21T13:31:02.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:02 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2059866463' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi097-79618-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi097-79618-31"}]: dispatch 2024-03-21T13:31:02.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:02 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi097-79618-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi097-79618-31"}]: dispatch 2024-03-21T13:31:02.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:02 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-19"}]: dispatch 2024-03-21T13:31:02.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:02 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-19"}]: dispatch 2024-03-21T13:31:02.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:02 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:02.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:02 smithi144 ceph-mon[33633]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:31:02.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:02 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-19", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:31:02.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:02 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi097-79618-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:31:02.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:02 smithi144 ceph-mon[33633]: osdmap e186: 8 total, 8 up, 8 in 2024-03-21T13:31:02.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:02 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2059866463' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi097-79618-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi097-79618-31"}]: dispatch 2024-03-21T13:31:02.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:02 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi097-79618-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi097-79618-31"}]: dispatch 2024-03-21T13:31:02.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:02 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-19"}]: dispatch 2024-03-21T13:31:02.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:02 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-19"}]: dispatch 2024-03-21T13:31:02.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:02 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:03.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:03 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-19"}]': finished 2024-03-21T13:31:03.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:03 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-19", "mode": "writeback"}]: dispatch 2024-03-21T13:31:03.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:03 smithi097 ceph-mon[30040]: pgmap v245: 387 pgs: 64 unknown, 323 active+clean; 585 KiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:31:03.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:03 smithi097 ceph-mon[30040]: osdmap e187: 8 total, 8 up, 8 in 2024-03-21T13:31:03.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:03 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2565728708' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi097-79636-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:03.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:03 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-19", "mode": "writeback"}]: dispatch 2024-03-21T13:31:03.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:03 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi097-79636-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:03.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:03 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:03.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:03 smithi097 ceph-mon[30040]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:31:03.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:03 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi097-79618-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi097-79618-31"}]': finished 2024-03-21T13:31:03.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:03 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-19", "mode": "writeback"}]': finished 2024-03-21T13:31:03.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:03 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi097-79636-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:03.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:03 smithi097 ceph-mon[30040]: osdmap e188: 8 total, 8 up, 8 in 2024-03-21T13:31:03.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:03 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:31:03.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:03 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:31:03.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:03 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-19"}]': finished 2024-03-21T13:31:03.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:03 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-19", "mode": "writeback"}]: dispatch 2024-03-21T13:31:03.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:03 smithi097 ceph-mon[39416]: pgmap v245: 387 pgs: 64 unknown, 323 active+clean; 585 KiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:31:03.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:03 smithi097 ceph-mon[39416]: osdmap e187: 8 total, 8 up, 8 in 2024-03-21T13:31:03.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:03 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2565728708' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi097-79636-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:03.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:03 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-19", "mode": "writeback"}]: dispatch 2024-03-21T13:31:03.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:03 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi097-79636-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:03.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:03 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:03.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:03 smithi097 ceph-mon[39416]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:31:03.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:03 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi097-79618-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi097-79618-31"}]': finished 2024-03-21T13:31:03.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:03 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-19", "mode": "writeback"}]': finished 2024-03-21T13:31:03.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:03 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi097-79636-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:03.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:03 smithi097 ceph-mon[39416]: osdmap e188: 8 total, 8 up, 8 in 2024-03-21T13:31:03.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:03 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:31:03.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:03 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:31:03.871 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:31:03 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:31:03.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:03 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-19"}]': finished 2024-03-21T13:31:03.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:03 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-19", "mode": "writeback"}]: dispatch 2024-03-21T13:31:03.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:03 smithi144 ceph-mon[33633]: pgmap v245: 387 pgs: 64 unknown, 323 active+clean; 585 KiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:31:03.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:03 smithi144 ceph-mon[33633]: osdmap e187: 8 total, 8 up, 8 in 2024-03-21T13:31:03.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:03 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2565728708' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi097-79636-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:03.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:03 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-19", "mode": "writeback"}]: dispatch 2024-03-21T13:31:03.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:03 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi097-79636-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:03.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:03 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:03.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:03 smithi144 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:31:03.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:03 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi097-79618-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi097-79618-31"}]': finished 2024-03-21T13:31:03.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:03 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-19", "mode": "writeback"}]': finished 2024-03-21T13:31:03.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:03 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi097-79636-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:03.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:03 smithi144 ceph-mon[33633]: osdmap e188: 8 total, 8 up, 8 in 2024-03-21T13:31:03.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:03 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:31:03.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:03 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:31:04.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:04 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:31:04.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:04 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:04.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:04 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:31:04.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:04 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-19"}]: dispatch 2024-03-21T13:31:04.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:04 smithi144 ceph-mon[33633]: osdmap e189: 8 total, 8 up, 8 in 2024-03-21T13:31:04.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:04 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-19"}]: dispatch 2024-03-21T13:31:05.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:04 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:31:05.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:04 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:05.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:04 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:31:05.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:04 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-19"}]: dispatch 2024-03-21T13:31:05.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:04 smithi097 ceph-mon[39416]: osdmap e189: 8 total, 8 up, 8 in 2024-03-21T13:31:05.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:04 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-19"}]: dispatch 2024-03-21T13:31:05.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:04 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:31:05.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:04 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:05.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:04 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:31:05.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:04 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-19"}]: dispatch 2024-03-21T13:31:05.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:04 smithi097 ceph-mon[30040]: osdmap e189: 8 total, 8 up, 8 in 2024-03-21T13:31:05.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:04 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-19"}]: dispatch 2024-03-21T13:31:05.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:05 smithi144 ceph-mon[33633]: pgmap v248: 363 pgs: 8 creating+peering, 355 active+clean; 586 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T13:31:05.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:05 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:05.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:05 smithi144 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:31:05.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:05 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-19"}]': finished 2024-03-21T13:31:05.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:05 smithi144 ceph-mon[33633]: osdmap e190: 8 total, 8 up, 8 in 2024-03-21T13:31:05.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:05 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4183579785' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi097-79636-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:05.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:05 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2059866463' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi097-79618-31"}]: dispatch 2024-03-21T13:31:05.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:05 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi097-79636-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:05.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:05 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi097-79618-31"}]: dispatch 2024-03-21T13:31:06.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:05 smithi097 ceph-mon[39416]: pgmap v248: 363 pgs: 8 creating+peering, 355 active+clean; 586 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T13:31:06.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:05 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:06.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:05 smithi097 ceph-mon[39416]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:31:06.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:05 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-19"}]': finished 2024-03-21T13:31:06.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:05 smithi097 ceph-mon[39416]: osdmap e190: 8 total, 8 up, 8 in 2024-03-21T13:31:06.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:05 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4183579785' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi097-79636-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:06.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:05 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2059866463' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi097-79618-31"}]: dispatch 2024-03-21T13:31:06.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:05 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi097-79636-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:06.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:05 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi097-79618-31"}]: dispatch 2024-03-21T13:31:06.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:05 smithi097 ceph-mon[30040]: pgmap v248: 363 pgs: 8 creating+peering, 355 active+clean; 586 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T13:31:06.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:05 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:06.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:05 smithi097 ceph-mon[30040]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:31:06.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:05 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-19"}]': finished 2024-03-21T13:31:06.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:05 smithi097 ceph-mon[30040]: osdmap e190: 8 total, 8 up, 8 in 2024-03-21T13:31:06.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:05 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4183579785' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi097-79636-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:06.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:05 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2059866463' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi097-79618-31"}]: dispatch 2024-03-21T13:31:06.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:05 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi097-79636-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:06.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:05 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi097-79618-31"}]: dispatch 2024-03-21T13:31:06.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:06 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:06.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:06 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi097-79636-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:06.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:06 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi097-79618-31"}]': finished 2024-03-21T13:31:06.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:06 smithi144 ceph-mon[33633]: osdmap e191: 8 total, 8 up, 8 in 2024-03-21T13:31:06.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:06 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2059866463' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi097-79618-31"}]: dispatch 2024-03-21T13:31:06.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:06 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi097-79618-31"}]: dispatch 2024-03-21T13:31:07.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:06 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:07.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:06 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi097-79636-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:07.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:06 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi097-79618-31"}]': finished 2024-03-21T13:31:07.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:06 smithi097 ceph-mon[39416]: osdmap e191: 8 total, 8 up, 8 in 2024-03-21T13:31:07.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:06 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2059866463' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi097-79618-31"}]: dispatch 2024-03-21T13:31:07.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:06 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi097-79618-31"}]: dispatch 2024-03-21T13:31:07.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:06 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:07.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:06 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi097-79636-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:07.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:06 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi097-79618-31"}]': finished 2024-03-21T13:31:07.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:06 smithi097 ceph-mon[30040]: osdmap e191: 8 total, 8 up, 8 in 2024-03-21T13:31:07.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:06 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2059866463' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi097-79618-31"}]: dispatch 2024-03-21T13:31:07.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:06 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi097-79618-31"}]: dispatch 2024-03-21T13:31:07.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:07 smithi144 ceph-mon[33633]: pgmap v251: 355 pgs: 32 unknown, 323 active+clean; 586 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s; 31 B/s, 0 objects/s recovering 2024-03-21T13:31:07.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:07 smithi144 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:31:07.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:07 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:07.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:07 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi097-79618-31"}]': finished 2024-03-21T13:31:07.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:07 smithi144 ceph-mon[33633]: osdmap e192: 8 total, 8 up, 8 in 2024-03-21T13:31:07.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:07 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:07.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:07 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:07.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:07 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3473711171' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi097-79618-32"}]: dispatch 2024-03-21T13:31:07.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:07 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3473711171' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi097-79618-32"}]: dispatch 2024-03-21T13:31:07.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:07 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3473711171' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi097-79618-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:08.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:07 smithi097 ceph-mon[30040]: pgmap v251: 355 pgs: 32 unknown, 323 active+clean; 586 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s; 31 B/s, 0 objects/s recovering 2024-03-21T13:31:08.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:07 smithi097 ceph-mon[30040]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:31:08.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:07 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:08.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:07 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi097-79618-31"}]': finished 2024-03-21T13:31:08.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:07 smithi097 ceph-mon[30040]: osdmap e192: 8 total, 8 up, 8 in 2024-03-21T13:31:08.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:07 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:08.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:07 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:08.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:07 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3473711171' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi097-79618-32"}]: dispatch 2024-03-21T13:31:08.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:07 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3473711171' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi097-79618-32"}]: dispatch 2024-03-21T13:31:08.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:07 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3473711171' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi097-79618-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:08.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:07 smithi097 ceph-mon[39416]: pgmap v251: 355 pgs: 32 unknown, 323 active+clean; 586 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s; 31 B/s, 0 objects/s recovering 2024-03-21T13:31:08.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:07 smithi097 ceph-mon[39416]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:31:08.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:07 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:08.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:07 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi097-79618-31"}]': finished 2024-03-21T13:31:08.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:07 smithi097 ceph-mon[39416]: osdmap e192: 8 total, 8 up, 8 in 2024-03-21T13:31:08.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:07 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:08.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:07 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:08.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:07 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3473711171' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi097-79618-32"}]: dispatch 2024-03-21T13:31:08.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:07 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3473711171' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi097-79618-32"}]: dispatch 2024-03-21T13:31:08.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:07 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3473711171' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi097-79618-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:08.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:08 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:08.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:08 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:08.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:08 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3473711171' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi097-79618-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:31:08.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:08 smithi144 ceph-mon[33633]: osdmap e193: 8 total, 8 up, 8 in 2024-03-21T13:31:08.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:08 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3473711171' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi097-79618-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi097-79618-32"}]: dispatch 2024-03-21T13:31:08.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:08 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/544808249' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi097-79636-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:08.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:08 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:31:08.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:08 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:31:08.954 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:08 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:08.954 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:08 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:08.954 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:08 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3473711171' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi097-79618-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:31:08.954 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:08 smithi097 ceph-mon[39416]: osdmap e193: 8 total, 8 up, 8 in 2024-03-21T13:31:08.954 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:08 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3473711171' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi097-79618-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi097-79618-32"}]: dispatch 2024-03-21T13:31:08.955 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:08 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/544808249' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi097-79636-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:08.955 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:08 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:31:08.955 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:08 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:31:08.955 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:08 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:08.955 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:08 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:08.955 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:08 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3473711171' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi097-79618-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:31:08.955 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:08 smithi097 ceph-mon[30040]: osdmap e193: 8 total, 8 up, 8 in 2024-03-21T13:31:08.955 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:08 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3473711171' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi097-79618-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi097-79618-32"}]: dispatch 2024-03-21T13:31:08.955 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:08 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/544808249' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi097-79636-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:08.955 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:08 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:31:08.955 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:08 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:31:09.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:09 smithi144 ceph-mon[33633]: pgmap v254: 387 pgs: 23 creating+activating, 22 creating+peering, 19 unknown, 323 active+clean; 586 KiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:31:09.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:09 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:09.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:09 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/544808249' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi097-79636-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:09.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:09 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-21", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:31:09.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:09 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-21"}]: dispatch 2024-03-21T13:31:09.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:09 smithi144 ceph-mon[33633]: osdmap e194: 8 total, 8 up, 8 in 2024-03-21T13:31:09.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:09 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-21"}]: dispatch 2024-03-21T13:31:10.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:09 smithi097 ceph-mon[30040]: pgmap v254: 387 pgs: 23 creating+activating, 22 creating+peering, 19 unknown, 323 active+clean; 586 KiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:31:10.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:09 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:10.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:09 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/544808249' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi097-79636-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:10.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:09 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-21", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:31:10.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:09 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-21"}]: dispatch 2024-03-21T13:31:10.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:09 smithi097 ceph-mon[30040]: osdmap e194: 8 total, 8 up, 8 in 2024-03-21T13:31:10.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:09 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-21"}]: dispatch 2024-03-21T13:31:10.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:09 smithi097 ceph-mon[39416]: pgmap v254: 387 pgs: 23 creating+activating, 22 creating+peering, 19 unknown, 323 active+clean; 586 KiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:31:10.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:09 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:10.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:09 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/544808249' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi097-79636-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:10.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:09 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-21", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:31:10.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:09 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-21"}]: dispatch 2024-03-21T13:31:10.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:09 smithi097 ceph-mon[39416]: osdmap e194: 8 total, 8 up, 8 in 2024-03-21T13:31:10.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:09 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-21"}]: dispatch 2024-03-21T13:31:10.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:10 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:31:10.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:10 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:10.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:10 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3473711171' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi097-79618-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi097-79618-32"}]': finished 2024-03-21T13:31:10.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:10 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-21"}]': finished 2024-03-21T13:31:10.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:10 smithi144 ceph-mon[33633]: osdmap e195: 8 total, 8 up, 8 in 2024-03-21T13:31:10.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:10 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-21", "mode": "writeback"}]: dispatch 2024-03-21T13:31:10.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:10 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3152405655' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi097-79636-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:10.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:10 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-21", "mode": "writeback"}]: dispatch 2024-03-21T13:31:11.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:10 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:31:11.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:10 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:11.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:10 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3473711171' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi097-79618-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi097-79618-32"}]': finished 2024-03-21T13:31:11.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:10 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-21"}]': finished 2024-03-21T13:31:11.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:10 smithi097 ceph-mon[39416]: osdmap e195: 8 total, 8 up, 8 in 2024-03-21T13:31:11.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:10 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-21", "mode": "writeback"}]: dispatch 2024-03-21T13:31:11.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:10 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3152405655' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi097-79636-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:11.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:10 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-21", "mode": "writeback"}]: dispatch 2024-03-21T13:31:11.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:10 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:31:11.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:10 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:11.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:10 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3473711171' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi097-79618-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi097-79618-32"}]': finished 2024-03-21T13:31:11.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:10 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-21"}]': finished 2024-03-21T13:31:11.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:10 smithi097 ceph-mon[30040]: osdmap e195: 8 total, 8 up, 8 in 2024-03-21T13:31:11.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:10 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-21", "mode": "writeback"}]: dispatch 2024-03-21T13:31:11.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:10 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3152405655' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi097-79636-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:11.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:10 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-21", "mode": "writeback"}]: dispatch 2024-03-21T13:31:11.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:11 smithi144 ceph-mon[33633]: pgmap v257: 427 pgs: 23 creating+activating, 22 creating+peering, 59 unknown, 323 active+clean; 586 KiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:31:11.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:11 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:11.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:11 smithi144 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:31:11.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:11 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3152405655' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi097-79636-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:11.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:11 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-21", "mode": "writeback"}]': finished 2024-03-21T13:31:11.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:11 smithi144 ceph-mon[33633]: osdmap e196: 8 total, 8 up, 8 in 2024-03-21T13:31:11.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:11 smithi097 ceph-mon[39416]: pgmap v257: 427 pgs: 23 creating+activating, 22 creating+peering, 59 unknown, 323 active+clean; 586 KiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:31:11.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:11 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:11.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:11 smithi097 ceph-mon[39416]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:31:11.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:11 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3152405655' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi097-79636-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:11.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:11 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-21", "mode": "writeback"}]': finished 2024-03-21T13:31:11.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:11 smithi097 ceph-mon[39416]: osdmap e196: 8 total, 8 up, 8 in 2024-03-21T13:31:11.950 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:11 smithi097 ceph-mon[30040]: pgmap v257: 427 pgs: 23 creating+activating, 22 creating+peering, 59 unknown, 323 active+clean; 586 KiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:31:11.950 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:11 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:11.951 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:11 smithi097 ceph-mon[30040]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:31:11.951 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:11 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3152405655' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi097-79636-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:11.951 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:11 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-21", "mode": "writeback"}]': finished 2024-03-21T13:31:11.951 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:11 smithi097 ceph-mon[30040]: osdmap e196: 8 total, 8 up, 8 in 2024-03-21T13:31:12.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:31:11 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:31:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:31:13.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:12 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:31:13.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:12 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:31:13.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:12 smithi097 ceph-mon[39416]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:31:13.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:12 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:13.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:12 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:31:13.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:12 smithi097 ceph-mon[39416]: osdmap e197: 8 total, 8 up, 8 in 2024-03-21T13:31:13.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:12 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-21"}]: dispatch 2024-03-21T13:31:13.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:12 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2866407076' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi097-79636-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:13.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:12 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-21"}]: dispatch 2024-03-21T13:31:13.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:12 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi097-79636-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:13.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:12 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3473711171' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi097-79618-32"}]: dispatch 2024-03-21T13:31:13.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:12 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:31:13.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:12 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:31:13.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:12 smithi097 ceph-mon[30040]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:31:13.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:12 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:13.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:12 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:31:13.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:12 smithi097 ceph-mon[30040]: osdmap e197: 8 total, 8 up, 8 in 2024-03-21T13:31:13.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:12 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-21"}]: dispatch 2024-03-21T13:31:13.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:12 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2866407076' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi097-79636-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:13.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:12 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-21"}]: dispatch 2024-03-21T13:31:13.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:12 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi097-79636-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:13.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:12 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3473711171' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi097-79618-32"}]: dispatch 2024-03-21T13:31:13.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:12 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:31:13.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:12 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:31:13.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:12 smithi144 ceph-mon[33633]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:31:13.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:12 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:13.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:12 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:31:13.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:12 smithi144 ceph-mon[33633]: osdmap e197: 8 total, 8 up, 8 in 2024-03-21T13:31:13.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:12 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-21"}]: dispatch 2024-03-21T13:31:13.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:12 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2866407076' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi097-79636-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:13.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:12 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-21"}]: dispatch 2024-03-21T13:31:13.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:12 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi097-79636-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:13.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:12 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3473711171' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi097-79618-32"}]: dispatch 2024-03-21T13:31:13.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:13 smithi144 ceph-mon[33633]: pgmap v260: 451 pgs: 23 creating+activating, 22 creating+peering, 83 unknown, 323 active+clean; 586 KiB data, 625 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:31:13.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:13 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:13.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:13 smithi144 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:31:13.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:13 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-21"}]': finished 2024-03-21T13:31:13.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:13 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi097-79636-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:13.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:13 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3473711171' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi097-79618-32"}]': finished 2024-03-21T13:31:13.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:13 smithi144 ceph-mon[33633]: osdmap e198: 8 total, 8 up, 8 in 2024-03-21T13:31:13.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:13 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3473711171' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi097-79618-32"}]: dispatch 2024-03-21T13:31:13.872 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:31:13 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:31:14.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:13 smithi097 ceph-mon[39416]: pgmap v260: 451 pgs: 23 creating+activating, 22 creating+peering, 83 unknown, 323 active+clean; 586 KiB data, 625 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:31:14.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:13 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:14.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:13 smithi097 ceph-mon[39416]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:31:14.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:13 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-21"}]': finished 2024-03-21T13:31:14.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:13 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi097-79636-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:14.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:13 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3473711171' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi097-79618-32"}]': finished 2024-03-21T13:31:14.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:13 smithi097 ceph-mon[39416]: osdmap e198: 8 total, 8 up, 8 in 2024-03-21T13:31:14.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:13 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3473711171' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi097-79618-32"}]: dispatch 2024-03-21T13:31:14.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:13 smithi097 ceph-mon[30040]: pgmap v260: 451 pgs: 23 creating+activating, 22 creating+peering, 83 unknown, 323 active+clean; 586 KiB data, 625 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:31:14.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:13 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:14.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:13 smithi097 ceph-mon[30040]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:31:14.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:13 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-21"}]': finished 2024-03-21T13:31:14.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:13 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi097-79636-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:14.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:13 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3473711171' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi097-79618-32"}]': finished 2024-03-21T13:31:14.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:13 smithi097 ceph-mon[30040]: osdmap e198: 8 total, 8 up, 8 in 2024-03-21T13:31:14.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:13 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3473711171' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi097-79618-32"}]: dispatch 2024-03-21T13:31:15.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:14 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:31:15.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:14 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:15.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:14 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3473711171' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi097-79618-32"}]': finished 2024-03-21T13:31:15.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:14 smithi097 ceph-mon[39416]: osdmap e199: 8 total, 8 up, 8 in 2024-03-21T13:31:15.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:14 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2748596068' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi097-79618-33"}]: dispatch 2024-03-21T13:31:15.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:14 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi097-79710-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-03-21T13:31:15.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:14 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi097-79618-33"}]: dispatch 2024-03-21T13:31:15.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:14 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2748596068' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi097-79618-33"}]: dispatch 2024-03-21T13:31:15.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:14 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi097-79710-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-03-21T13:31:15.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:14 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi097-79618-33"}]: dispatch 2024-03-21T13:31:15.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:14 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2748596068' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi097-79618-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:15.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:14 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi097-79618-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:15.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:14 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:31:15.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:14 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:15.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:14 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3473711171' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi097-79618-32"}]': finished 2024-03-21T13:31:15.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:14 smithi097 ceph-mon[30040]: osdmap e199: 8 total, 8 up, 8 in 2024-03-21T13:31:15.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:14 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2748596068' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi097-79618-33"}]: dispatch 2024-03-21T13:31:15.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:14 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi097-79710-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-03-21T13:31:15.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:14 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi097-79618-33"}]: dispatch 2024-03-21T13:31:15.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:14 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2748596068' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi097-79618-33"}]: dispatch 2024-03-21T13:31:15.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:14 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi097-79710-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-03-21T13:31:15.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:14 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi097-79618-33"}]: dispatch 2024-03-21T13:31:15.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:14 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2748596068' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi097-79618-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:15.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:14 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi097-79618-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:15.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:14 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:31:15.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:14 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:15.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:14 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3473711171' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi097-79618-32"}]': finished 2024-03-21T13:31:15.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:14 smithi144 ceph-mon[33633]: osdmap e199: 8 total, 8 up, 8 in 2024-03-21T13:31:15.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:14 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2748596068' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi097-79618-33"}]: dispatch 2024-03-21T13:31:15.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:14 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi097-79710-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-03-21T13:31:15.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:14 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi097-79618-33"}]: dispatch 2024-03-21T13:31:15.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:14 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2748596068' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi097-79618-33"}]: dispatch 2024-03-21T13:31:15.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:14 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi097-79710-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-03-21T13:31:15.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:14 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi097-79618-33"}]: dispatch 2024-03-21T13:31:15.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:14 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2748596068' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi097-79618-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:15.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:14 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi097-79618-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:16.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:15 smithi097 ceph-mon[39416]: pgmap v263: 387 pgs: 387 active+clean; 586 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-21T13:31:16.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:15 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:16.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:15 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi097-79710-1", "var": "pgp_num_actual", "val": "30"}]': finished 2024-03-21T13:31:16.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:15 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi097-79618-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:31:16.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:15 smithi097 ceph-mon[39416]: osdmap e200: 8 total, 8 up, 8 in 2024-03-21T13:31:16.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:15 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2748596068' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi097-79618-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi097-79618-33"}]: dispatch 2024-03-21T13:31:16.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:15 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi097-79618-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi097-79618-33"}]: dispatch 2024-03-21T13:31:16.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:15 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:16.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:15 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:16.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:15 smithi097 ceph-mon[30040]: pgmap v263: 387 pgs: 387 active+clean; 586 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-21T13:31:16.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:15 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:16.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:15 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi097-79710-1", "var": "pgp_num_actual", "val": "30"}]': finished 2024-03-21T13:31:16.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:15 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi097-79618-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:31:16.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:15 smithi097 ceph-mon[30040]: osdmap e200: 8 total, 8 up, 8 in 2024-03-21T13:31:16.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:15 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2748596068' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi097-79618-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi097-79618-33"}]: dispatch 2024-03-21T13:31:16.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:15 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi097-79618-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi097-79618-33"}]: dispatch 2024-03-21T13:31:16.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:15 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:16.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:15 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:16.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:15 smithi144 ceph-mon[33633]: pgmap v263: 387 pgs: 387 active+clean; 586 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-21T13:31:16.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:15 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:16.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:15 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi097-79710-1", "var": "pgp_num_actual", "val": "30"}]': finished 2024-03-21T13:31:16.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:15 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi097-79618-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:31:16.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:15 smithi144 ceph-mon[33633]: osdmap e200: 8 total, 8 up, 8 in 2024-03-21T13:31:16.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:15 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2748596068' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi097-79618-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi097-79618-33"}]: dispatch 2024-03-21T13:31:16.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:15 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi097-79618-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi097-79618-33"}]: dispatch 2024-03-21T13:31:16.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:15 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:16.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:15 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:17.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:16 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:17.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:16 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:17.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:16 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi097-79710-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-21T13:31:17.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:16 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi097-79710-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-21T13:31:17.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:16 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:17.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:16 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi097-79710-1","var":"pgp_num","val":"11"}]': finished 2024-03-21T13:31:17.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:16 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:17.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:16 smithi144 ceph-mon[33633]: osdmap e201: 8 total, 8 up, 8 in 2024-03-21T13:31:17.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:16 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:17.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:16 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:17.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:16 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi097-79710-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-21T13:31:17.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:16 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi097-79710-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-21T13:31:17.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:16 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:17.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:16 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi097-79710-1","var":"pgp_num","val":"11"}]': finished 2024-03-21T13:31:17.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:16 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:17.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:16 smithi097 ceph-mon[39416]: osdmap e201: 8 total, 8 up, 8 in 2024-03-21T13:31:17.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:16 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:17.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:16 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:17.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:16 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi097-79710-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-21T13:31:17.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:16 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi097-79710-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-21T13:31:17.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:16 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:17.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:16 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi097-79710-1","var":"pgp_num","val":"11"}]': finished 2024-03-21T13:31:17.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:16 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2912801869' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T13:31:17.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:16 smithi097 ceph-mon[30040]: osdmap e201: 8 total, 8 up, 8 in 2024-03-21T13:31:18.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:17 smithi144 ceph-mon[33633]: pgmap v266: 355 pgs: 32 unknown, 323 active+clean; 586 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-21T13:31:18.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:17 smithi144 ceph-mon[33633]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:31:18.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:17 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi097-79618-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi097-79618-33"}]': finished 2024-03-21T13:31:18.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:17 smithi144 ceph-mon[33633]: osdmap e202: 8 total, 8 up, 8 in 2024-03-21T13:31:18.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:17 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3285860707' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi097-79636-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:18.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:17 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi097-79636-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:18.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:17 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1904373518' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi097-79710-2"}]: dispatch 2024-03-21T13:31:18.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:17 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi097-79710-2"}]: dispatch 2024-03-21T13:31:18.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:17 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1904373518' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi097-79710-2"}]: dispatch 2024-03-21T13:31:18.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:17 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:31:18.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:17 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi097-79710-2"}]: dispatch 2024-03-21T13:31:18.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:17 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:31:18.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:17 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1904373518' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi097-79710-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:18.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:17 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi097-79710-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:18.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:17 smithi097 ceph-mon[30040]: pgmap v266: 355 pgs: 32 unknown, 323 active+clean; 586 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-21T13:31:18.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:17 smithi097 ceph-mon[30040]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:31:18.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:17 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi097-79618-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi097-79618-33"}]': finished 2024-03-21T13:31:18.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:17 smithi097 ceph-mon[30040]: osdmap e202: 8 total, 8 up, 8 in 2024-03-21T13:31:18.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:17 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3285860707' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi097-79636-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:18.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:17 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi097-79636-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:18.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:17 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1904373518' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi097-79710-2"}]: dispatch 2024-03-21T13:31:18.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:17 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi097-79710-2"}]: dispatch 2024-03-21T13:31:18.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:17 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1904373518' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi097-79710-2"}]: dispatch 2024-03-21T13:31:18.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:17 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:31:18.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:17 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi097-79710-2"}]: dispatch 2024-03-21T13:31:18.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:17 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:31:18.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:17 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1904373518' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi097-79710-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:18.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:17 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi097-79710-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:18.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:17 smithi097 ceph-mon[39416]: pgmap v266: 355 pgs: 32 unknown, 323 active+clean; 586 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-21T13:31:18.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:17 smithi097 ceph-mon[39416]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:31:18.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:17 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi097-79618-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi097-79618-33"}]': finished 2024-03-21T13:31:18.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:17 smithi097 ceph-mon[39416]: osdmap e202: 8 total, 8 up, 8 in 2024-03-21T13:31:18.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:17 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3285860707' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi097-79636-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:18.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:17 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi097-79636-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:18.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:17 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1904373518' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi097-79710-2"}]: dispatch 2024-03-21T13:31:18.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:17 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi097-79710-2"}]: dispatch 2024-03-21T13:31:18.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:17 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1904373518' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi097-79710-2"}]: dispatch 2024-03-21T13:31:18.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:17 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:31:18.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:17 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi097-79710-2"}]: dispatch 2024-03-21T13:31:18.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:17 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:31:18.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:17 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1904373518' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi097-79710-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:18.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:17 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi097-79710-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:19.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:19 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi097-79636-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:19.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:19 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-23", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:31:19.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:19 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi097-79710-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:31:19.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:19 smithi097 ceph-mon[39416]: osdmap e203: 8 total, 8 up, 8 in 2024-03-21T13:31:19.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:19 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-23"}]: dispatch 2024-03-21T13:31:19.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:19 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1904373518' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi097-79710-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi097-79710-2"}]: dispatch 2024-03-21T13:31:19.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:19 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi097-79710-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi097-79710-2"}]: dispatch 2024-03-21T13:31:19.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:19 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-23"}]: dispatch 2024-03-21T13:31:19.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:19 smithi097 ceph-mon[39416]: pgmap v269: 364 pgs: 40 creating+peering, 324 active+clean; 584 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:31:19.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:19 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi097-79636-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:19.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:19 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-23", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:31:19.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:19 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi097-79710-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:31:19.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:19 smithi097 ceph-mon[30040]: osdmap e203: 8 total, 8 up, 8 in 2024-03-21T13:31:19.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:19 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-23"}]: dispatch 2024-03-21T13:31:19.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:19 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1904373518' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi097-79710-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi097-79710-2"}]: dispatch 2024-03-21T13:31:19.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:19 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi097-79710-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi097-79710-2"}]: dispatch 2024-03-21T13:31:19.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:19 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-23"}]: dispatch 2024-03-21T13:31:19.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:19 smithi097 ceph-mon[30040]: pgmap v269: 364 pgs: 40 creating+peering, 324 active+clean; 584 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:31:19.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:19 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi097-79636-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:19.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:19 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-23", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:31:19.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:19 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi097-79710-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:31:19.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:19 smithi144 ceph-mon[33633]: osdmap e203: 8 total, 8 up, 8 in 2024-03-21T13:31:19.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:19 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-23"}]: dispatch 2024-03-21T13:31:19.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:19 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1904373518' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi097-79710-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi097-79710-2"}]: dispatch 2024-03-21T13:31:19.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:19 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi097-79710-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi097-79710-2"}]: dispatch 2024-03-21T13:31:19.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:19 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-23"}]: dispatch 2024-03-21T13:31:19.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:19 smithi144 ceph-mon[33633]: pgmap v269: 364 pgs: 40 creating+peering, 324 active+clean; 584 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:31:20.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:20 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-23"}]': finished 2024-03-21T13:31:20.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:20 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-23", "mode": "writeback"}]: dispatch 2024-03-21T13:31:20.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:20 smithi144 ceph-mon[33633]: osdmap e204: 8 total, 8 up, 8 in 2024-03-21T13:31:20.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:20 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2748596068' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi097-79618-33"}]: dispatch 2024-03-21T13:31:20.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:20 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-23", "mode": "writeback"}]: dispatch 2024-03-21T13:31:20.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:20 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3237924076' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi097-79636-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:20.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:20 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi097-79618-33"}]: dispatch 2024-03-21T13:31:20.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:20 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi097-79636-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:20.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:20 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-23"}]': finished 2024-03-21T13:31:20.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:20 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-23", "mode": "writeback"}]: dispatch 2024-03-21T13:31:20.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:20 smithi097 ceph-mon[39416]: osdmap e204: 8 total, 8 up, 8 in 2024-03-21T13:31:20.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:20 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2748596068' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi097-79618-33"}]: dispatch 2024-03-21T13:31:20.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:20 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-23", "mode": "writeback"}]: dispatch 2024-03-21T13:31:20.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:20 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3237924076' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi097-79636-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:20.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:20 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi097-79618-33"}]: dispatch 2024-03-21T13:31:20.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:20 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi097-79636-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:20.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:20 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-23"}]': finished 2024-03-21T13:31:20.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:20 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-23", "mode": "writeback"}]: dispatch 2024-03-21T13:31:20.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:20 smithi097 ceph-mon[30040]: osdmap e204: 8 total, 8 up, 8 in 2024-03-21T13:31:20.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:20 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2748596068' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi097-79618-33"}]: dispatch 2024-03-21T13:31:20.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:20 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-23", "mode": "writeback"}]: dispatch 2024-03-21T13:31:20.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:20 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3237924076' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi097-79636-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:20.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:20 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi097-79618-33"}]: dispatch 2024-03-21T13:31:20.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:20 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi097-79636-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:21.465 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAiexpected=5 2024-03-21T13:31:21.465 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : seek to 14:89d3ae78:::11:head 2024-03-21T13:31:21.465 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : cursor()=14:89d3ae78:::11:head expected=14:89d3ae78:::11:head 2024-03-21T13:31:21.465 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: > 14:89d3ae78:::11:head -> 11 2024-03-21T13:31:21.465 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : entry=11 expected=11 2024-03-21T13:31:21.465 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : seek to 14:f905c69b:::2:head 2024-03-21T13:31:21.465 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : cursor()=14:f905c69b:::2:head expected=14:f905c69b:::2:head 2024-03-21T13:31:21.465 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: > 14:f905c69b:::2:head -> 2 2024-03-21T13:31:21.466 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : entry=2 expected=2 2024-03-21T13:31:21.466 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : seek to 14:e960b815:::9:head 2024-03-21T13:31:21.466 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : cursor()=14:e960b815:::9:head expected=14:e960b815:::9:head 2024-03-21T13:31:21.466 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: > 14:e960b815:::9:head -> 9 2024-03-21T13:31:21.466 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : entry=9 expected=9 2024-03-21T13:31:21.466 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : seek to 14:863748b0:::15:head 2024-03-21T13:31:21.466 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : cursor()=14:863748b0:::15:head expected=14:863748b0:::15:head 2024-03-21T13:31:21.466 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: > 14:863748b0:::15:head -> 15 2024-03-21T13:31:21.466 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : entry=15 expected=15 2024-03-21T13:31:21.466 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : seek to 14:6cac518f:::0:head 2024-03-21T13:31:21.466 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : cursor()=14:6cac518f:::0:head expected=14:6cac518f:::0:head 2024-03-21T13:31:21.466 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: > 14:6cac518f:::0:head -> 0 2024-03-21T13:31:21.466 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : entry=0 expected=0 2024-03-21T13:31:21.467 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : seek to 14:62a1935d:::14:head 2024-03-21T13:31:21.467 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : cursor()=14:62a1935d:::14:head expected=14:62a1935d:::14:head 2024-03-21T13:31:21.467 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: > 14:62a1935d:::14:head -> 14 2024-03-21T13:31:21.467 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : entry=14 expected=14 2024-03-21T13:31:21.467 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : seek to 14:5c6b0b28:::7:head 2024-03-21T13:31:21.467 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : cursor()=14:5c6b0b28:::7:head expected=14:5c6b0b28:::7:head 2024-03-21T13:31:21.467 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: > 14:5c6b0b28:::7:head -> 7 2024-03-21T13:31:21.467 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : entry=7 expected=7 2024-03-21T13:31:21.467 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : seek to 14:566253c9:::13:head 2024-03-21T13:31:21.467 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : cursor()=14:566253c9:::13:head expected=14:566253c9:::13:head 2024-03-21T13:31:21.467 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: > 14:566253c9:::13:head -> 13 2024-03-21T13:31:21.467 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: : entry=13 expected=13 2024-03-21T13:31:21.467 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: [ OK ] LibRadosList.ListObjectsCursor (281 ms) 2024-03-21T13:31:21.467 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjects 2024-03-21T13:31:21.467 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: [ OK ] LibRadosList.EnumerateObjects (106182 ms) 2024-03-21T13:31:21.467 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjectsSplit 2024-03-21T13:31:21.468 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: split 0/5 -> MIN 14:33333333::::head 2024-03-21T13:31:21.468 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: split 1/5 -> 14:33333333::::head 14:66666666::::head 2024-03-21T13:31:21.468 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: split 2/5 -> 14:66666666::::head 14:99999999::::head 2024-03-21T13:31:21.468 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: split 3/5 -> 14:99999999::::head 14:cccccccc::::head 2024-03-21T13:31:21.468 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: split 4/5 -> 14:cccccccc::::head MAX 2024-03-21T13:31:21.468 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: [ OK ] LibRadosList.EnumerateObjectsSplit (20197 ms) 2024-03-21T13:31:21.468 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: [----------] 7 tests from LibRadosList (127286 ms total) 2024-03-21T13:31:21.468 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: 2024-03-21T13:31:21.468 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: [----------] 3 tests from LibRadosListEC 2024-03-21T13:31:21.468 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: [ RUN ] LibRadosListEC.ListObjects 2024-03-21T13:31:21.468 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: [ OK ] LibRadosListEC.ListObjects (1055 ms) 2024-03-21T13:31:21.468 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsNS 2024-03-21T13:31:21.468 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: myset foo1,foo2,foo3 2024-03-21T13:31:21.468 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: foo1 2024-03-21T13:31:21.468 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: foo2 2024-03-21T13:31:21.469 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: foo3 2024-03-21T13:31:21.469 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: myset foo1,foo4,foo5 2024-03-21T13:31:21.469 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: foo4 2024-03-21T13:31:21.469 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: foo5 2024-03-21T13:31:21.469 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: foo1 2024-03-21T13:31:21.469 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: myset foo6,foo7 2024-03-21T13:31:21.469 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: foo7 2024-03-21T13:31:21.469 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: foo6 2024-03-21T13:31:21.469 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-03-21T13:31:21.469 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: ns1:foo4 2024-03-21T13:31:21.469 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: ns1:foo5 2024-03-21T13:31:21.469 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: ns2:foo7 2024-03-21T13:31:21.469 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: ns2:foo6 2024-03-21T13:31:21.469 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: ns1:foo1 2024-03-21T13:31:21.469 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: :foo1 2024-03-21T13:31:21.469 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: :foo2 2024-03-21T13:31:21.470 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: :foo3 2024-03-21T13:31:21.470 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsNS (31 ms) 2024-03-21T13:31:21.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:21 smithi097 ceph-mon[30040]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:31:21.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:21 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi097-79710-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi097-79710-2"}]': finished 2024-03-21T13:31:21.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:21 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-23", "mode": "writeback"}]': finished 2024-03-21T13:31:21.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:21 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi097-79618-33"}]': finished 2024-03-21T13:31:21.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:21 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi097-79636-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:21.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:21 smithi097 ceph-mon[30040]: osdmap e205: 8 total, 8 up, 8 in 2024-03-21T13:31:21.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:21 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2748596068' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi097-79618-33"}]: dispatch 2024-03-21T13:31:21.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:21 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi097-79618-33"}]: dispatch 2024-03-21T13:31:21.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:21 smithi097 ceph-mon[30040]: pgmap v272: 396 pgs: 40 unknown, 32 creating+peering, 324 active+clean; 584 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:31:21.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:21 smithi097 ceph-mon[39416]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:31:21.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:21 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi097-79710-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi097-79710-2"}]': finished 2024-03-21T13:31:21.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:21 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-23", "mode": "writeback"}]': finished 2024-03-21T13:31:21.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:21 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi097-79618-33"}]': finished 2024-03-21T13:31:21.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:21 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi097-79636-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:21.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:21 smithi097 ceph-mon[39416]: osdmap e205: 8 total, 8 up, 8 in 2024-03-21T13:31:21.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:21 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2748596068' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi097-79618-33"}]: dispatch 2024-03-21T13:31:21.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:21 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi097-79618-33"}]: dispatch 2024-03-21T13:31:21.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:21 smithi097 ceph-mon[39416]: pgmap v272: 396 pgs: 40 unknown, 32 creating+peering, 324 active+clean; 584 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:31:21.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:21 smithi144 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:31:21.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:21 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi097-79710-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi097-79710-2"}]': finished 2024-03-21T13:31:21.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:21 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-23", "mode": "writeback"}]': finished 2024-03-21T13:31:21.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:21 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi097-79618-33"}]': finished 2024-03-21T13:31:21.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:21 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi097-79636-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:21.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:21 smithi144 ceph-mon[33633]: osdmap e205: 8 total, 8 up, 8 in 2024-03-21T13:31:21.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:21 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2748596068' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi097-79618-33"}]: dispatch 2024-03-21T13:31:21.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:21 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi097-79618-33"}]: dispatch 2024-03-21T13:31:21.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:21 smithi144 ceph-mon[33633]: pgmap v272: 396 pgs: 40 unknown, 32 creating+peering, 324 active+clean; 584 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:31:22.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:31:21 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:31:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:31:22.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:22 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi097-79618-33"}]': finished 2024-03-21T13:31:22.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:22 smithi097 ceph-mon[39416]: osdmap e206: 8 total, 8 up, 8 in 2024-03-21T13:31:22.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:22 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2194008781' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi097-79636-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:22.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:22 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi097-79636-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:22.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:22 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3038703198' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi097-79618-34"}]: dispatch 2024-03-21T13:31:22.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:22 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi097-79618-34"}]: dispatch 2024-03-21T13:31:22.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:22 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3038703198' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi097-79618-34"}]: dispatch 2024-03-21T13:31:22.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:22 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi097-79618-34"}]: dispatch 2024-03-21T13:31:22.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:22 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3038703198' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi097-79618-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:22.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:22 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi097-79618-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:22.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:22 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:31:22.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:22 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:31:22.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:22 smithi097 ceph-mon[39416]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:31:22.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:22 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi097-79618-33"}]': finished 2024-03-21T13:31:22.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:22 smithi097 ceph-mon[30040]: osdmap e206: 8 total, 8 up, 8 in 2024-03-21T13:31:22.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:22 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2194008781' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi097-79636-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:22.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:22 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi097-79636-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:22.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:22 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3038703198' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi097-79618-34"}]: dispatch 2024-03-21T13:31:22.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:22 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi097-79618-34"}]: dispatch 2024-03-21T13:31:22.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:22 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3038703198' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi097-79618-34"}]: dispatch 2024-03-21T13:31:22.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:22 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi097-79618-34"}]: dispatch 2024-03-21T13:31:22.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:22 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3038703198' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi097-79618-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:22.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:22 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi097-79618-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:22.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:22 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:31:22.806 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:22 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:31:22.806 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:22 smithi097 ceph-mon[30040]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:31:22.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:22 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi097-79618-33"}]': finished 2024-03-21T13:31:22.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:22 smithi144 ceph-mon[33633]: osdmap e206: 8 total, 8 up, 8 in 2024-03-21T13:31:22.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:22 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2194008781' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi097-79636-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:22.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:22 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi097-79636-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:22.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:22 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3038703198' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi097-79618-34"}]: dispatch 2024-03-21T13:31:22.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:22 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi097-79618-34"}]: dispatch 2024-03-21T13:31:22.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:22 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3038703198' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi097-79618-34"}]: dispatch 2024-03-21T13:31:22.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:22 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi097-79618-34"}]: dispatch 2024-03-21T13:31:22.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:22 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3038703198' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi097-79618-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:22.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:22 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi097-79618-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:22.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:22 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:31:22.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:22 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:31:22.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:22 smithi144 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:31:23.871 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:31:23 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:31:23.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:23 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi097-79636-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:23.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:23 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi097-79618-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:31:23.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:23 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:31:23.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:23 smithi144 ceph-mon[33633]: osdmap e207: 8 total, 8 up, 8 in 2024-03-21T13:31:23.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:23 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3038703198' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi097-79618-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi097-79618-34"}]: dispatch 2024-03-21T13:31:23.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:23 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-23"}]: dispatch 2024-03-21T13:31:23.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:23 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi097-79618-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi097-79618-34"}]: dispatch 2024-03-21T13:31:23.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:23 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1904373518' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi097-79710-2"}]: dispatch 2024-03-21T13:31:23.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:23 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi097-79710-2"}]: dispatch 2024-03-21T13:31:23.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:23 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-23"}]: dispatch 2024-03-21T13:31:23.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:23 smithi144 ceph-mon[33633]: pgmap v275: 420 pgs: 64 unknown, 32 creating+peering, 324 active+clean; 584 KiB data, 614 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:31:23.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:23 smithi144 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:31:23.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:23 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi097-79710-2"}]': finished 2024-03-21T13:31:23.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:23 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-23"}]': finished 2024-03-21T13:31:23.873 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: [ RUN ] LibRadosListEC.L 0x55ca2a5b4e70 2024-03-21T13:31:23.873 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b4fa0 2024-03-21T13:31:23.873 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b50b0 2024-03-21T13:31:23.873 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b5a30 2024-03-21T13:31:23.873 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b5470 2024-03-21T13:31:23.873 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b55a0 2024-03-21T13:31:23.873 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b56d0 2024-03-21T13:31:23.873 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b5800 2024-03-21T13:31:23.873 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b6030 2024-03-21T13:31:23.873 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b5b80 2024-03-21T13:31:23.873 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b5cb0 2024-03-21T13:31:23.873 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b5dc0 2024-03-21T13:31:23.873 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b5ef0 2024-03-21T13:31:23.873 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b6630 2024-03-21T13:31:23.874 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b6180 2024-03-21T13:31:23.874 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b62d0 2024-03-21T13:31:23.874 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b6400 2024-03-21T13:31:23.874 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b6510 2024-03-21T13:31:23.874 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b6c10 2024-03-21T13:31:23.874 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b6760 2024-03-21T13:31:23.874 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b6d40 2024-03-21T13:31:23.874 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b6e90 2024-03-21T13:31:23.874 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b42d0 2024-03-21T13:31:23.874 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b4400 2024-03-21T13:31:23.874 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b4550 2024-03-21T13:31:23.874 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b6ff0 2024-03-21T13:31:23.874 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b74e0 2024-03-21T13:31:23.874 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b7610 2024-03-21T13:31:23.874 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b7bf0 2024-03-21T13:31:23.875 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b7140 2024-03-21T13:31:23.875 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b7270 2024-03-21T13:31:23.875 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b7380 2024-03-21T13:31:23.875 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b7d20 2024-03-21T13:31:23.875 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b7740 2024-03-21T13:31:23.875 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b7870 2024-03-21T13:31:23.875 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b79c0 2024-03-21T13:31:23.875 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b7e30 2024-03-21T13:31:23.875 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b7f80 2024-03-21T13:31:23.875 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b8560 2024-03-21T13:31:23.875 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b80b0 2024-03-21T13:31:23.875 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b6870 2024-03-21T13:31:23.875 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b6980 2024-03-21T13:31:23.875 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b6ad0 2024-03-21T13:31:23.875 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b8b20 2024-03-21T13:31:23.875 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b8670 2024-03-21T13:31:23.876 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b81c0 2024-03-21T13:31:23.876 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b82d0 2024-03-21T13:31:23.876 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b83e0 2024-03-21T13:31:23.876 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b9120 2024-03-21T13:31:23.876 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b8c70 2024-03-21T13:31:23.876 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b8da0 2024-03-21T13:31:23.876 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b8eb0 2024-03-21T13:31:23.876 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b9000 2024-03-21T13:31:23.876 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b7ad0 2024-03-21T13:31:23.876 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b92a0 2024-03-21T13:31:23.876 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b93f0 2024-03-21T13:31:23.876 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b99f0 2024-03-21T13:31:23.876 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b9540 2024-03-21T13:31:23.876 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b9690 2024-03-21T13:31:23.876 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b97e0 2024-03-21T13:31:23.877 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b9b00 2024-03-21T13:31:23.877 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b9ff0 2024-03-21T13:31:23.877 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5ba120 2024-03-21T13:31:23.877 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5ba700 2024-03-21T13:31:23.877 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b9c50 2024-03-21T13:31:23.877 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b9d80 2024-03-21T13:31:23.877 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b9eb0 2024-03-21T13:31:23.877 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5ba850 2024-03-21T13:31:23.877 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5ba250 2024-03-21T13:31:23.877 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5ba380 2024-03-21T13:31:23.877 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5ba490 2024-03-21T13:31:23.877 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5ba5c0 2024-03-21T13:31:23.877 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bae30 2024-03-21T13:31:23.877 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5ba980 2024-03-21T13:31:23.877 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5baab0 2024-03-21T13:31:23.877 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5babc0 2024-03-21T13:31:23.877 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bacf0 2024-03-21T13:31:23.878 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bb470 2024-03-21T13:31:23.878 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bafc0 2024-03-21T13:31:23.878 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bb110 2024-03-21T13:31:23.878 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bb220 2024-03-21T13:31:23.878 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bb350 2024-03-21T13:31:24.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:23 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi097-79636-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:24.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:23 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi097-79618-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:31:24.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:23 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:31:24.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:23 smithi097 ceph-mon[30040]: osdmap e207: 8 total, 8 up, 8 in 2024-03-21T13:31:24.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:23 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3038703198' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi097-79618-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi097-79618-34"}]: dispatch 2024-03-21T13:31:24.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:23 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-23"}]: dispatch 2024-03-21T13:31:24.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:23 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi097-79618-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi097-79618-34"}]: dispatch 2024-03-21T13:31:24.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:23 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1904373518' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi097-79710-2"}]: dispatch 2024-03-21T13:31:24.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:23 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi097-79710-2"}]: dispatch 2024-03-21T13:31:24.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:23 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-23"}]: dispatch 2024-03-21T13:31:24.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:23 smithi097 ceph-mon[30040]: pgmap v275: 420 pgs: 64 unknown, 32 creating+peering, 324 active+clean; 584 KiB data, 614 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:31:24.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:23 smithi097 ceph-mon[30040]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:31:24.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:23 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi097-79710-2"}]': finished 2024-03-21T13:31:24.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:23 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-23"}]': finished 2024-03-21T13:31:24.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:23 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi097-79636-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:24.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:23 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi097-79618-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:31:24.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:23 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:31:24.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:23 smithi097 ceph-mon[39416]: osdmap e207: 8 total, 8 up, 8 in 2024-03-21T13:31:24.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:23 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3038703198' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi097-79618-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi097-79618-34"}]: dispatch 2024-03-21T13:31:24.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:23 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-23"}]: dispatch 2024-03-21T13:31:24.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:23 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi097-79618-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi097-79618-34"}]: dispatch 2024-03-21T13:31:24.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:23 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1904373518' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi097-79710-2"}]: dispatch 2024-03-21T13:31:24.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:23 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi097-79710-2"}]: dispatch 2024-03-21T13:31:24.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:23 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-23"}]: dispatch 2024-03-21T13:31:24.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:23 smithi097 ceph-mon[39416]: pgmap v275: 420 pgs: 64 unknown, 32 creating+peering, 324 active+clean; 584 KiB data, 614 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:31:24.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:23 smithi097 ceph-mon[39416]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:31:24.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:23 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi097-79710-2"}]': finished 2024-03-21T13:31:24.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:23 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-23"}]': finished 2024-03-21T13:31:25.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:24 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1904373518' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi097-79710-2"}]: dispatch 2024-03-21T13:31:25.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:24 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:31:25.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:24 smithi097 ceph-mon[39416]: osdmap e208: 8 total, 8 up, 8 in 2024-03-21T13:31:25.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:24 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi097-79710-2"}]: dispatch 2024-03-21T13:31:25.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:24 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1777188350' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi097-79636-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:25.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:24 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi097-79618-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi097-79618-34"}]': finished 2024-03-21T13:31:25.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:24 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi097-79710-2"}]': finished 2024-03-21T13:31:25.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:24 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1777188350' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi097-79636-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:25.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:24 smithi097 ceph-mon[39416]: osdmap e209: 8 total, 8 up, 8 in 2024-03-21T13:31:25.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:24 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1904373518' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi097-79710-2"}]: dispatch 2024-03-21T13:31:25.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:24 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:31:25.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:24 smithi097 ceph-mon[30040]: osdmap e208: 8 total, 8 up, 8 in 2024-03-21T13:31:25.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:24 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi097-79710-2"}]: dispatch 2024-03-21T13:31:25.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:24 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1777188350' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi097-79636-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:25.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:24 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi097-79618-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi097-79618-34"}]': finished 2024-03-21T13:31:25.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:24 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi097-79710-2"}]': finished 2024-03-21T13:31:25.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:24 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1777188350' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi097-79636-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:25.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:24 smithi097 ceph-mon[30040]: osdmap e209: 8 total, 8 up, 8 in 2024-03-21T13:31:25.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:24 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1904373518' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi097-79710-2"}]: dispatch 2024-03-21T13:31:25.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:24 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:31:25.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:24 smithi144 ceph-mon[33633]: osdmap e208: 8 total, 8 up, 8 in 2024-03-21T13:31:25.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:24 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi097-79710-2"}]: dispatch 2024-03-21T13:31:25.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:24 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1777188350' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi097-79636-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:25.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:24 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi097-79618-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi097-79618-34"}]': finished 2024-03-21T13:31:25.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:24 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi097-79710-2"}]': finished 2024-03-21T13:31:25.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:24 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1777188350' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi097-79636-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:25.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:24 smithi144 ceph-mon[33633]: osdmap e209: 8 total, 8 up, 8 in 2024-03-21T13:31:26.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:25 smithi097 ceph-mon[39416]: pgmap v278: 428 pgs: 40 unknown, 388 active+clean; 584 KiB data, 590 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-21T13:31:26.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:25 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:31:26.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:25 smithi097 ceph-mon[39416]: osdmap e210: 8 total, 8 up, 8 in 2024-03-21T13:31:26.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:25 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3524198090' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79710-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:26.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:25 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:26.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:25 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79710-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:26.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:25 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:26.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:25 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/261855635' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi097-79636-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:26.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:25 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi097-79636-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:26.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:25 smithi097 ceph-mon[30040]: pgmap v278: 428 pgs: 40 unknown, 388 active+clean; 584 KiB data, 590 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-21T13:31:26.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:25 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:31:26.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:25 smithi097 ceph-mon[30040]: osdmap e210: 8 total, 8 up, 8 in 2024-03-21T13:31:26.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:25 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3524198090' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79710-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:26.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:25 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:26.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:25 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79710-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:26.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:25 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:26.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:25 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/261855635' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi097-79636-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:26.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:25 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi097-79636-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:26.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:25 smithi144 ceph-mon[33633]: pgmap v278: 428 pgs: 40 unknown, 388 active+clean; 584 KiB data, 590 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-21T13:31:26.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:25 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:31:26.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:25 smithi144 ceph-mon[33633]: osdmap e210: 8 total, 8 up, 8 in 2024-03-21T13:31:26.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:25 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3524198090' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79710-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:26.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:25 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:26.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:25 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79710-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:26.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:25 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:26.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:25 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/261855635' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi097-79636-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:26.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:25 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi097-79636-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:27.259 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: staistObjectsStart 2024-03-21T13:31:27.259 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: 1 0 2024-03-21T13:31:27.259 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: 10 0 2024-03-21T13:31:27.259 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: 13 0 2024-03-21T13:31:27.259 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: 7 0 2024-03-21T13:31:27.259 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: 14 0 2024-03-21T13:31:27.259 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: 0 0 2024-03-21T13:31:27.259 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: 15 0 2024-03-21T13:31:27.259 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: 11 0 2024-03-21T13:31:27.259 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: 5 0 2024-03-21T13:31:27.259 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: 8 0 2024-03-21T13:31:27.260 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: 6 0 2024-03-21T13:31:27.260 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: 3 0 2024-03-21T13:31:27.260 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: 4 0 2024-03-21T13:31:27.260 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: 12 0 2024-03-21T13:31:27.260 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: 9 0 2024-03-21T13:31:27.260 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: 2 0 2024-03-21T13:31:27.260 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-03-21T13:31:27.260 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsStart (43 ms) 2024-03-21T13:31:27.260 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: [----------] 3 tests from LibRadosListEC (1129 ms total) 2024-03-21T13:31:27.260 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: 2024-03-21T13:31:27.260 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: [----------] 1 test from LibRadosListNP 2024-03-21T13:31:27.260 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: [ RUN ] LibRadosListNP.ListObjectsError 2024-03-21T13:31:27.260 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: [ OK ] LibRadosListNP.ListObjectsError (2852 ms) 2024-03-21T13:31:27.260 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: [----------] 1 test from LibRadosListNP (2852 ms total) 2024-03-21T13:31:27.260 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: 2024-03-21T13:31:27.261 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: [----------] Global test environment tear-down 2024-03-21T13:31:27.261 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: [==========] 11 tests from 3 test suites ran. (147778 ms total) 2024-03-21T13:31:27.261 INFO:tasks.workunit.client.0.smithi097.stdout: api_list: [ PASSED ] 11 tests. 2024-03-21T13:31:27.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:27 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79710-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:27.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:27 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:27.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:27 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi097-79636-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:27.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:27 smithi097 ceph-mon[30040]: osdmap e211: 8 total, 8 up, 8 in 2024-03-21T13:31:27.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:27 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3038703198' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi097-79618-34"}]: dispatch 2024-03-21T13:31:27.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:27 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi097-79618-34"}]: dispatch 2024-03-21T13:31:27.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:27 smithi097 ceph-mon[30040]: pgmap v281: 516 pgs: 128 unknown, 388 active+clean; 584 KiB data, 590 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-21T13:31:27.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:27 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3524198090' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi097-79710-3","pool2":"test-rados-api-smithi097-79710-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-03-21T13:31:27.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:27 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi097-79710-3","pool2":"test-rados-api-smithi097-79710-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-03-21T13:31:27.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:27 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi097-79618-34"}]': finished 2024-03-21T13:31:27.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:27 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi097-79710-3","pool2":"test-rados-api-smithi097-79710-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-03-21T13:31:27.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:27 smithi097 ceph-mon[30040]: osdmap e212: 8 total, 8 up, 8 in 2024-03-21T13:31:27.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:27 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3038703198' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi097-79618-34"}]: dispatch 2024-03-21T13:31:27.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:27 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi097-79618-34"}]: dispatch 2024-03-21T13:31:27.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:27 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:31:27.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:27 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:31:27.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:27 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79710-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:27.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:27 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:27.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:27 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi097-79636-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:27.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:27 smithi097 ceph-mon[39416]: osdmap e211: 8 total, 8 up, 8 in 2024-03-21T13:31:27.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:27 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3038703198' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi097-79618-34"}]: dispatch 2024-03-21T13:31:27.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:27 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi097-79618-34"}]: dispatch 2024-03-21T13:31:27.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:27 smithi097 ceph-mon[39416]: pgmap v281: 516 pgs: 128 unknown, 388 active+clean; 584 KiB data, 590 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-21T13:31:27.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:27 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3524198090' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi097-79710-3","pool2":"test-rados-api-smithi097-79710-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-03-21T13:31:27.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:27 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi097-79710-3","pool2":"test-rados-api-smithi097-79710-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-03-21T13:31:27.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:27 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi097-79618-34"}]': finished 2024-03-21T13:31:27.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:27 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi097-79710-3","pool2":"test-rados-api-smithi097-79710-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-03-21T13:31:27.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:27 smithi097 ceph-mon[39416]: osdmap e212: 8 total, 8 up, 8 in 2024-03-21T13:31:27.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:27 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3038703198' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi097-79618-34"}]: dispatch 2024-03-21T13:31:27.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:27 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi097-79618-34"}]: dispatch 2024-03-21T13:31:27.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:27 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:31:27.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:27 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:31:27.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:27 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79710-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:27.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:27 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:27.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:27 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi097-79636-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:27.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:27 smithi144 ceph-mon[33633]: osdmap e211: 8 total, 8 up, 8 in 2024-03-21T13:31:27.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:27 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3038703198' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi097-79618-34"}]: dispatch 2024-03-21T13:31:27.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:27 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi097-79618-34"}]: dispatch 2024-03-21T13:31:27.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:27 smithi144 ceph-mon[33633]: pgmap v281: 516 pgs: 128 unknown, 388 active+clean; 584 KiB data, 590 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-21T13:31:27.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:27 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3524198090' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi097-79710-3","pool2":"test-rados-api-smithi097-79710-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-03-21T13:31:27.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:27 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi097-79710-3","pool2":"test-rados-api-smithi097-79710-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-03-21T13:31:27.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:27 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi097-79618-34"}]': finished 2024-03-21T13:31:27.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:27 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi097-79710-3","pool2":"test-rados-api-smithi097-79710-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-03-21T13:31:27.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:27 smithi144 ceph-mon[33633]: osdmap e212: 8 total, 8 up, 8 in 2024-03-21T13:31:27.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:27 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3038703198' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi097-79618-34"}]: dispatch 2024-03-21T13:31:27.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:27 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi097-79618-34"}]: dispatch 2024-03-21T13:31:27.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:27 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:31:27.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:27 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:31:28.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:28 smithi097 ceph-mon[30040]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:31:28.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:28 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi097-79618-34"}]': finished 2024-03-21T13:31:28.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:28 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-25", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:31:28.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:28 smithi097 ceph-mon[30040]: osdmap e213: 8 total, 8 up, 8 in 2024-03-21T13:31:28.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:28 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-25"}]: dispatch 2024-03-21T13:31:28.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:28 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-25"}]: dispatch 2024-03-21T13:31:28.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:28 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1827099008' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi097-79618-35"}]: dispatch 2024-03-21T13:31:28.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:28 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi097-79618-35"}]: dispatch 2024-03-21T13:31:28.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:28 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1827099008' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi097-79618-35"}]: dispatch 2024-03-21T13:31:28.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:28 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi097-79618-35"}]: dispatch 2024-03-21T13:31:28.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:28 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1827099008' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi097-79618-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:28.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:28 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi097-79618-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:28.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:28 smithi097 ceph-mon[39416]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:31:28.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:28 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi097-79618-34"}]': finished 2024-03-21T13:31:28.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:28 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-25", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:31:28.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:28 smithi097 ceph-mon[39416]: osdmap e213: 8 total, 8 up, 8 in 2024-03-21T13:31:28.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:28 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-25"}]: dispatch 2024-03-21T13:31:28.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:28 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-25"}]: dispatch 2024-03-21T13:31:28.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:28 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1827099008' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi097-79618-35"}]: dispatch 2024-03-21T13:31:28.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:28 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi097-79618-35"}]: dispatch 2024-03-21T13:31:28.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:28 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1827099008' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi097-79618-35"}]: dispatch 2024-03-21T13:31:28.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:28 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi097-79618-35"}]: dispatch 2024-03-21T13:31:28.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:28 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1827099008' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi097-79618-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:28.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:28 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi097-79618-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:28.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:28 smithi144 ceph-mon[33633]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:31:28.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:28 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi097-79618-34"}]': finished 2024-03-21T13:31:28.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:28 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-25", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:31:28.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:28 smithi144 ceph-mon[33633]: osdmap e213: 8 total, 8 up, 8 in 2024-03-21T13:31:28.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:28 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-25"}]: dispatch 2024-03-21T13:31:28.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:28 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-25"}]: dispatch 2024-03-21T13:31:28.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:28 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1827099008' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi097-79618-35"}]: dispatch 2024-03-21T13:31:28.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:28 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi097-79618-35"}]: dispatch 2024-03-21T13:31:28.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:28 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1827099008' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi097-79618-35"}]: dispatch 2024-03-21T13:31:28.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:28 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi097-79618-35"}]: dispatch 2024-03-21T13:31:28.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:28 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1827099008' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi097-79618-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:28.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:28 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi097-79618-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:29.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:29 smithi097 ceph-mon[39416]: pgmap v284: 420 pgs: 420 active+clean; 584 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-21T13:31:29.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:29 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-25"}]': finished 2024-03-21T13:31:29.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:29 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi097-79618-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:31:29.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:29 smithi097 ceph-mon[39416]: osdmap e214: 8 total, 8 up, 8 in 2024-03-21T13:31:29.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:29 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1827099008' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi097-79618-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi097-79618-35"}]: dispatch 2024-03-21T13:31:29.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:29 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-25", "mode": "writeback"}]: dispatch 2024-03-21T13:31:29.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:29 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi097-79618-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi097-79618-35"}]: dispatch 2024-03-21T13:31:29.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:29 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-25", "mode": "writeback"}]: dispatch 2024-03-21T13:31:29.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:29 smithi097 ceph-mon[30040]: pgmap v284: 420 pgs: 420 active+clean; 584 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-21T13:31:29.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:29 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-25"}]': finished 2024-03-21T13:31:29.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:29 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi097-79618-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:31:29.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:29 smithi097 ceph-mon[30040]: osdmap e214: 8 total, 8 up, 8 in 2024-03-21T13:31:29.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:29 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1827099008' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi097-79618-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi097-79618-35"}]: dispatch 2024-03-21T13:31:29.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:29 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-25", "mode": "writeback"}]: dispatch 2024-03-21T13:31:29.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:29 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi097-79618-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi097-79618-35"}]: dispatch 2024-03-21T13:31:29.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:29 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-25", "mode": "writeback"}]: dispatch 2024-03-21T13:31:29.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:29 smithi144 ceph-mon[33633]: pgmap v284: 420 pgs: 420 active+clean; 584 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-21T13:31:29.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:29 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-25"}]': finished 2024-03-21T13:31:29.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:29 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi097-79618-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:31:29.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:29 smithi144 ceph-mon[33633]: osdmap e214: 8 total, 8 up, 8 in 2024-03-21T13:31:29.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:29 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1827099008' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi097-79618-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi097-79618-35"}]: dispatch 2024-03-21T13:31:29.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:29 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-25", "mode": "writeback"}]: dispatch 2024-03-21T13:31:29.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:29 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi097-79618-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi097-79618-35"}]: dispatch 2024-03-21T13:31:29.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:29 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-25", "mode": "writeback"}]: dispatch 2024-03-21T13:31:30.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:30 smithi097 ceph-mon[39416]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:31:30.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:30 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-25", "mode": "writeback"}]': finished 2024-03-21T13:31:30.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:30 smithi097 ceph-mon[30040]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:31:30.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:30 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-25", "mode": "writeback"}]': finished 2024-03-21T13:31:30.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:30 smithi144 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:31:30.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:30 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-25", "mode": "writeback"}]': finished 2024-03-21T13:31:31.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:31 smithi097 ceph-mon[39416]: pgmap v287: 356 pgs: 356 active+clean; 584 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-21T13:31:31.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:31 smithi097 ceph-mon[39416]: osdmap e215: 8 total, 8 up, 8 in 2024-03-21T13:31:31.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:31 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:31:31.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:31 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:31:31.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:31 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi097-79618-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi097-79618-35"}]': finished 2024-03-21T13:31:31.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:31 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:31:31.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:31 smithi097 ceph-mon[39416]: osdmap e216: 8 total, 8 up, 8 in 2024-03-21T13:31:31.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:31 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-25"}]: dispatch 2024-03-21T13:31:31.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:31 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-25"}]: dispatch 2024-03-21T13:31:31.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:31 smithi097 ceph-mon[30040]: pgmap v287: 356 pgs: 356 active+clean; 584 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-21T13:31:31.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:31 smithi097 ceph-mon[30040]: osdmap e215: 8 total, 8 up, 8 in 2024-03-21T13:31:31.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:31 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:31:31.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:31 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:31:31.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:31 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi097-79618-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi097-79618-35"}]': finished 2024-03-21T13:31:31.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:31 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:31:31.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:31 smithi097 ceph-mon[30040]: osdmap e216: 8 total, 8 up, 8 in 2024-03-21T13:31:31.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:31 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-25"}]: dispatch 2024-03-21T13:31:31.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:31 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-25"}]: dispatch 2024-03-21T13:31:31.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:31 smithi144 ceph-mon[33633]: pgmap v287: 356 pgs: 356 active+clean; 584 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-21T13:31:31.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:31 smithi144 ceph-mon[33633]: osdmap e215: 8 total, 8 up, 8 in 2024-03-21T13:31:31.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:31 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:31:31.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:31 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:31:31.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:31 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi097-79618-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi097-79618-35"}]': finished 2024-03-21T13:31:31.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:31 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:31:31.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:31 smithi144 ceph-mon[33633]: osdmap e216: 8 total, 8 up, 8 in 2024-03-21T13:31:31.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:31 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-25"}]: dispatch 2024-03-21T13:31:31.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:31 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-25"}]: dispatch 2024-03-21T13:31:32.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:31:31 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:31:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:31:32.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:32 smithi097 ceph-mon[30040]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:31:32.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:32 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-25"}]': finished 2024-03-21T13:31:32.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:32 smithi097 ceph-mon[30040]: osdmap e217: 8 total, 8 up, 8 in 2024-03-21T13:31:32.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:32 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3016927670' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi097-79636-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:32.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:32 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:31:32.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:32 smithi097 ceph-mon[39416]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:31:32.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:32 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-25"}]': finished 2024-03-21T13:31:32.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:32 smithi097 ceph-mon[39416]: osdmap e217: 8 total, 8 up, 8 in 2024-03-21T13:31:32.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:32 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3016927670' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi097-79636-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:32.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:32 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:31:32.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:32 smithi144 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:31:32.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:32 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-25"}]': finished 2024-03-21T13:31:32.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:32 smithi144 ceph-mon[33633]: osdmap e217: 8 total, 8 up, 8 in 2024-03-21T13:31:32.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:32 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3016927670' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi097-79636-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:32.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:32 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:31:33.871 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:31:33 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:31:34.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:34 smithi097 ceph-mon[39416]: pgmap v290: 364 pgs: 40 unknown, 324 active+clean; 584 KiB data, 595 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:31:34.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:34 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:31:34.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:34 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:31:34.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:34 smithi097 ceph-mon[39416]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:31:34.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:34 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3016927670' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi097-79636-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:34.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:34 smithi097 ceph-mon[39416]: osdmap e218: 8 total, 8 up, 8 in 2024-03-21T13:31:34.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:34 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1827099008' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi097-79618-35"}]: dispatch 2024-03-21T13:31:34.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:34 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi097-79618-35"}]: dispatch 2024-03-21T13:31:34.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:34 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:31:34.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:34 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:31:34.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:34 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:31:34.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:34 smithi097 ceph-mon[30040]: pgmap v290: 364 pgs: 40 unknown, 324 active+clean; 584 KiB data, 595 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:31:34.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:34 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:31:34.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:34 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:31:34.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:34 smithi097 ceph-mon[30040]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:31:34.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:34 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3016927670' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi097-79636-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:34.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:34 smithi097 ceph-mon[30040]: osdmap e218: 8 total, 8 up, 8 in 2024-03-21T13:31:34.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:34 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1827099008' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi097-79618-35"}]: dispatch 2024-03-21T13:31:34.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:34 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi097-79618-35"}]: dispatch 2024-03-21T13:31:34.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:34 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:31:34.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:34 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:31:34.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:34 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:31:34.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:34 smithi144 ceph-mon[33633]: pgmap v290: 364 pgs: 40 unknown, 324 active+clean; 584 KiB data, 595 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:31:34.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:34 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:31:34.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:34 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:31:34.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:34 smithi144 ceph-mon[33633]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:31:34.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:34 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3016927670' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi097-79636-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:34.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:34 smithi144 ceph-mon[33633]: osdmap e218: 8 total, 8 up, 8 in 2024-03-21T13:31:34.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:34 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1827099008' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi097-79618-35"}]: dispatch 2024-03-21T13:31:34.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:34 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi097-79618-35"}]: dispatch 2024-03-21T13:31:34.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:34 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:31:34.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:34 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:31:34.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:34 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:31:35.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:35 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:31:35.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:35 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi097-79618-35"}]': finished 2024-03-21T13:31:35.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:35 smithi097 ceph-mon[39416]: osdmap e219: 8 total, 8 up, 8 in 2024-03-21T13:31:35.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:35 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1827099008' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi097-79618-35"}]: dispatch 2024-03-21T13:31:35.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:35 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:35.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:35 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi097-79618-35"}]: dispatch 2024-03-21T13:31:35.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:35 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:35.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:35 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:31:35.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:35 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi097-79618-35"}]': finished 2024-03-21T13:31:35.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:35 smithi097 ceph-mon[30040]: osdmap e219: 8 total, 8 up, 8 in 2024-03-21T13:31:35.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:35 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1827099008' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi097-79618-35"}]: dispatch 2024-03-21T13:31:35.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:35 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:35.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:35 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi097-79618-35"}]: dispatch 2024-03-21T13:31:35.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:35 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:35.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:35 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:31:35.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:35 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi097-79618-35"}]': finished 2024-03-21T13:31:35.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:35 smithi144 ceph-mon[33633]: osdmap e219: 8 total, 8 up, 8 in 2024-03-21T13:31:35.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:35 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1827099008' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi097-79618-35"}]: dispatch 2024-03-21T13:31:35.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:35 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:35.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:35 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi097-79618-35"}]: dispatch 2024-03-21T13:31:35.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:35 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:36.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:36 smithi097 ceph-mon[39416]: pgmap v293: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 563 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:31:36.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:36 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi097-79618-35"}]': finished 2024-03-21T13:31:36.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:36 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:36.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:36 smithi097 ceph-mon[39416]: osdmap e220: 8 total, 8 up, 8 in 2024-03-21T13:31:36.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:36 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3689591702' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi097-79636-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:36.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:36 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi097-79636-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:36.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:36 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3905813033' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi097-79618-36"}]: dispatch 2024-03-21T13:31:36.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:36 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi097-79618-36"}]: dispatch 2024-03-21T13:31:36.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:36 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3905813033' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi097-79618-36"}]: dispatch 2024-03-21T13:31:36.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:36 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi097-79618-36"}]: dispatch 2024-03-21T13:31:36.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:36 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3905813033' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi097-79618-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:36.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:36 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi097-79618-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:36.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:36 smithi097 ceph-mon[30040]: pgmap v293: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 563 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:31:36.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:36 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi097-79618-35"}]': finished 2024-03-21T13:31:36.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:36 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:36.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:36 smithi097 ceph-mon[30040]: osdmap e220: 8 total, 8 up, 8 in 2024-03-21T13:31:36.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:36 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3689591702' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi097-79636-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:36.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:36 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi097-79636-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:36.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:36 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3905813033' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi097-79618-36"}]: dispatch 2024-03-21T13:31:36.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:36 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi097-79618-36"}]: dispatch 2024-03-21T13:31:36.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:36 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3905813033' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi097-79618-36"}]: dispatch 2024-03-21T13:31:36.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:36 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi097-79618-36"}]: dispatch 2024-03-21T13:31:36.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:36 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3905813033' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi097-79618-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:36.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:36 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi097-79618-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:36.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:36 smithi144 ceph-mon[33633]: pgmap v293: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 563 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:31:36.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:36 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi097-79618-35"}]': finished 2024-03-21T13:31:36.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:36 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:36.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:36 smithi144 ceph-mon[33633]: osdmap e220: 8 total, 8 up, 8 in 2024-03-21T13:31:36.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:36 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3689591702' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi097-79636-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:36.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:36 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi097-79636-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:36.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:36 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3905813033' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi097-79618-36"}]: dispatch 2024-03-21T13:31:36.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:36 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi097-79618-36"}]: dispatch 2024-03-21T13:31:36.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:36 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3905813033' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi097-79618-36"}]: dispatch 2024-03-21T13:31:36.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:36 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi097-79618-36"}]: dispatch 2024-03-21T13:31:36.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:36 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3905813033' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi097-79618-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:36.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:36 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi097-79618-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:37.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:37 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi097-79636-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:37.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:37 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi097-79618-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:31:37.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:37 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3905813033' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi097-79618-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi097-79618-36"}]: dispatch 2024-03-21T13:31:37.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:37 smithi097 ceph-mon[39416]: osdmap e221: 8 total, 8 up, 8 in 2024-03-21T13:31:37.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:37 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi097-79618-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi097-79618-36"}]: dispatch 2024-03-21T13:31:37.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:37 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:31:37.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:37 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:31:37.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:37 smithi097 ceph-mon[39416]: pgmap v296: 356 pgs: 64 unknown, 292 active+clean; 584 KiB data, 563 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:31:37.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:37 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi097-79636-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:37.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:37 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi097-79618-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:31:37.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:37 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3905813033' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi097-79618-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi097-79618-36"}]: dispatch 2024-03-21T13:31:37.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:37 smithi097 ceph-mon[30040]: osdmap e221: 8 total, 8 up, 8 in 2024-03-21T13:31:37.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:37 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi097-79618-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi097-79618-36"}]: dispatch 2024-03-21T13:31:37.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:37 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:31:37.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:37 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:31:37.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:37 smithi097 ceph-mon[30040]: pgmap v296: 356 pgs: 64 unknown, 292 active+clean; 584 KiB data, 563 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:31:37.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:37 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi097-79636-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:37.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:37 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi097-79618-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:31:37.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:37 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3905813033' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi097-79618-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi097-79618-36"}]: dispatch 2024-03-21T13:31:37.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:37 smithi144 ceph-mon[33633]: osdmap e221: 8 total, 8 up, 8 in 2024-03-21T13:31:37.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:37 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi097-79618-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi097-79618-36"}]: dispatch 2024-03-21T13:31:37.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:37 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:31:37.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:37 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:31:37.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:37 smithi144 ceph-mon[33633]: pgmap v296: 356 pgs: 64 unknown, 292 active+clean; 584 KiB data, 563 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:31:38.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:38 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-27", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:31:38.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:38 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-27"}]: dispatch 2024-03-21T13:31:38.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:38 smithi097 ceph-mon[39416]: osdmap e222: 8 total, 8 up, 8 in 2024-03-21T13:31:38.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:38 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-27"}]: dispatch 2024-03-21T13:31:38.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:38 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3253170645' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi097-79636-46"}]: dispatch 2024-03-21T13:31:38.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:38 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi097-79636-46"}]: dispatch 2024-03-21T13:31:38.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:38 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3253170645' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi097-79636-46"}]: dispatch 2024-03-21T13:31:38.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:38 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi097-79636-46"}]: dispatch 2024-03-21T13:31:38.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:38 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3253170645' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi097-79636-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:38.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:38 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi097-79636-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:38.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:38 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-27", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:31:38.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:38 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-27"}]: dispatch 2024-03-21T13:31:38.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:38 smithi097 ceph-mon[30040]: osdmap e222: 8 total, 8 up, 8 in 2024-03-21T13:31:38.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:38 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-27"}]: dispatch 2024-03-21T13:31:38.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:38 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3253170645' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi097-79636-46"}]: dispatch 2024-03-21T13:31:38.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:38 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi097-79636-46"}]: dispatch 2024-03-21T13:31:38.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:38 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3253170645' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi097-79636-46"}]: dispatch 2024-03-21T13:31:38.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:38 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi097-79636-46"}]: dispatch 2024-03-21T13:31:38.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:38 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3253170645' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi097-79636-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:38.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:38 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi097-79636-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:38.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:38 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-27", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:31:38.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:38 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-27"}]: dispatch 2024-03-21T13:31:38.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:38 smithi144 ceph-mon[33633]: osdmap e222: 8 total, 8 up, 8 in 2024-03-21T13:31:38.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:38 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-27"}]: dispatch 2024-03-21T13:31:38.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:38 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3253170645' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi097-79636-46"}]: dispatch 2024-03-21T13:31:38.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:38 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi097-79636-46"}]: dispatch 2024-03-21T13:31:38.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:38 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3253170645' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi097-79636-46"}]: dispatch 2024-03-21T13:31:38.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:38 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi097-79636-46"}]: dispatch 2024-03-21T13:31:38.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:38 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3253170645' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi097-79636-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:38.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:38 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi097-79636-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:39.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:39 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi097-79618-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi097-79618-36"}]': finished 2024-03-21T13:31:39.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:39 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-27"}]': finished 2024-03-21T13:31:39.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:39 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi097-79636-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:31:39.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:39 smithi144 ceph-mon[33633]: osdmap e223: 8 total, 8 up, 8 in 2024-03-21T13:31:39.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:39 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-27", "mode": "writeback"}]: dispatch 2024-03-21T13:31:39.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:39 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3253170645' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi097-79636-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi097-79636-46"}]: dispatch 2024-03-21T13:31:39.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:39 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-27", "mode": "writeback"}]: dispatch 2024-03-21T13:31:39.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:39 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi097-79636-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi097-79636-46"}]: dispatch 2024-03-21T13:31:39.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:39 smithi144 ceph-mon[33633]: pgmap v299: 332 pgs: 8 creating+peering, 324 active+clean; 584 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-21T13:31:39.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:39 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi097-79618-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi097-79618-36"}]': finished 2024-03-21T13:31:39.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:39 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-27"}]': finished 2024-03-21T13:31:39.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:39 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi097-79636-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:31:39.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:39 smithi097 ceph-mon[39416]: osdmap e223: 8 total, 8 up, 8 in 2024-03-21T13:31:39.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:39 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-27", "mode": "writeback"}]: dispatch 2024-03-21T13:31:39.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:39 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3253170645' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi097-79636-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi097-79636-46"}]: dispatch 2024-03-21T13:31:39.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:39 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-27", "mode": "writeback"}]: dispatch 2024-03-21T13:31:39.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:39 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi097-79636-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi097-79636-46"}]: dispatch 2024-03-21T13:31:39.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:39 smithi097 ceph-mon[39416]: pgmap v299: 332 pgs: 8 creating+peering, 324 active+clean; 584 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-21T13:31:39.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:39 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi097-79618-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi097-79618-36"}]': finished 2024-03-21T13:31:39.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:39 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-27"}]': finished 2024-03-21T13:31:39.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:39 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi097-79636-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:31:39.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:39 smithi097 ceph-mon[30040]: osdmap e223: 8 total, 8 up, 8 in 2024-03-21T13:31:39.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:39 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-27", "mode": "writeback"}]: dispatch 2024-03-21T13:31:39.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:39 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3253170645' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi097-79636-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi097-79636-46"}]: dispatch 2024-03-21T13:31:39.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:39 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-27", "mode": "writeback"}]: dispatch 2024-03-21T13:31:39.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:39 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi097-79636-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi097-79636-46"}]: dispatch 2024-03-21T13:31:39.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:39 smithi097 ceph-mon[30040]: pgmap v299: 332 pgs: 8 creating+peering, 324 active+clean; 584 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-21T13:31:40.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:40 smithi144 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:31:40.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:40 smithi144 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:31:40.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:40 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-27", "mode": "writeback"}]': finished 2024-03-21T13:31:40.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:40 smithi144 ceph-mon[33633]: osdmap e224: 8 total, 8 up, 8 in 2024-03-21T13:31:40.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:40 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:31:40.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:40 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:31:40.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:40 smithi097 ceph-mon[39416]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:31:40.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:40 smithi097 ceph-mon[39416]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:31:40.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:40 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-27", "mode": "writeback"}]': finished 2024-03-21T13:31:40.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:40 smithi097 ceph-mon[39416]: osdmap e224: 8 total, 8 up, 8 in 2024-03-21T13:31:40.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:40 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:31:40.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:40 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:31:40.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:40 smithi097 ceph-mon[30040]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:31:40.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:40 smithi097 ceph-mon[30040]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:31:40.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:40 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-27", "mode": "writeback"}]': finished 2024-03-21T13:31:40.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:40 smithi097 ceph-mon[30040]: osdmap e224: 8 total, 8 up, 8 in 2024-03-21T13:31:40.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:40 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:31:40.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:40 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:31:41.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:41 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi097-79636-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi097-79636-46"}]': finished 2024-03-21T13:31:41.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:41 smithi144 ceph-mon[33633]: osdmap e225: 8 total, 8 up, 8 in 2024-03-21T13:31:41.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:41 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3905813033' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi097-79618-36"}]: dispatch 2024-03-21T13:31:41.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:41 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi097-79618-36"}]: dispatch 2024-03-21T13:31:41.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:41 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:31:41.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:41 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:31:41.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:41 smithi144 ceph-mon[33633]: pgmap v302: 332 pgs: 8 unknown, 324 active+clean; 584 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-21T13:31:41.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:41 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi097-79636-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi097-79636-46"}]': finished 2024-03-21T13:31:41.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:41 smithi097 ceph-mon[39416]: osdmap e225: 8 total, 8 up, 8 in 2024-03-21T13:31:41.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:41 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3905813033' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi097-79618-36"}]: dispatch 2024-03-21T13:31:41.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:41 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi097-79618-36"}]: dispatch 2024-03-21T13:31:41.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:41 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:31:41.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:41 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:31:41.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:41 smithi097 ceph-mon[39416]: pgmap v302: 332 pgs: 8 unknown, 324 active+clean; 584 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-21T13:31:41.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:41 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi097-79636-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi097-79636-46"}]': finished 2024-03-21T13:31:41.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:41 smithi097 ceph-mon[30040]: osdmap e225: 8 total, 8 up, 8 in 2024-03-21T13:31:41.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:41 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3905813033' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi097-79618-36"}]: dispatch 2024-03-21T13:31:41.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:41 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi097-79618-36"}]: dispatch 2024-03-21T13:31:41.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:41 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:31:41.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:41 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:31:41.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:41 smithi097 ceph-mon[30040]: pgmap v302: 332 pgs: 8 unknown, 324 active+clean; 584 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-21T13:31:42.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:31:41 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:31:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:31:42.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:42 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi097-79618-36"}]': finished 2024-03-21T13:31:42.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:42 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:31:42.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:42 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3905813033' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi097-79618-36"}]: dispatch 2024-03-21T13:31:42.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:42 smithi144 ceph-mon[33633]: osdmap e226: 8 total, 8 up, 8 in 2024-03-21T13:31:42.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:42 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-27"}]: dispatch 2024-03-21T13:31:42.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:42 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi097-79618-36"}]: dispatch 2024-03-21T13:31:42.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:42 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-27"}]: dispatch 2024-03-21T13:31:42.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:42 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi097-79618-36"}]': finished 2024-03-21T13:31:42.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:42 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:31:42.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:42 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3905813033' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi097-79618-36"}]: dispatch 2024-03-21T13:31:42.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:42 smithi097 ceph-mon[39416]: osdmap e226: 8 total, 8 up, 8 in 2024-03-21T13:31:42.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:42 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-27"}]: dispatch 2024-03-21T13:31:42.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:42 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi097-79618-36"}]: dispatch 2024-03-21T13:31:42.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:42 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-27"}]: dispatch 2024-03-21T13:31:42.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:42 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi097-79618-36"}]': finished 2024-03-21T13:31:42.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:42 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:31:42.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:42 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3905813033' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi097-79618-36"}]: dispatch 2024-03-21T13:31:42.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:42 smithi097 ceph-mon[30040]: osdmap e226: 8 total, 8 up, 8 in 2024-03-21T13:31:42.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:42 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-27"}]: dispatch 2024-03-21T13:31:42.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:42 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi097-79618-36"}]: dispatch 2024-03-21T13:31:42.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:42 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-27"}]: dispatch 2024-03-21T13:31:43.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:31:43 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:31:43.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:43 smithi144 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:31:43.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:43 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi097-79618-36"}]': finished 2024-03-21T13:31:43.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:43 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-27"}]': finished 2024-03-21T13:31:43.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:43 smithi144 ceph-mon[33633]: osdmap e227: 8 total, 8 up, 8 in 2024-03-21T13:31:43.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:43 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3253170645' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi097-79636-46"}]: dispatch 2024-03-21T13:31:43.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:43 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi097-79636-46"}]: dispatch 2024-03-21T13:31:43.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:43 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3525649546' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi097-79618-37"}]: dispatch 2024-03-21T13:31:43.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:43 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3525649546' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi097-79618-37"}]: dispatch 2024-03-21T13:31:43.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:43 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3525649546' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi097-79618-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:43.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:43 smithi144 ceph-mon[33633]: pgmap v305: 324 pgs: 324 active+clean; 584 KiB data, 564 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:31:43.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:43 smithi097 ceph-mon[39416]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:31:43.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:43 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi097-79618-36"}]': finished 2024-03-21T13:31:43.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:43 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-27"}]': finished 2024-03-21T13:31:43.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:43 smithi097 ceph-mon[39416]: osdmap e227: 8 total, 8 up, 8 in 2024-03-21T13:31:43.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:43 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3253170645' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi097-79636-46"}]: dispatch 2024-03-21T13:31:43.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:43 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi097-79636-46"}]: dispatch 2024-03-21T13:31:43.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:43 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3525649546' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi097-79618-37"}]: dispatch 2024-03-21T13:31:43.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:43 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3525649546' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi097-79618-37"}]: dispatch 2024-03-21T13:31:43.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:43 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3525649546' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi097-79618-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:43.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:43 smithi097 ceph-mon[39416]: pgmap v305: 324 pgs: 324 active+clean; 584 KiB data, 564 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:31:43.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:43 smithi097 ceph-mon[30040]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:31:43.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:43 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi097-79618-36"}]': finished 2024-03-21T13:31:43.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:43 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-27"}]': finished 2024-03-21T13:31:43.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:43 smithi097 ceph-mon[30040]: osdmap e227: 8 total, 8 up, 8 in 2024-03-21T13:31:43.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:43 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3253170645' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi097-79636-46"}]: dispatch 2024-03-21T13:31:43.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:43 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi097-79636-46"}]: dispatch 2024-03-21T13:31:43.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:43 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3525649546' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi097-79618-37"}]: dispatch 2024-03-21T13:31:43.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:43 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3525649546' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi097-79618-37"}]: dispatch 2024-03-21T13:31:43.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:43 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3525649546' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi097-79618-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:43.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:43 smithi097 ceph-mon[30040]: pgmap v305: 324 pgs: 324 active+clean; 584 KiB data, 564 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:31:44.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:44 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi097-79636-46"}]': finished 2024-03-21T13:31:44.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:44 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3525649546' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi097-79618-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:31:44.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:44 smithi144 ceph-mon[33633]: osdmap e228: 8 total, 8 up, 8 in 2024-03-21T13:31:44.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:44 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3253170645' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi097-79636-46"}]: dispatch 2024-03-21T13:31:44.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:44 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3525649546' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi097-79618-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi097-79618-37"}]: dispatch 2024-03-21T13:31:44.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:44 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi097-79636-46"}]: dispatch 2024-03-21T13:31:44.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:44 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:31:45.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:44 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi097-79636-46"}]': finished 2024-03-21T13:31:45.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:44 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3525649546' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi097-79618-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:31:45.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:44 smithi097 ceph-mon[39416]: osdmap e228: 8 total, 8 up, 8 in 2024-03-21T13:31:45.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:44 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3253170645' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi097-79636-46"}]: dispatch 2024-03-21T13:31:45.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:44 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3525649546' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi097-79618-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi097-79618-37"}]: dispatch 2024-03-21T13:31:45.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:44 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi097-79636-46"}]: dispatch 2024-03-21T13:31:45.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:44 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:31:45.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:44 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi097-79636-46"}]': finished 2024-03-21T13:31:45.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:44 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3525649546' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi097-79618-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:31:45.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:44 smithi097 ceph-mon[30040]: osdmap e228: 8 total, 8 up, 8 in 2024-03-21T13:31:45.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:44 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3253170645' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi097-79636-46"}]: dispatch 2024-03-21T13:31:45.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:44 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3525649546' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi097-79618-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi097-79618-37"}]: dispatch 2024-03-21T13:31:45.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:44 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi097-79636-46"}]: dispatch 2024-03-21T13:31:45.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:44 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:31:45.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:45 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi097-79636-46"}]': finished 2024-03-21T13:31:45.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:45 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2632894404' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi097-79636-47"}]: dispatch 2024-03-21T13:31:45.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:45 smithi144 ceph-mon[33633]: pgmap v308: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 565 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:31:45.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:45 smithi144 ceph-mon[33633]: osdmap e229: 8 total, 8 up, 8 in 2024-03-21T13:31:45.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:45 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:45.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:45 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi097-79636-47"}]: dispatch 2024-03-21T13:31:45.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:45 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2632894404' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi097-79636-47"}]: dispatch 2024-03-21T13:31:45.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:45 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:45.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:45 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi097-79636-47"}]: dispatch 2024-03-21T13:31:45.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:45 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2632894404' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi097-79636-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:45.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:45 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi097-79636-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:45.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:45 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3525649546' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi097-79618-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi097-79618-37"}]': finished 2024-03-21T13:31:45.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:45 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:45.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:45 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi097-79636-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:31:45.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:45 smithi144 ceph-mon[33633]: osdmap e230: 8 total, 8 up, 8 in 2024-03-21T13:31:45.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:45 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2632894404' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi097-79636-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi097-79636-47"}]: dispatch 2024-03-21T13:31:45.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:45 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi097-79636-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi097-79636-47"}]: dispatch 2024-03-21T13:31:46.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:45 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi097-79636-46"}]': finished 2024-03-21T13:31:46.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:45 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2632894404' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi097-79636-47"}]: dispatch 2024-03-21T13:31:46.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:45 smithi097 ceph-mon[39416]: pgmap v308: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 565 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:31:46.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:45 smithi097 ceph-mon[39416]: osdmap e229: 8 total, 8 up, 8 in 2024-03-21T13:31:46.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:45 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:46.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:45 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi097-79636-47"}]: dispatch 2024-03-21T13:31:46.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:45 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2632894404' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi097-79636-47"}]: dispatch 2024-03-21T13:31:46.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:45 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:46.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:45 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi097-79636-47"}]: dispatch 2024-03-21T13:31:46.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:45 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2632894404' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi097-79636-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:46.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:45 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi097-79636-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:46.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:45 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3525649546' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi097-79618-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi097-79618-37"}]': finished 2024-03-21T13:31:46.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:45 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:46.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:45 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi097-79636-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:31:46.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:45 smithi097 ceph-mon[39416]: osdmap e230: 8 total, 8 up, 8 in 2024-03-21T13:31:46.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:45 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2632894404' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi097-79636-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi097-79636-47"}]: dispatch 2024-03-21T13:31:46.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:45 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi097-79636-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi097-79636-47"}]: dispatch 2024-03-21T13:31:46.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:45 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi097-79636-46"}]': finished 2024-03-21T13:31:46.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:45 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2632894404' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi097-79636-47"}]: dispatch 2024-03-21T13:31:46.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:45 smithi097 ceph-mon[30040]: pgmap v308: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 565 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:31:46.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:45 smithi097 ceph-mon[30040]: osdmap e229: 8 total, 8 up, 8 in 2024-03-21T13:31:46.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:45 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:46.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:45 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi097-79636-47"}]: dispatch 2024-03-21T13:31:46.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:45 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2632894404' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi097-79636-47"}]: dispatch 2024-03-21T13:31:46.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:45 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:46.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:45 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi097-79636-47"}]: dispatch 2024-03-21T13:31:46.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:45 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2632894404' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi097-79636-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:46.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:45 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi097-79636-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:46.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:45 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3525649546' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi097-79618-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi097-79618-37"}]': finished 2024-03-21T13:31:46.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:45 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:46.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:45 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi097-79636-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:31:46.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:45 smithi097 ceph-mon[30040]: osdmap e230: 8 total, 8 up, 8 in 2024-03-21T13:31:46.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:45 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2632894404' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi097-79636-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi097-79636-47"}]: dispatch 2024-03-21T13:31:46.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:45 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi097-79636-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi097-79636-47"}]: dispatch 2024-03-21T13:31:46.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:46 smithi144 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:31:46.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:46 smithi144 ceph-mon[33633]: osdmap e231: 8 total, 8 up, 8 in 2024-03-21T13:31:47.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:46 smithi097 ceph-mon[39416]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:31:47.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:46 smithi097 ceph-mon[39416]: osdmap e231: 8 total, 8 up, 8 in 2024-03-21T13:31:47.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:46 smithi097 ceph-mon[30040]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:31:47.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:46 smithi097 ceph-mon[30040]: osdmap e231: 8 total, 8 up, 8 in 2024-03-21T13:31:47.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:47 smithi144 ceph-mon[33633]: pgmap v311: 332 pgs: 40 unknown, 292 active+clean; 584 KiB data, 565 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:31:47.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:47 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi097-79636-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi097-79636-47"}]': finished 2024-03-21T13:31:47.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:47 smithi144 ceph-mon[33633]: osdmap e232: 8 total, 8 up, 8 in 2024-03-21T13:31:47.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:47 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3525649546' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi097-79618-37"}]: dispatch 2024-03-21T13:31:47.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:47 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:31:47.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:47 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:31:48.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:47 smithi097 ceph-mon[30040]: pgmap v311: 332 pgs: 40 unknown, 292 active+clean; 584 KiB data, 565 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:31:48.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:47 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi097-79636-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi097-79636-47"}]': finished 2024-03-21T13:31:48.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:47 smithi097 ceph-mon[30040]: osdmap e232: 8 total, 8 up, 8 in 2024-03-21T13:31:48.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:47 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3525649546' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi097-79618-37"}]: dispatch 2024-03-21T13:31:48.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:47 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:31:48.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:47 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:31:48.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:47 smithi097 ceph-mon[39416]: pgmap v311: 332 pgs: 40 unknown, 292 active+clean; 584 KiB data, 565 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:31:48.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:47 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi097-79636-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi097-79636-47"}]': finished 2024-03-21T13:31:48.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:47 smithi097 ceph-mon[39416]: osdmap e232: 8 total, 8 up, 8 in 2024-03-21T13:31:48.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:47 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3525649546' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi097-79618-37"}]: dispatch 2024-03-21T13:31:48.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:47 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:31:48.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:47 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:31:49.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:49 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3525649546' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi097-79618-37"}]': finished 2024-03-21T13:31:49.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:49 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-29", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:31:49.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:49 smithi097 ceph-mon[30040]: osdmap e233: 8 total, 8 up, 8 in 2024-03-21T13:31:49.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:49 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3525649546' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi097-79618-37"}]: dispatch 2024-03-21T13:31:49.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:49 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-29"}]: dispatch 2024-03-21T13:31:49.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:49 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-29"}]: dispatch 2024-03-21T13:31:49.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:49 smithi097 ceph-mon[30040]: pgmap v314: 332 pgs: 3 creating+activating, 329 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.0 MiB/s wr, 1 op/s 2024-03-21T13:31:49.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:49 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3525649546' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi097-79618-37"}]': finished 2024-03-21T13:31:49.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:49 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-29", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:31:49.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:49 smithi097 ceph-mon[39416]: osdmap e233: 8 total, 8 up, 8 in 2024-03-21T13:31:49.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:49 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3525649546' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi097-79618-37"}]: dispatch 2024-03-21T13:31:49.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:49 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-29"}]: dispatch 2024-03-21T13:31:49.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:49 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-29"}]: dispatch 2024-03-21T13:31:49.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:49 smithi097 ceph-mon[39416]: pgmap v314: 332 pgs: 3 creating+activating, 329 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.0 MiB/s wr, 1 op/s 2024-03-21T13:31:49.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:49 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3525649546' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi097-79618-37"}]': finished 2024-03-21T13:31:49.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:49 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-29", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:31:49.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:49 smithi144 ceph-mon[33633]: osdmap e233: 8 total, 8 up, 8 in 2024-03-21T13:31:49.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:49 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3525649546' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi097-79618-37"}]: dispatch 2024-03-21T13:31:49.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:49 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-29"}]: dispatch 2024-03-21T13:31:49.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:49 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-29"}]: dispatch 2024-03-21T13:31:49.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:49 smithi144 ceph-mon[33633]: pgmap v314: 332 pgs: 3 creating+activating, 329 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.0 MiB/s wr, 1 op/s 2024-03-21T13:31:50.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:50 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3525649546' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi097-79618-37"}]': finished 2024-03-21T13:31:50.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:50 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-29"}]': finished 2024-03-21T13:31:50.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:50 smithi097 ceph-mon[39416]: osdmap e234: 8 total, 8 up, 8 in 2024-03-21T13:31:50.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:50 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-29", "mode": "writeback"}]: dispatch 2024-03-21T13:31:50.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:50 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-29", "mode": "writeback"}]: dispatch 2024-03-21T13:31:50.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:50 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2632894404' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi097-79636-47"}]: dispatch 2024-03-21T13:31:50.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:50 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi097-79636-47"}]: dispatch 2024-03-21T13:31:50.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:50 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/991300143' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi097-79618-38"}]: dispatch 2024-03-21T13:31:50.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:50 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/991300143' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi097-79618-38"}]: dispatch 2024-03-21T13:31:50.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:50 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/991300143' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi097-79618-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:50.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:50 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3525649546' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi097-79618-37"}]': finished 2024-03-21T13:31:50.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:50 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-29"}]': finished 2024-03-21T13:31:50.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:50 smithi097 ceph-mon[30040]: osdmap e234: 8 total, 8 up, 8 in 2024-03-21T13:31:50.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:50 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-29", "mode": "writeback"}]: dispatch 2024-03-21T13:31:50.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:50 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-29", "mode": "writeback"}]: dispatch 2024-03-21T13:31:50.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:50 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2632894404' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi097-79636-47"}]: dispatch 2024-03-21T13:31:50.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:50 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi097-79636-47"}]: dispatch 2024-03-21T13:31:50.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:50 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/991300143' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi097-79618-38"}]: dispatch 2024-03-21T13:31:50.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:50 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/991300143' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi097-79618-38"}]: dispatch 2024-03-21T13:31:50.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:50 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/991300143' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi097-79618-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:50.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:50 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3525649546' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi097-79618-37"}]': finished 2024-03-21T13:31:50.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:50 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-29"}]': finished 2024-03-21T13:31:50.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:50 smithi144 ceph-mon[33633]: osdmap e234: 8 total, 8 up, 8 in 2024-03-21T13:31:50.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:50 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-29", "mode": "writeback"}]: dispatch 2024-03-21T13:31:50.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:50 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-29", "mode": "writeback"}]: dispatch 2024-03-21T13:31:50.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:50 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2632894404' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi097-79636-47"}]: dispatch 2024-03-21T13:31:50.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:50 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi097-79636-47"}]: dispatch 2024-03-21T13:31:50.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:50 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/991300143' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi097-79618-38"}]: dispatch 2024-03-21T13:31:50.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:50 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/991300143' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi097-79618-38"}]: dispatch 2024-03-21T13:31:50.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:50 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/991300143' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi097-79618-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:51.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:51 smithi097 ceph-mon[30040]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:31:51.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:51 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-29", "mode": "writeback"}]': finished 2024-03-21T13:31:51.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:51 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi097-79636-47"}]': finished 2024-03-21T13:31:51.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:51 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/991300143' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi097-79618-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:31:51.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:51 smithi097 ceph-mon[30040]: osdmap e235: 8 total, 8 up, 8 in 2024-03-21T13:31:51.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:51 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/991300143' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi097-79618-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi097-79618-38"}]: dispatch 2024-03-21T13:31:51.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:51 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2632894404' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi097-79636-47"}]: dispatch 2024-03-21T13:31:51.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:51 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi097-79636-47"}]: dispatch 2024-03-21T13:31:51.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:51 smithi097 ceph-mon[30040]: pgmap v317: 324 pgs: 324 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-03-21T13:31:51.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:51 smithi097 ceph-mon[39416]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:31:51.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:51 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-29", "mode": "writeback"}]': finished 2024-03-21T13:31:51.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:51 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi097-79636-47"}]': finished 2024-03-21T13:31:51.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:51 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/991300143' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi097-79618-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:31:51.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:51 smithi097 ceph-mon[39416]: osdmap e235: 8 total, 8 up, 8 in 2024-03-21T13:31:51.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:51 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/991300143' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi097-79618-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi097-79618-38"}]: dispatch 2024-03-21T13:31:51.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:51 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2632894404' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi097-79636-47"}]: dispatch 2024-03-21T13:31:51.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:51 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi097-79636-47"}]: dispatch 2024-03-21T13:31:51.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:51 smithi097 ceph-mon[39416]: pgmap v317: 324 pgs: 324 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-03-21T13:31:51.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:51 smithi144 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:31:51.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:51 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-29", "mode": "writeback"}]': finished 2024-03-21T13:31:51.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:51 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi097-79636-47"}]': finished 2024-03-21T13:31:51.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:51 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/991300143' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi097-79618-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:31:51.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:51 smithi144 ceph-mon[33633]: osdmap e235: 8 total, 8 up, 8 in 2024-03-21T13:31:51.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:51 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/991300143' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi097-79618-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi097-79618-38"}]: dispatch 2024-03-21T13:31:51.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:51 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2632894404' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi097-79636-47"}]: dispatch 2024-03-21T13:31:51.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:51 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi097-79636-47"}]: dispatch 2024-03-21T13:31:51.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:51 smithi144 ceph-mon[33633]: pgmap v317: 324 pgs: 324 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-03-21T13:31:52.262 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:31:51 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:31:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:31:52.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:52 smithi097 ceph-mon[39416]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:31:52.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:52 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi097-79636-47"}]': finished 2024-03-21T13:31:52.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:52 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2039473853' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi097-79636-48"}]: dispatch 2024-03-21T13:31:52.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:52 smithi097 ceph-mon[39416]: osdmap e236: 8 total, 8 up, 8 in 2024-03-21T13:31:52.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:52 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi097-79636-48"}]: dispatch 2024-03-21T13:31:52.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:52 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2039473853' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi097-79636-48"}]: dispatch 2024-03-21T13:31:52.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:52 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi097-79636-48"}]: dispatch 2024-03-21T13:31:52.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:52 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2039473853' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi097-79636-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:52.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:52 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi097-79636-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:52.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:52 smithi097 ceph-mon[30040]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:31:52.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:52 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi097-79636-47"}]': finished 2024-03-21T13:31:52.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:52 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2039473853' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi097-79636-48"}]: dispatch 2024-03-21T13:31:52.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:52 smithi097 ceph-mon[30040]: osdmap e236: 8 total, 8 up, 8 in 2024-03-21T13:31:52.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:52 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi097-79636-48"}]: dispatch 2024-03-21T13:31:52.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:52 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2039473853' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi097-79636-48"}]: dispatch 2024-03-21T13:31:52.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:52 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi097-79636-48"}]: dispatch 2024-03-21T13:31:52.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:52 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2039473853' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi097-79636-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:52.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:52 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi097-79636-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:52.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:52 smithi144 ceph-mon[33633]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:31:52.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:52 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi097-79636-47"}]': finished 2024-03-21T13:31:52.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:52 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2039473853' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi097-79636-48"}]: dispatch 2024-03-21T13:31:52.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:52 smithi144 ceph-mon[33633]: osdmap e236: 8 total, 8 up, 8 in 2024-03-21T13:31:52.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:52 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi097-79636-48"}]: dispatch 2024-03-21T13:31:52.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:52 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2039473853' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi097-79636-48"}]: dispatch 2024-03-21T13:31:52.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:52 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi097-79636-48"}]: dispatch 2024-03-21T13:31:52.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:52 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2039473853' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi097-79636-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:52.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:52 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi097-79636-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:53.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:53 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/991300143' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi097-79618-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi097-79618-38"}]': finished 2024-03-21T13:31:53.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:53 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi097-79636-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:31:53.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:53 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2039473853' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi097-79636-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi097-79636-48"}]: dispatch 2024-03-21T13:31:53.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:53 smithi097 ceph-mon[39416]: osdmap e237: 8 total, 8 up, 8 in 2024-03-21T13:31:53.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:53 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi097-79636-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi097-79636-48"}]: dispatch 2024-03-21T13:31:53.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:53 smithi097 ceph-mon[39416]: pgmap v320: 332 pgs: 8 unknown, 324 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:31:53.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:53 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/991300143' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi097-79618-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi097-79618-38"}]': finished 2024-03-21T13:31:53.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:53 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi097-79636-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:31:53.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:53 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2039473853' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi097-79636-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi097-79636-48"}]: dispatch 2024-03-21T13:31:53.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:53 smithi097 ceph-mon[30040]: osdmap e237: 8 total, 8 up, 8 in 2024-03-21T13:31:53.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:53 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi097-79636-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi097-79636-48"}]: dispatch 2024-03-21T13:31:53.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:53 smithi097 ceph-mon[30040]: pgmap v320: 332 pgs: 8 unknown, 324 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:31:53.569 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:53 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/991300143' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi097-79618-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi097-79618-38"}]': finished 2024-03-21T13:31:53.569 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:53 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi097-79636-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:31:53.569 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:53 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2039473853' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi097-79636-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi097-79636-48"}]: dispatch 2024-03-21T13:31:53.570 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:53 smithi144 ceph-mon[33633]: osdmap e237: 8 total, 8 up, 8 in 2024-03-21T13:31:53.570 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:53 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi097-79636-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi097-79636-48"}]: dispatch 2024-03-21T13:31:53.570 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:53 smithi144 ceph-mon[33633]: pgmap v320: 332 pgs: 8 unknown, 324 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:31:53.871 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:31:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:31:54.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:54 smithi097 ceph-mon[30040]: osdmap e238: 8 total, 8 up, 8 in 2024-03-21T13:31:54.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:54 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:31:54.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:54 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:31:54.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:54 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:31:54.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:54 smithi097 ceph-mon[39416]: osdmap e238: 8 total, 8 up, 8 in 2024-03-21T13:31:54.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:54 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:31:54.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:54 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:31:54.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:54 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:31:54.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:54 smithi144 ceph-mon[33633]: osdmap e238: 8 total, 8 up, 8 in 2024-03-21T13:31:54.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:54 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:31:54.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:54 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:31:54.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:54 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:31:55.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:55 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi097-79636-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi097-79636-48"}]': finished 2024-03-21T13:31:55.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:55 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:31:55.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:55 smithi144 ceph-mon[33633]: osdmap e239: 8 total, 8 up, 8 in 2024-03-21T13:31:55.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:55 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/991300143' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi097-79618-38"}]: dispatch 2024-03-21T13:31:55.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:55 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-29"}]: dispatch 2024-03-21T13:31:55.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:55 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-29"}]: dispatch 2024-03-21T13:31:55.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:55 smithi144 ceph-mon[33633]: pgmap v323: 332 pgs: 8 unknown, 324 active+clean; 8.6 MiB data, 598 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 0 B/s wr, 26 op/s 2024-03-21T13:31:55.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:55 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:31:55.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:55 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi097-79636-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi097-79636-48"}]': finished 2024-03-21T13:31:55.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:55 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:31:55.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:55 smithi097 ceph-mon[30040]: osdmap e239: 8 total, 8 up, 8 in 2024-03-21T13:31:55.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:55 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/991300143' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi097-79618-38"}]: dispatch 2024-03-21T13:31:55.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:55 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-29"}]: dispatch 2024-03-21T13:31:55.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:55 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-29"}]: dispatch 2024-03-21T13:31:55.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:55 smithi097 ceph-mon[30040]: pgmap v323: 332 pgs: 8 unknown, 324 active+clean; 8.6 MiB data, 598 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 0 B/s wr, 26 op/s 2024-03-21T13:31:55.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:55 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:31:55.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:55 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi097-79636-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi097-79636-48"}]': finished 2024-03-21T13:31:55.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:55 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:31:55.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:55 smithi097 ceph-mon[39416]: osdmap e239: 8 total, 8 up, 8 in 2024-03-21T13:31:55.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:55 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/991300143' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi097-79618-38"}]: dispatch 2024-03-21T13:31:55.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:55 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-29"}]: dispatch 2024-03-21T13:31:55.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:55 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-29"}]: dispatch 2024-03-21T13:31:55.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:55 smithi097 ceph-mon[39416]: pgmap v323: 332 pgs: 8 unknown, 324 active+clean; 8.6 MiB data, 598 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 0 B/s wr, 26 op/s 2024-03-21T13:31:55.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:55 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:31:56.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:56 smithi144 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:31:56.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:56 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/991300143' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi097-79618-38"}]': finished 2024-03-21T13:31:56.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:56 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-29"}]': finished 2024-03-21T13:31:56.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:56 smithi144 ceph-mon[33633]: osdmap e240: 8 total, 8 up, 8 in 2024-03-21T13:31:56.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:56 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/991300143' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi097-79618-38"}]: dispatch 2024-03-21T13:31:56.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:56 smithi097 ceph-mon[30040]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:31:56.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:56 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/991300143' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi097-79618-38"}]': finished 2024-03-21T13:31:56.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:56 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-29"}]': finished 2024-03-21T13:31:56.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:56 smithi097 ceph-mon[30040]: osdmap e240: 8 total, 8 up, 8 in 2024-03-21T13:31:56.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:56 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/991300143' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi097-79618-38"}]: dispatch 2024-03-21T13:31:56.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:56 smithi097 ceph-mon[39416]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:31:56.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:56 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/991300143' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi097-79618-38"}]': finished 2024-03-21T13:31:56.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:56 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-29"}]': finished 2024-03-21T13:31:56.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:56 smithi097 ceph-mon[39416]: osdmap e240: 8 total, 8 up, 8 in 2024-03-21T13:31:56.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:56 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/991300143' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi097-79618-38"}]: dispatch 2024-03-21T13:31:57.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:57 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/991300143' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi097-79618-38"}]': finished 2024-03-21T13:31:57.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:57 smithi144 ceph-mon[33633]: osdmap e241: 8 total, 8 up, 8 in 2024-03-21T13:31:57.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:57 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2039473853' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi097-79636-48"}]: dispatch 2024-03-21T13:31:57.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:57 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3549298695' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi097-79618-39"}]: dispatch 2024-03-21T13:31:57.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:57 smithi144 ceph-mon[33633]: pgmap v326: 292 pgs: 292 active+clean; 4.6 MiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:31:57.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:57 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi097-79636-48"}]: dispatch 2024-03-21T13:31:57.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:57 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi097-79618-39"}]: dispatch 2024-03-21T13:31:57.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:57 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3549298695' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi097-79618-39"}]: dispatch 2024-03-21T13:31:57.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:57 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi097-79618-39"}]: dispatch 2024-03-21T13:31:57.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:57 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3549298695' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi097-79618-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:57.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:57 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi097-79618-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:57.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:57 smithi144 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:31:57.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:57 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi097-79636-48"}]': finished 2024-03-21T13:31:57.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:57 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi097-79618-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:31:57.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:57 smithi144 ceph-mon[33633]: osdmap e242: 8 total, 8 up, 8 in 2024-03-21T13:31:57.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:57 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2039473853' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi097-79636-48"}]: dispatch 2024-03-21T13:31:57.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:57 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3549298695' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi097-79618-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi097-79618-39"}]: dispatch 2024-03-21T13:31:57.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:57 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:57.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:57 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi097-79636-48"}]: dispatch 2024-03-21T13:31:57.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:57 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi097-79618-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi097-79618-39"}]: dispatch 2024-03-21T13:31:57.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:57 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:57.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:57 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/991300143' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi097-79618-38"}]': finished 2024-03-21T13:31:57.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:57 smithi097 ceph-mon[30040]: osdmap e241: 8 total, 8 up, 8 in 2024-03-21T13:31:57.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:57 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2039473853' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi097-79636-48"}]: dispatch 2024-03-21T13:31:57.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:57 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3549298695' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi097-79618-39"}]: dispatch 2024-03-21T13:31:57.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:57 smithi097 ceph-mon[30040]: pgmap v326: 292 pgs: 292 active+clean; 4.6 MiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:31:57.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:57 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi097-79636-48"}]: dispatch 2024-03-21T13:31:57.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:57 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi097-79618-39"}]: dispatch 2024-03-21T13:31:57.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:57 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3549298695' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi097-79618-39"}]: dispatch 2024-03-21T13:31:57.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:57 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi097-79618-39"}]: dispatch 2024-03-21T13:31:57.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:57 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3549298695' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi097-79618-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:57.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:57 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi097-79618-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:57.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:57 smithi097 ceph-mon[30040]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:31:57.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:57 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi097-79636-48"}]': finished 2024-03-21T13:31:57.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:57 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi097-79618-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:31:57.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:57 smithi097 ceph-mon[30040]: osdmap e242: 8 total, 8 up, 8 in 2024-03-21T13:31:57.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:57 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2039473853' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi097-79636-48"}]: dispatch 2024-03-21T13:31:57.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:57 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3549298695' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi097-79618-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi097-79618-39"}]: dispatch 2024-03-21T13:31:57.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:57 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:57.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:57 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi097-79636-48"}]: dispatch 2024-03-21T13:31:57.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:57 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi097-79618-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi097-79618-39"}]: dispatch 2024-03-21T13:31:57.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:57 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:57.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:57 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/991300143' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi097-79618-38"}]': finished 2024-03-21T13:31:57.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:57 smithi097 ceph-mon[39416]: osdmap e241: 8 total, 8 up, 8 in 2024-03-21T13:31:57.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:57 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2039473853' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi097-79636-48"}]: dispatch 2024-03-21T13:31:57.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:57 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3549298695' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi097-79618-39"}]: dispatch 2024-03-21T13:31:57.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:57 smithi097 ceph-mon[39416]: pgmap v326: 292 pgs: 292 active+clean; 4.6 MiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:31:57.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:57 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi097-79636-48"}]: dispatch 2024-03-21T13:31:57.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:57 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi097-79618-39"}]: dispatch 2024-03-21T13:31:57.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:57 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3549298695' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi097-79618-39"}]: dispatch 2024-03-21T13:31:57.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:57 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi097-79618-39"}]: dispatch 2024-03-21T13:31:57.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:57 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3549298695' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi097-79618-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:57.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:57 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi097-79618-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:57.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:57 smithi097 ceph-mon[39416]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:31:57.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:57 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi097-79636-48"}]': finished 2024-03-21T13:31:57.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:57 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi097-79618-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:31:57.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:57 smithi097 ceph-mon[39416]: osdmap e242: 8 total, 8 up, 8 in 2024-03-21T13:31:57.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:57 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2039473853' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi097-79636-48"}]: dispatch 2024-03-21T13:31:57.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:57 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3549298695' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi097-79618-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi097-79618-39"}]: dispatch 2024-03-21T13:31:57.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:57 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:57.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:57 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi097-79636-48"}]: dispatch 2024-03-21T13:31:57.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:57 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi097-79618-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi097-79618-39"}]: dispatch 2024-03-21T13:31:57.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:57 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:31:59.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:59 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi097-79636-48"}]': finished 2024-03-21T13:31:59.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:59 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:59.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:59 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:31:59.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:59 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2845129661' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi097-79636-49"}]: dispatch 2024-03-21T13:31:59.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:59 smithi097 ceph-mon[39416]: pgmap v329: 324 pgs: 9 creating+activating, 22 creating+peering, 293 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:31:59.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:59 smithi097 ceph-mon[39416]: osdmap e243: 8 total, 8 up, 8 in 2024-03-21T13:31:59.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:59 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:31:59.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:59 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi097-79636-49"}]: dispatch 2024-03-21T13:31:59.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:59 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2845129661' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi097-79636-49"}]: dispatch 2024-03-21T13:31:59.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:59 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi097-79636-49"}]: dispatch 2024-03-21T13:31:59.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:59 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2845129661' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi097-79636-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:59.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:31:59 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi097-79636-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:59.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:59 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi097-79636-48"}]': finished 2024-03-21T13:31:59.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:59 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:59.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:59 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:31:59.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:59 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2845129661' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi097-79636-49"}]: dispatch 2024-03-21T13:31:59.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:59 smithi097 ceph-mon[30040]: pgmap v329: 324 pgs: 9 creating+activating, 22 creating+peering, 293 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:31:59.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:59 smithi097 ceph-mon[30040]: osdmap e243: 8 total, 8 up, 8 in 2024-03-21T13:31:59.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:59 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:31:59.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:59 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi097-79636-49"}]: dispatch 2024-03-21T13:31:59.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:59 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2845129661' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi097-79636-49"}]: dispatch 2024-03-21T13:31:59.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:59 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi097-79636-49"}]: dispatch 2024-03-21T13:31:59.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:59 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2845129661' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi097-79636-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:59.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:31:59 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi097-79636-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:59.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:59 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi097-79636-48"}]': finished 2024-03-21T13:31:59.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:59 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:31:59.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:59 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:31:59.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:59 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2845129661' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi097-79636-49"}]: dispatch 2024-03-21T13:31:59.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:59 smithi144 ceph-mon[33633]: pgmap v329: 324 pgs: 9 creating+activating, 22 creating+peering, 293 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:31:59.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:59 smithi144 ceph-mon[33633]: osdmap e243: 8 total, 8 up, 8 in 2024-03-21T13:31:59.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:59 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:31:59.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:59 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi097-79636-49"}]: dispatch 2024-03-21T13:31:59.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:59 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2845129661' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi097-79636-49"}]: dispatch 2024-03-21T13:31:59.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:59 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi097-79636-49"}]: dispatch 2024-03-21T13:31:59.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:59 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2845129661' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi097-79636-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:31:59.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:31:59 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi097-79636-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:00.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:00 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi097-79618-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi097-79618-39"}]': finished 2024-03-21T13:32:00.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:00 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-31", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:32:00.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:00 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi097-79636-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:32:00.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:00 smithi097 ceph-mon[30040]: osdmap e244: 8 total, 8 up, 8 in 2024-03-21T13:32:00.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:00 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2845129661' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi097-79636-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi097-79636-49"}]: dispatch 2024-03-21T13:32:00.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:00 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-31"}]: dispatch 2024-03-21T13:32:00.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:00 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi097-79636-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi097-79636-49"}]: dispatch 2024-03-21T13:32:00.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:00 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-31"}]: dispatch 2024-03-21T13:32:00.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:00 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi097-79618-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi097-79618-39"}]': finished 2024-03-21T13:32:00.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:00 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-31", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:32:00.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:00 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi097-79636-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:32:00.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:00 smithi097 ceph-mon[39416]: osdmap e244: 8 total, 8 up, 8 in 2024-03-21T13:32:00.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:00 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2845129661' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi097-79636-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi097-79636-49"}]: dispatch 2024-03-21T13:32:00.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:00 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-31"}]: dispatch 2024-03-21T13:32:00.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:00 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi097-79636-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi097-79636-49"}]: dispatch 2024-03-21T13:32:00.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:00 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-31"}]: dispatch 2024-03-21T13:32:00.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:00 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi097-79618-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi097-79618-39"}]': finished 2024-03-21T13:32:00.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:00 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-31", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:32:00.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:00 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi097-79636-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:32:00.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:00 smithi144 ceph-mon[33633]: osdmap e244: 8 total, 8 up, 8 in 2024-03-21T13:32:00.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:00 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2845129661' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi097-79636-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi097-79636-49"}]: dispatch 2024-03-21T13:32:00.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:00 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-31"}]: dispatch 2024-03-21T13:32:00.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:00 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi097-79636-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi097-79636-49"}]: dispatch 2024-03-21T13:32:00.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:00 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-31"}]: dispatch 2024-03-21T13:32:01.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:01 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-31"}]': finished 2024-03-21T13:32:01.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:01 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-31", "mode": "writeback"}]: dispatch 2024-03-21T13:32:01.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:01 smithi097 ceph-mon[39416]: pgmap v332: 332 pgs: 8 unknown, 9 creating+activating, 22 creating+peering, 293 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-03-21T13:32:01.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:01 smithi097 ceph-mon[39416]: osdmap e245: 8 total, 8 up, 8 in 2024-03-21T13:32:01.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:01 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-31", "mode": "writeback"}]: dispatch 2024-03-21T13:32:01.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:01 smithi097 ceph-mon[39416]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:32:01.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:01 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi097-79636-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi097-79636-49"}]': finished 2024-03-21T13:32:01.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:01 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-31", "mode": "writeback"}]': finished 2024-03-21T13:32:01.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:01 smithi097 ceph-mon[39416]: osdmap e246: 8 total, 8 up, 8 in 2024-03-21T13:32:01.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:01 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3549298695' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi097-79618-39"}]: dispatch 2024-03-21T13:32:01.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:01 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi097-79618-39"}]: dispatch 2024-03-21T13:32:01.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:01 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:32:01.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:01 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:32:01.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:01 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-31"}]': finished 2024-03-21T13:32:01.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:01 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-31", "mode": "writeback"}]: dispatch 2024-03-21T13:32:01.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:01 smithi097 ceph-mon[30040]: pgmap v332: 332 pgs: 8 unknown, 9 creating+activating, 22 creating+peering, 293 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-03-21T13:32:01.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:01 smithi097 ceph-mon[30040]: osdmap e245: 8 total, 8 up, 8 in 2024-03-21T13:32:01.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:01 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-31", "mode": "writeback"}]: dispatch 2024-03-21T13:32:01.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:01 smithi097 ceph-mon[30040]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:32:01.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:01 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi097-79636-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi097-79636-49"}]': finished 2024-03-21T13:32:01.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:01 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-31", "mode": "writeback"}]': finished 2024-03-21T13:32:01.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:01 smithi097 ceph-mon[30040]: osdmap e246: 8 total, 8 up, 8 in 2024-03-21T13:32:01.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:01 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3549298695' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi097-79618-39"}]: dispatch 2024-03-21T13:32:01.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:01 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi097-79618-39"}]: dispatch 2024-03-21T13:32:01.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:01 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:32:01.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:01 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:32:01.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:01 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-31"}]': finished 2024-03-21T13:32:01.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:01 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-31", "mode": "writeback"}]: dispatch 2024-03-21T13:32:01.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:01 smithi144 ceph-mon[33633]: pgmap v332: 332 pgs: 8 unknown, 9 creating+activating, 22 creating+peering, 293 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-03-21T13:32:01.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:01 smithi144 ceph-mon[33633]: osdmap e245: 8 total, 8 up, 8 in 2024-03-21T13:32:01.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:01 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-31", "mode": "writeback"}]: dispatch 2024-03-21T13:32:01.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:01 smithi144 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:32:01.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:01 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi097-79636-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi097-79636-49"}]': finished 2024-03-21T13:32:01.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:01 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-31", "mode": "writeback"}]': finished 2024-03-21T13:32:01.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:01 smithi144 ceph-mon[33633]: osdmap e246: 8 total, 8 up, 8 in 2024-03-21T13:32:01.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:01 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3549298695' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi097-79618-39"}]: dispatch 2024-03-21T13:32:01.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:01 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi097-79618-39"}]: dispatch 2024-03-21T13:32:01.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:01 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:32:01.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:01 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:32:02.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:32:01 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:32:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:32:03.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:03 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi097-79618-39"}]': finished 2024-03-21T13:32:03.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:03 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:32:03.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:03 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3549298695' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi097-79618-39"}]: dispatch 2024-03-21T13:32:03.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:03 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-31"}]: dispatch 2024-03-21T13:32:03.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:03 smithi097 ceph-mon[39416]: osdmap e247: 8 total, 8 up, 8 in 2024-03-21T13:32:03.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:03 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi097-79618-39"}]: dispatch 2024-03-21T13:32:03.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:03 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-31"}]: dispatch 2024-03-21T13:32:03.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:03 smithi097 ceph-mon[39416]: pgmap v335: 332 pgs: 8 unknown, 9 creating+activating, 22 creating+peering, 293 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:32:03.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:03 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi097-79618-39"}]': finished 2024-03-21T13:32:03.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:03 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:32:03.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:03 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3549298695' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi097-79618-39"}]: dispatch 2024-03-21T13:32:03.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:03 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-31"}]: dispatch 2024-03-21T13:32:03.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:03 smithi097 ceph-mon[30040]: osdmap e247: 8 total, 8 up, 8 in 2024-03-21T13:32:03.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:03 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi097-79618-39"}]: dispatch 2024-03-21T13:32:03.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:03 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-31"}]: dispatch 2024-03-21T13:32:03.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:03 smithi097 ceph-mon[30040]: pgmap v335: 332 pgs: 8 unknown, 9 creating+activating, 22 creating+peering, 293 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:32:03.573 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:03 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi097-79618-39"}]': finished 2024-03-21T13:32:03.573 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:03 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:32:03.573 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:03 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3549298695' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi097-79618-39"}]: dispatch 2024-03-21T13:32:03.573 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:03 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-31"}]: dispatch 2024-03-21T13:32:03.573 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:03 smithi144 ceph-mon[33633]: osdmap e247: 8 total, 8 up, 8 in 2024-03-21T13:32:03.574 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:03 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi097-79618-39"}]: dispatch 2024-03-21T13:32:03.574 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:03 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-31"}]: dispatch 2024-03-21T13:32:03.574 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:03 smithi144 ceph-mon[33633]: pgmap v335: 332 pgs: 8 unknown, 9 creating+activating, 22 creating+peering, 293 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:32:03.871 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:32:03 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:32:04.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:04 smithi097 ceph-mon[30040]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:32:04.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:04 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi097-79618-39"}]': finished 2024-03-21T13:32:04.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:04 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-31"}]': finished 2024-03-21T13:32:04.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:04 smithi097 ceph-mon[30040]: osdmap e248: 8 total, 8 up, 8 in 2024-03-21T13:32:04.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:04 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2845129661' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi097-79636-49"}]: dispatch 2024-03-21T13:32:04.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:04 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi097-79636-49"}]: dispatch 2024-03-21T13:32:04.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:04 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3512586087' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi097-79618-40"}]: dispatch 2024-03-21T13:32:04.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:04 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi097-79618-40"}]: dispatch 2024-03-21T13:32:04.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:04 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3512586087' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi097-79618-40"}]: dispatch 2024-03-21T13:32:04.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:04 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi097-79618-40"}]: dispatch 2024-03-21T13:32:04.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:04 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3512586087' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi097-79618-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:04.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:04 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi097-79618-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:04.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:04 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:32:04.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:04 smithi097 ceph-mon[39416]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:32:04.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:04 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi097-79618-39"}]': finished 2024-03-21T13:32:04.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:04 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-31"}]': finished 2024-03-21T13:32:04.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:04 smithi097 ceph-mon[39416]: osdmap e248: 8 total, 8 up, 8 in 2024-03-21T13:32:04.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:04 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2845129661' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi097-79636-49"}]: dispatch 2024-03-21T13:32:04.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:04 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi097-79636-49"}]: dispatch 2024-03-21T13:32:04.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:04 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3512586087' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi097-79618-40"}]: dispatch 2024-03-21T13:32:04.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:04 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi097-79618-40"}]: dispatch 2024-03-21T13:32:04.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:04 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3512586087' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi097-79618-40"}]: dispatch 2024-03-21T13:32:04.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:04 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi097-79618-40"}]: dispatch 2024-03-21T13:32:04.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:04 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3512586087' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi097-79618-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:04.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:04 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi097-79618-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:04.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:04 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:32:04.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:04 smithi144 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:32:04.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:04 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi097-79618-39"}]': finished 2024-03-21T13:32:04.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:04 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-31"}]': finished 2024-03-21T13:32:04.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:04 smithi144 ceph-mon[33633]: osdmap e248: 8 total, 8 up, 8 in 2024-03-21T13:32:04.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:04 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2845129661' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi097-79636-49"}]: dispatch 2024-03-21T13:32:04.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:04 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi097-79636-49"}]: dispatch 2024-03-21T13:32:04.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:04 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3512586087' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi097-79618-40"}]: dispatch 2024-03-21T13:32:04.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:04 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi097-79618-40"}]: dispatch 2024-03-21T13:32:04.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:04 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3512586087' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi097-79618-40"}]: dispatch 2024-03-21T13:32:04.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:04 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi097-79618-40"}]: dispatch 2024-03-21T13:32:04.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:04 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3512586087' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi097-79618-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:04.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:04 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi097-79618-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:04.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:04 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:32:05.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:05 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi097-79636-49"}]': finished 2024-03-21T13:32:05.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:05 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi097-79618-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:32:05.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:05 smithi097 ceph-mon[30040]: osdmap e249: 8 total, 8 up, 8 in 2024-03-21T13:32:05.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:05 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2845129661' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi097-79636-49"}]: dispatch 2024-03-21T13:32:05.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:05 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3512586087' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi097-79618-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi097-79618-40"}]: dispatch 2024-03-21T13:32:05.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:05 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi097-79636-49"}]: dispatch 2024-03-21T13:32:05.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:05 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi097-79618-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi097-79618-40"}]: dispatch 2024-03-21T13:32:05.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:05 smithi097 ceph-mon[30040]: pgmap v338: 292 pgs: 292 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:32:05.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:05 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi097-79636-49"}]': finished 2024-03-21T13:32:05.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:05 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi097-79618-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:32:05.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:05 smithi097 ceph-mon[39416]: osdmap e249: 8 total, 8 up, 8 in 2024-03-21T13:32:05.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:05 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2845129661' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi097-79636-49"}]: dispatch 2024-03-21T13:32:05.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:05 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3512586087' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi097-79618-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi097-79618-40"}]: dispatch 2024-03-21T13:32:05.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:05 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi097-79636-49"}]: dispatch 2024-03-21T13:32:05.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:05 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi097-79618-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi097-79618-40"}]: dispatch 2024-03-21T13:32:05.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:05 smithi097 ceph-mon[39416]: pgmap v338: 292 pgs: 292 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:32:05.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:05 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi097-79636-49"}]': finished 2024-03-21T13:32:05.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:05 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi097-79618-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:32:05.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:05 smithi144 ceph-mon[33633]: osdmap e249: 8 total, 8 up, 8 in 2024-03-21T13:32:05.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:05 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2845129661' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi097-79636-49"}]: dispatch 2024-03-21T13:32:05.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:05 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3512586087' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi097-79618-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi097-79618-40"}]: dispatch 2024-03-21T13:32:05.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:05 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi097-79636-49"}]: dispatch 2024-03-21T13:32:05.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:05 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi097-79618-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi097-79618-40"}]: dispatch 2024-03-21T13:32:05.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:05 smithi144 ceph-mon[33633]: pgmap v338: 292 pgs: 292 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:32:06.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:06 smithi097 ceph-mon[39416]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:32:06.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:06 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi097-79636-49"}]': finished 2024-03-21T13:32:06.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:06 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2965425103' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi097-79636-50"}]: dispatch 2024-03-21T13:32:06.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:06 smithi097 ceph-mon[39416]: osdmap e250: 8 total, 8 up, 8 in 2024-03-21T13:32:06.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:06 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi097-79636-50"}]: dispatch 2024-03-21T13:32:06.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:06 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:32:06.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:06 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:32:06.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:06 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2965425103' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi097-79636-50"}]: dispatch 2024-03-21T13:32:06.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:06 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi097-79636-50"}]: dispatch 2024-03-21T13:32:06.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:06 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2965425103' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi097-79636-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:06.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:06 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi097-79636-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:06.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:06 smithi097 ceph-mon[30040]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:32:06.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:06 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi097-79636-49"}]': finished 2024-03-21T13:32:06.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:06 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2965425103' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi097-79636-50"}]: dispatch 2024-03-21T13:32:06.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:06 smithi097 ceph-mon[30040]: osdmap e250: 8 total, 8 up, 8 in 2024-03-21T13:32:06.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:06 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi097-79636-50"}]: dispatch 2024-03-21T13:32:06.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:06 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:32:06.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:06 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:32:06.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:06 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2965425103' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi097-79636-50"}]: dispatch 2024-03-21T13:32:06.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:06 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi097-79636-50"}]: dispatch 2024-03-21T13:32:06.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:06 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2965425103' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi097-79636-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:06.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:06 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi097-79636-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:06.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:06 smithi144 ceph-mon[33633]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:32:06.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:06 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi097-79636-49"}]': finished 2024-03-21T13:32:06.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:06 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2965425103' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi097-79636-50"}]: dispatch 2024-03-21T13:32:06.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:06 smithi144 ceph-mon[33633]: osdmap e250: 8 total, 8 up, 8 in 2024-03-21T13:32:06.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:06 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi097-79636-50"}]: dispatch 2024-03-21T13:32:06.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:06 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:32:06.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:06 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:32:06.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:06 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2965425103' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi097-79636-50"}]: dispatch 2024-03-21T13:32:06.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:06 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi097-79636-50"}]: dispatch 2024-03-21T13:32:06.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:06 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2965425103' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi097-79636-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:06.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:06 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi097-79636-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:07.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:07 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi097-79618-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi097-79618-40"}]': finished 2024-03-21T13:32:07.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:07 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:32:07.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:07 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi097-79636-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:32:07.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:07 smithi097 ceph-mon[30040]: osdmap e251: 8 total, 8 up, 8 in 2024-03-21T13:32:07.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:07 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2965425103' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi097-79636-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi097-79636-50"}]: dispatch 2024-03-21T13:32:07.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:07 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:32:07.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:07 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi097-79636-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi097-79636-50"}]: dispatch 2024-03-21T13:32:07.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:07 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:32:07.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:07 smithi097 ceph-mon[30040]: pgmap v341: 332 pgs: 40 unknown, 292 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:32:07.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:07 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi097-79618-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi097-79618-40"}]': finished 2024-03-21T13:32:07.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:07 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:32:07.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:07 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi097-79636-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:32:07.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:07 smithi097 ceph-mon[39416]: osdmap e251: 8 total, 8 up, 8 in 2024-03-21T13:32:07.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:07 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2965425103' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi097-79636-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi097-79636-50"}]: dispatch 2024-03-21T13:32:07.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:07 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:32:07.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:07 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi097-79636-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi097-79636-50"}]: dispatch 2024-03-21T13:32:07.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:07 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:32:07.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:07 smithi097 ceph-mon[39416]: pgmap v341: 332 pgs: 40 unknown, 292 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:32:07.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:07 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi097-79618-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi097-79618-40"}]': finished 2024-03-21T13:32:07.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:07 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:32:07.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:07 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi097-79636-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:32:07.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:07 smithi144 ceph-mon[33633]: osdmap e251: 8 total, 8 up, 8 in 2024-03-21T13:32:07.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:07 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2965425103' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi097-79636-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi097-79636-50"}]: dispatch 2024-03-21T13:32:07.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:07 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:32:07.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:07 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi097-79636-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi097-79636-50"}]: dispatch 2024-03-21T13:32:07.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:07 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:32:07.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:07 smithi144 ceph-mon[33633]: pgmap v341: 332 pgs: 40 unknown, 292 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:32:08.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:08 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-33", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:32:08.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:08 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-33"}]: dispatch 2024-03-21T13:32:08.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:08 smithi097 ceph-mon[39416]: osdmap e252: 8 total, 8 up, 8 in 2024-03-21T13:32:08.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:08 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-33"}]: dispatch 2024-03-21T13:32:08.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:08 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-33", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:32:08.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:08 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-33"}]: dispatch 2024-03-21T13:32:08.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:08 smithi097 ceph-mon[30040]: osdmap e252: 8 total, 8 up, 8 in 2024-03-21T13:32:08.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:08 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-33"}]: dispatch 2024-03-21T13:32:08.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:08 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-33", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:32:08.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:08 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-33"}]: dispatch 2024-03-21T13:32:08.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:08 smithi144 ceph-mon[33633]: osdmap e252: 8 total, 8 up, 8 in 2024-03-21T13:32:08.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:08 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-33"}]: dispatch 2024-03-21T13:32:09.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:09 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi097-79636-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi097-79636-50"}]': finished 2024-03-21T13:32:09.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:09 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-33"}]': finished 2024-03-21T13:32:09.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:09 smithi144 ceph-mon[33633]: osdmap e253: 8 total, 8 up, 8 in 2024-03-21T13:32:09.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:09 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-33", "mode": "writeback"}]: dispatch 2024-03-21T13:32:09.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:09 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3512586087' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi097-79618-40"}]: dispatch 2024-03-21T13:32:09.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:09 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-33", "mode": "writeback"}]: dispatch 2024-03-21T13:32:09.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:09 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi097-79618-40"}]: dispatch 2024-03-21T13:32:09.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:09 smithi144 ceph-mon[33633]: pgmap v344: 332 pgs: 8 creating+peering, 324 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:32:09.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:09 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi097-79636-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi097-79636-50"}]': finished 2024-03-21T13:32:09.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:09 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-33"}]': finished 2024-03-21T13:32:09.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:09 smithi097 ceph-mon[30040]: osdmap e253: 8 total, 8 up, 8 in 2024-03-21T13:32:09.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:09 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-33", "mode": "writeback"}]: dispatch 2024-03-21T13:32:09.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:09 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3512586087' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi097-79618-40"}]: dispatch 2024-03-21T13:32:09.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:09 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-33", "mode": "writeback"}]: dispatch 2024-03-21T13:32:09.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:09 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi097-79618-40"}]: dispatch 2024-03-21T13:32:09.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:09 smithi097 ceph-mon[30040]: pgmap v344: 332 pgs: 8 creating+peering, 324 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:32:09.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:09 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi097-79636-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi097-79636-50"}]': finished 2024-03-21T13:32:09.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:09 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-33"}]': finished 2024-03-21T13:32:09.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:09 smithi097 ceph-mon[39416]: osdmap e253: 8 total, 8 up, 8 in 2024-03-21T13:32:09.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:09 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-33", "mode": "writeback"}]: dispatch 2024-03-21T13:32:09.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:09 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3512586087' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi097-79618-40"}]: dispatch 2024-03-21T13:32:09.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:09 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-33", "mode": "writeback"}]: dispatch 2024-03-21T13:32:09.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:09 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi097-79618-40"}]: dispatch 2024-03-21T13:32:09.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:09 smithi097 ceph-mon[39416]: pgmap v344: 332 pgs: 8 creating+peering, 324 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:32:10.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:10 smithi144 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:32:10.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:10 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-33", "mode": "writeback"}]': finished 2024-03-21T13:32:10.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:10 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi097-79618-40"}]': finished 2024-03-21T13:32:10.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:10 smithi144 ceph-mon[33633]: osdmap e254: 8 total, 8 up, 8 in 2024-03-21T13:32:10.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:10 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3512586087' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi097-79618-40"}]: dispatch 2024-03-21T13:32:10.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:10 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi097-79618-40"}]: dispatch 2024-03-21T13:32:10.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:10 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:32:10.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:10 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:32:10.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:10 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:32:10.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:10 smithi097 ceph-mon[39416]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:32:10.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:10 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-33", "mode": "writeback"}]': finished 2024-03-21T13:32:10.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:10 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi097-79618-40"}]': finished 2024-03-21T13:32:10.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:10 smithi097 ceph-mon[39416]: osdmap e254: 8 total, 8 up, 8 in 2024-03-21T13:32:10.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:10 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3512586087' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi097-79618-40"}]: dispatch 2024-03-21T13:32:10.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:10 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi097-79618-40"}]: dispatch 2024-03-21T13:32:10.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:10 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:32:10.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:10 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:32:10.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:10 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:32:10.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:10 smithi097 ceph-mon[30040]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:32:10.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:10 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-33", "mode": "writeback"}]': finished 2024-03-21T13:32:10.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:10 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi097-79618-40"}]': finished 2024-03-21T13:32:10.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:10 smithi097 ceph-mon[30040]: osdmap e254: 8 total, 8 up, 8 in 2024-03-21T13:32:10.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:10 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3512586087' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi097-79618-40"}]: dispatch 2024-03-21T13:32:10.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:10 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi097-79618-40"}]: dispatch 2024-03-21T13:32:10.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:10 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:32:10.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:10 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:32:10.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:10 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:32:11.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:11 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi097-79618-40"}]': finished 2024-03-21T13:32:11.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:11 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:32:11.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:11 smithi144 ceph-mon[33633]: osdmap e255: 8 total, 8 up, 8 in 2024-03-21T13:32:11.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:11 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-33"}]: dispatch 2024-03-21T13:32:11.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:11 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-33"}]: dispatch 2024-03-21T13:32:11.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:11 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2965425103' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi097-79636-50"}]: dispatch 2024-03-21T13:32:11.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:11 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi097-79636-50"}]: dispatch 2024-03-21T13:32:11.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:11 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4260986539' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi097-79618-41"}]: dispatch 2024-03-21T13:32:11.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:11 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi097-79618-41"}]: dispatch 2024-03-21T13:32:11.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:11 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4260986539' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi097-79618-41"}]: dispatch 2024-03-21T13:32:11.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:11 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi097-79618-41"}]: dispatch 2024-03-21T13:32:11.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:11 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4260986539' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi097-79618-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:11.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:11 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi097-79618-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:11.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:11 smithi144 ceph-mon[33633]: pgmap v347: 324 pgs: 324 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:32:11.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:11 smithi144 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:32:11.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:11 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-33"}]': finished 2024-03-21T13:32:11.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:11 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi097-79636-50"}]': finished 2024-03-21T13:32:11.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:11 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi097-79618-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:32:11.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:11 smithi144 ceph-mon[33633]: osdmap e256: 8 total, 8 up, 8 in 2024-03-21T13:32:11.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:11 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2965425103' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi097-79636-50"}]: dispatch 2024-03-21T13:32:11.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:11 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4260986539' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi097-79618-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi097-79618-41"}]: dispatch 2024-03-21T13:32:11.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:11 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi097-79636-50"}]: dispatch 2024-03-21T13:32:11.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:11 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi097-79618-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi097-79618-41"}]: dispatch 2024-03-21T13:32:11.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:11 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi097-79618-40"}]': finished 2024-03-21T13:32:11.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:11 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:32:11.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:11 smithi097 ceph-mon[39416]: osdmap e255: 8 total, 8 up, 8 in 2024-03-21T13:32:11.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:11 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-33"}]: dispatch 2024-03-21T13:32:11.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:11 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-33"}]: dispatch 2024-03-21T13:32:11.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:11 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2965425103' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi097-79636-50"}]: dispatch 2024-03-21T13:32:11.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:11 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi097-79636-50"}]: dispatch 2024-03-21T13:32:11.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:11 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4260986539' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi097-79618-41"}]: dispatch 2024-03-21T13:32:11.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:11 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi097-79618-41"}]: dispatch 2024-03-21T13:32:11.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:11 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4260986539' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi097-79618-41"}]: dispatch 2024-03-21T13:32:11.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:11 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi097-79618-41"}]: dispatch 2024-03-21T13:32:11.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:11 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4260986539' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi097-79618-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:11.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:11 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi097-79618-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:11.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:11 smithi097 ceph-mon[39416]: pgmap v347: 324 pgs: 324 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:32:11.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:11 smithi097 ceph-mon[39416]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:32:11.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:11 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-33"}]': finished 2024-03-21T13:32:11.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:11 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi097-79636-50"}]': finished 2024-03-21T13:32:11.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:11 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi097-79618-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:32:11.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:11 smithi097 ceph-mon[39416]: osdmap e256: 8 total, 8 up, 8 in 2024-03-21T13:32:11.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:11 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2965425103' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi097-79636-50"}]: dispatch 2024-03-21T13:32:11.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:11 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4260986539' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi097-79618-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi097-79618-41"}]: dispatch 2024-03-21T13:32:11.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:11 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi097-79636-50"}]: dispatch 2024-03-21T13:32:11.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:11 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi097-79618-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi097-79618-41"}]: dispatch 2024-03-21T13:32:11.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:11 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi097-79618-40"}]': finished 2024-03-21T13:32:11.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:11 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:32:11.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:11 smithi097 ceph-mon[30040]: osdmap e255: 8 total, 8 up, 8 in 2024-03-21T13:32:11.806 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:11 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-33"}]: dispatch 2024-03-21T13:32:11.806 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:11 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-33"}]: dispatch 2024-03-21T13:32:11.806 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:11 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2965425103' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi097-79636-50"}]: dispatch 2024-03-21T13:32:11.806 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:11 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi097-79636-50"}]: dispatch 2024-03-21T13:32:11.806 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:11 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4260986539' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi097-79618-41"}]: dispatch 2024-03-21T13:32:11.806 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:11 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi097-79618-41"}]: dispatch 2024-03-21T13:32:11.806 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:11 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4260986539' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi097-79618-41"}]: dispatch 2024-03-21T13:32:11.806 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:11 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi097-79618-41"}]: dispatch 2024-03-21T13:32:11.806 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:11 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4260986539' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi097-79618-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:11.806 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:11 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi097-79618-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:11.806 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:11 smithi097 ceph-mon[30040]: pgmap v347: 324 pgs: 324 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:32:11.807 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:11 smithi097 ceph-mon[30040]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:32:11.807 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:11 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-33"}]': finished 2024-03-21T13:32:11.807 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:11 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi097-79636-50"}]': finished 2024-03-21T13:32:11.807 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:11 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi097-79618-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:32:11.807 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:11 smithi097 ceph-mon[30040]: osdmap e256: 8 total, 8 up, 8 in 2024-03-21T13:32:11.807 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:11 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2965425103' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi097-79636-50"}]: dispatch 2024-03-21T13:32:11.807 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:11 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4260986539' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi097-79618-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi097-79618-41"}]: dispatch 2024-03-21T13:32:11.807 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:11 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi097-79636-50"}]: dispatch 2024-03-21T13:32:11.807 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:11 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi097-79618-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi097-79618-41"}]: dispatch 2024-03-21T13:32:12.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:32:11 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:32:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:32:13.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:13 smithi097 ceph-mon[39416]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:32:13.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:13 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi097-79636-50"}]': finished 2024-03-21T13:32:13.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:13 smithi097 ceph-mon[30040]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:32:13.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:13 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi097-79636-50"}]': finished 2024-03-21T13:32:13.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:13 smithi144 ceph-mon[33633]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:32:13.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:13 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi097-79636-50"}]': finished 2024-03-21T13:32:13.871 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:32:13 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:32:14.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:14 smithi097 ceph-mon[39416]: pgmap v350: 292 pgs: 292 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:32:14.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:14 smithi097 ceph-mon[39416]: osdmap e257: 8 total, 8 up, 8 in 2024-03-21T13:32:14.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:14 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi097-79618-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi097-79618-41"}]': finished 2024-03-21T13:32:14.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:14 smithi097 ceph-mon[39416]: osdmap e258: 8 total, 8 up, 8 in 2024-03-21T13:32:14.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:14 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2975093177' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi097-79636-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:32:14.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:14 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:32:14.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:14 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi097-79636-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:32:14.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:14 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:32:14.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:14 smithi097 ceph-mon[30040]: pgmap v350: 292 pgs: 292 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:32:14.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:14 smithi097 ceph-mon[30040]: osdmap e257: 8 total, 8 up, 8 in 2024-03-21T13:32:14.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:14 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi097-79618-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi097-79618-41"}]': finished 2024-03-21T13:32:14.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:14 smithi097 ceph-mon[30040]: osdmap e258: 8 total, 8 up, 8 in 2024-03-21T13:32:14.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:14 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2975093177' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi097-79636-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:32:14.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:14 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:32:14.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:14 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi097-79636-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:32:14.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:14 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:32:14.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:14 smithi144 ceph-mon[33633]: pgmap v350: 292 pgs: 292 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:32:14.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:14 smithi144 ceph-mon[33633]: osdmap e257: 8 total, 8 up, 8 in 2024-03-21T13:32:14.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:14 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi097-79618-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi097-79618-41"}]': finished 2024-03-21T13:32:14.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:14 smithi144 ceph-mon[33633]: osdmap e258: 8 total, 8 up, 8 in 2024-03-21T13:32:14.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:14 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2975093177' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi097-79636-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:32:14.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:14 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:32:14.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:14 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi097-79636-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:32:14.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:14 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:32:15.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:15 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:32:15.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:15 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi097-79636-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:32:15.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:15 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:32:15.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:15 smithi097 ceph-mon[39416]: osdmap e259: 8 total, 8 up, 8 in 2024-03-21T13:32:15.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:15 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:32:15.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:15 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:32:15.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:15 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:32:15.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:15 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi097-79636-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:32:15.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:15 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:32:15.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:15 smithi097 ceph-mon[30040]: osdmap e259: 8 total, 8 up, 8 in 2024-03-21T13:32:15.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:15 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:32:15.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:15 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:32:15.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:15 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:32:15.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:15 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi097-79636-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:32:15.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:15 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:32:15.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:15 smithi144 ceph-mon[33633]: osdmap e259: 8 total, 8 up, 8 in 2024-03-21T13:32:15.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:15 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:32:15.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:15 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:32:16.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:16 smithi097 ceph-mon[39416]: pgmap v353: 364 pgs: 72 creating+peering, 292 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:32:16.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:16 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-35", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:32:16.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:16 smithi097 ceph-mon[39416]: osdmap e260: 8 total, 8 up, 8 in 2024-03-21T13:32:16.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:16 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-35"}]: dispatch 2024-03-21T13:32:16.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:16 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-35"}]: dispatch 2024-03-21T13:32:16.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:16 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4260986539' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi097-79618-41"}]: dispatch 2024-03-21T13:32:16.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:16 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi097-79618-41"}]: dispatch 2024-03-21T13:32:16.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:16 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4087183409' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi097-79636-52"}]: dispatch 2024-03-21T13:32:16.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:16 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi097-79636-52"}]: dispatch 2024-03-21T13:32:16.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:16 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4087183409' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi097-79636-52"}]: dispatch 2024-03-21T13:32:16.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:16 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi097-79636-52"}]: dispatch 2024-03-21T13:32:16.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:16 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4087183409' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi097-79636-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:16.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:16 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi097-79636-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:16.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:16 smithi097 ceph-mon[30040]: pgmap v353: 364 pgs: 72 creating+peering, 292 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:32:16.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:16 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-35", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:32:16.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:16 smithi097 ceph-mon[30040]: osdmap e260: 8 total, 8 up, 8 in 2024-03-21T13:32:16.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:16 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-35"}]: dispatch 2024-03-21T13:32:16.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:16 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-35"}]: dispatch 2024-03-21T13:32:16.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:16 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4260986539' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi097-79618-41"}]: dispatch 2024-03-21T13:32:16.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:16 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi097-79618-41"}]: dispatch 2024-03-21T13:32:16.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:16 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4087183409' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi097-79636-52"}]: dispatch 2024-03-21T13:32:16.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:16 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi097-79636-52"}]: dispatch 2024-03-21T13:32:16.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:16 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4087183409' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi097-79636-52"}]: dispatch 2024-03-21T13:32:16.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:16 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi097-79636-52"}]: dispatch 2024-03-21T13:32:16.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:16 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4087183409' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi097-79636-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:16.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:16 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi097-79636-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:16.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:16 smithi144 ceph-mon[33633]: pgmap v353: 364 pgs: 72 creating+peering, 292 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:32:16.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:16 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-35", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:32:16.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:16 smithi144 ceph-mon[33633]: osdmap e260: 8 total, 8 up, 8 in 2024-03-21T13:32:16.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:16 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-35"}]: dispatch 2024-03-21T13:32:16.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:16 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-35"}]: dispatch 2024-03-21T13:32:16.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:16 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4260986539' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi097-79618-41"}]: dispatch 2024-03-21T13:32:16.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:16 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi097-79618-41"}]: dispatch 2024-03-21T13:32:16.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:16 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4087183409' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi097-79636-52"}]: dispatch 2024-03-21T13:32:16.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:16 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi097-79636-52"}]: dispatch 2024-03-21T13:32:16.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:16 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4087183409' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi097-79636-52"}]: dispatch 2024-03-21T13:32:16.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:16 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi097-79636-52"}]: dispatch 2024-03-21T13:32:16.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:16 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4087183409' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi097-79636-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:16.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:16 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi097-79636-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:17.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:17 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-35"}]': finished 2024-03-21T13:32:17.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:17 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi097-79618-41"}]': finished 2024-03-21T13:32:17.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:17 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi097-79636-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:32:17.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:17 smithi144 ceph-mon[33633]: osdmap e261: 8 total, 8 up, 8 in 2024-03-21T13:32:17.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:17 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4087183409' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi097-79636-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi097-79636-52"}]: dispatch 2024-03-21T13:32:17.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:17 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-35", "mode": "writeback"}]: dispatch 2024-03-21T13:32:17.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:17 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi097-79636-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi097-79636-52"}]: dispatch 2024-03-21T13:32:17.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:17 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4260986539' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi097-79618-41"}]: dispatch 2024-03-21T13:32:17.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:17 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-35", "mode": "writeback"}]: dispatch 2024-03-21T13:32:17.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:17 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi097-79618-41"}]: dispatch 2024-03-21T13:32:17.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:17 smithi144 ceph-mon[33633]: pgmap v356: 324 pgs: 32 creating+peering, 292 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:32:17.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:17 smithi144 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:32:17.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:17 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-35"}]': finished 2024-03-21T13:32:17.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:17 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi097-79618-41"}]': finished 2024-03-21T13:32:17.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:17 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi097-79636-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:32:17.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:17 smithi097 ceph-mon[30040]: osdmap e261: 8 total, 8 up, 8 in 2024-03-21T13:32:17.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:17 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4087183409' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi097-79636-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi097-79636-52"}]: dispatch 2024-03-21T13:32:17.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:17 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-35", "mode": "writeback"}]: dispatch 2024-03-21T13:32:17.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:17 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi097-79636-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi097-79636-52"}]: dispatch 2024-03-21T13:32:17.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:17 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4260986539' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi097-79618-41"}]: dispatch 2024-03-21T13:32:17.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:17 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-35", "mode": "writeback"}]: dispatch 2024-03-21T13:32:17.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:17 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi097-79618-41"}]: dispatch 2024-03-21T13:32:17.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:17 smithi097 ceph-mon[30040]: pgmap v356: 324 pgs: 32 creating+peering, 292 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:32:17.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:17 smithi097 ceph-mon[30040]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:32:17.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:17 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-35"}]': finished 2024-03-21T13:32:17.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:17 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi097-79618-41"}]': finished 2024-03-21T13:32:17.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:17 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi097-79636-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:32:17.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:17 smithi097 ceph-mon[39416]: osdmap e261: 8 total, 8 up, 8 in 2024-03-21T13:32:17.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:17 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4087183409' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi097-79636-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi097-79636-52"}]: dispatch 2024-03-21T13:32:17.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:17 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-35", "mode": "writeback"}]: dispatch 2024-03-21T13:32:17.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:17 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi097-79636-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi097-79636-52"}]: dispatch 2024-03-21T13:32:17.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:17 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4260986539' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi097-79618-41"}]: dispatch 2024-03-21T13:32:17.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:17 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-35", "mode": "writeback"}]: dispatch 2024-03-21T13:32:17.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:17 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi097-79618-41"}]: dispatch 2024-03-21T13:32:17.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:17 smithi097 ceph-mon[39416]: pgmap v356: 324 pgs: 32 creating+peering, 292 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:32:17.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:17 smithi097 ceph-mon[39416]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:32:18.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:18 smithi144 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:32:18.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:18 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-35", "mode": "writeback"}]': finished 2024-03-21T13:32:18.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:18 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi097-79618-41"}]': finished 2024-03-21T13:32:18.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:18 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/536866655' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi097-79618-42"}]: dispatch 2024-03-21T13:32:18.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:18 smithi144 ceph-mon[33633]: osdmap e262: 8 total, 8 up, 8 in 2024-03-21T13:32:18.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:18 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi097-79618-42"}]: dispatch 2024-03-21T13:32:18.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:18 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/536866655' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi097-79618-42"}]: dispatch 2024-03-21T13:32:18.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:18 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi097-79618-42"}]: dispatch 2024-03-21T13:32:18.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:18 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/536866655' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi097-79618-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:18.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:18 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi097-79618-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:18.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:18 smithi097 ceph-mon[39416]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:32:18.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:18 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-35", "mode": "writeback"}]': finished 2024-03-21T13:32:18.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:18 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi097-79618-41"}]': finished 2024-03-21T13:32:18.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:18 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/536866655' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi097-79618-42"}]: dispatch 2024-03-21T13:32:18.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:18 smithi097 ceph-mon[39416]: osdmap e262: 8 total, 8 up, 8 in 2024-03-21T13:32:18.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:18 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi097-79618-42"}]: dispatch 2024-03-21T13:32:18.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:18 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/536866655' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi097-79618-42"}]: dispatch 2024-03-21T13:32:18.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:18 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi097-79618-42"}]: dispatch 2024-03-21T13:32:18.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:18 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/536866655' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi097-79618-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:18.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:18 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi097-79618-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:18.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:18 smithi097 ceph-mon[30040]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:32:18.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:18 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-35", "mode": "writeback"}]': finished 2024-03-21T13:32:18.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:18 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi097-79618-41"}]': finished 2024-03-21T13:32:18.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:18 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/536866655' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi097-79618-42"}]: dispatch 2024-03-21T13:32:18.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:18 smithi097 ceph-mon[30040]: osdmap e262: 8 total, 8 up, 8 in 2024-03-21T13:32:18.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:18 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi097-79618-42"}]: dispatch 2024-03-21T13:32:18.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:18 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/536866655' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi097-79618-42"}]: dispatch 2024-03-21T13:32:18.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:18 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi097-79618-42"}]: dispatch 2024-03-21T13:32:18.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:18 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/536866655' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi097-79618-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:18.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:18 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi097-79618-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:19.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:19 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi097-79636-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi097-79636-52"}]': finished 2024-03-21T13:32:19.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:19 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi097-79618-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:32:19.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:19 smithi144 ceph-mon[33633]: osdmap e263: 8 total, 8 up, 8 in 2024-03-21T13:32:19.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:19 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/536866655' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi097-79618-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi097-79618-42"}]: dispatch 2024-03-21T13:32:19.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:19 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi097-79618-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi097-79618-42"}]: dispatch 2024-03-21T13:32:19.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:19 smithi144 ceph-mon[33633]: pgmap v359: 332 pgs: 1 creating+peering, 7 unknown, 324 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:32:19.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:19 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi097-79636-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi097-79636-52"}]': finished 2024-03-21T13:32:19.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:19 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi097-79618-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:32:19.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:19 smithi097 ceph-mon[39416]: osdmap e263: 8 total, 8 up, 8 in 2024-03-21T13:32:19.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:19 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/536866655' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi097-79618-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi097-79618-42"}]: dispatch 2024-03-21T13:32:19.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:19 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi097-79618-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi097-79618-42"}]: dispatch 2024-03-21T13:32:19.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:19 smithi097 ceph-mon[39416]: pgmap v359: 332 pgs: 1 creating+peering, 7 unknown, 324 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:32:19.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:19 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi097-79636-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi097-79636-52"}]': finished 2024-03-21T13:32:19.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:19 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi097-79618-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:32:19.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:19 smithi097 ceph-mon[30040]: osdmap e263: 8 total, 8 up, 8 in 2024-03-21T13:32:19.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:19 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/536866655' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi097-79618-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi097-79618-42"}]: dispatch 2024-03-21T13:32:19.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:19 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi097-79618-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi097-79618-42"}]: dispatch 2024-03-21T13:32:19.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:19 smithi097 ceph-mon[30040]: pgmap v359: 332 pgs: 1 creating+peering, 7 unknown, 324 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:32:20.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:20 smithi144 ceph-mon[33633]: osdmap e264: 8 total, 8 up, 8 in 2024-03-21T13:32:20.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:20 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:32:20.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:20 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:32:20.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:20 smithi097 ceph-mon[39416]: osdmap e264: 8 total, 8 up, 8 in 2024-03-21T13:32:20.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:20 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:32:20.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:20 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:32:20.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:20 smithi097 ceph-mon[30040]: osdmap e264: 8 total, 8 up, 8 in 2024-03-21T13:32:20.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:20 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:32:20.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:20 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:32:21.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:21 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi097-79618-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi097-79618-42"}]': finished 2024-03-21T13:32:21.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:21 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:32:21.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:21 smithi144 ceph-mon[33633]: osdmap e265: 8 total, 8 up, 8 in 2024-03-21T13:32:21.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:21 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4087183409' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi097-79636-52"}]: dispatch 2024-03-21T13:32:21.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:21 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi097-79636-52"}]: dispatch 2024-03-21T13:32:21.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:21 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-35"}]: dispatch 2024-03-21T13:32:21.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:21 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-35"}]: dispatch 2024-03-21T13:32:21.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:21 smithi144 ceph-mon[33633]: pgmap v362: 332 pgs: 8 unknown, 324 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:32:21.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:21 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi097-79618-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi097-79618-42"}]': finished 2024-03-21T13:32:21.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:21 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:32:21.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:21 smithi097 ceph-mon[39416]: osdmap e265: 8 total, 8 up, 8 in 2024-03-21T13:32:21.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:21 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4087183409' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi097-79636-52"}]: dispatch 2024-03-21T13:32:21.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:21 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi097-79636-52"}]: dispatch 2024-03-21T13:32:21.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:21 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-35"}]: dispatch 2024-03-21T13:32:21.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:21 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-35"}]: dispatch 2024-03-21T13:32:21.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:21 smithi097 ceph-mon[39416]: pgmap v362: 332 pgs: 8 unknown, 324 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:32:21.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:21 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi097-79618-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi097-79618-42"}]': finished 2024-03-21T13:32:21.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:21 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:32:21.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:21 smithi097 ceph-mon[30040]: osdmap e265: 8 total, 8 up, 8 in 2024-03-21T13:32:21.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:21 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4087183409' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi097-79636-52"}]: dispatch 2024-03-21T13:32:21.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:21 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi097-79636-52"}]: dispatch 2024-03-21T13:32:21.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:21 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-35"}]: dispatch 2024-03-21T13:32:21.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:21 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-35"}]: dispatch 2024-03-21T13:32:21.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:21 smithi097 ceph-mon[30040]: pgmap v362: 332 pgs: 8 unknown, 324 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:32:22.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:32:21 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:32:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:32:22.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:22 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi097-79636-52"}]': finished 2024-03-21T13:32:22.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:22 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-35"}]': finished 2024-03-21T13:32:22.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:22 smithi097 ceph-mon[39416]: osdmap e266: 8 total, 8 up, 8 in 2024-03-21T13:32:22.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:22 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4087183409' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi097-79636-52"}]: dispatch 2024-03-21T13:32:22.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:22 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi097-79636-52"}]: dispatch 2024-03-21T13:32:22.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:22 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi097-79636-52"}]': finished 2024-03-21T13:32:22.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:22 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-35"}]': finished 2024-03-21T13:32:22.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:22 smithi097 ceph-mon[30040]: osdmap e266: 8 total, 8 up, 8 in 2024-03-21T13:32:22.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:22 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4087183409' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi097-79636-52"}]: dispatch 2024-03-21T13:32:22.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:22 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi097-79636-52"}]: dispatch 2024-03-21T13:32:22.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:22 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi097-79636-52"}]': finished 2024-03-21T13:32:22.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:22 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-35"}]': finished 2024-03-21T13:32:22.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:22 smithi144 ceph-mon[33633]: osdmap e266: 8 total, 8 up, 8 in 2024-03-21T13:32:22.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:22 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4087183409' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi097-79636-52"}]: dispatch 2024-03-21T13:32:22.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:22 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi097-79636-52"}]: dispatch 2024-03-21T13:32:23.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:23 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi097-79636-52"}]': finished 2024-03-21T13:32:23.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:23 smithi097 ceph-mon[39416]: osdmap e267: 8 total, 8 up, 8 in 2024-03-21T13:32:23.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:23 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/536866655' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi097-79618-42"}]: dispatch 2024-03-21T13:32:23.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:23 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi097-79618-42"}]: dispatch 2024-03-21T13:32:23.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:23 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4268387116' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi097-79636-53"}]: dispatch 2024-03-21T13:32:23.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:23 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi097-79636-53"}]: dispatch 2024-03-21T13:32:23.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:23 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4268387116' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi097-79636-53"}]: dispatch 2024-03-21T13:32:23.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:23 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi097-79636-53"}]: dispatch 2024-03-21T13:32:23.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:23 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4268387116' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi097-79636-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:23.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:23 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi097-79636-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:23.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:23 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:32:23.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:23 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:32:23.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:23 smithi097 ceph-mon[39416]: pgmap v365: 324 pgs: 324 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:32:23.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:23 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi097-79636-52"}]': finished 2024-03-21T13:32:23.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:23 smithi097 ceph-mon[30040]: osdmap e267: 8 total, 8 up, 8 in 2024-03-21T13:32:23.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:23 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/536866655' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi097-79618-42"}]: dispatch 2024-03-21T13:32:23.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:23 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi097-79618-42"}]: dispatch 2024-03-21T13:32:23.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:23 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4268387116' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi097-79636-53"}]: dispatch 2024-03-21T13:32:23.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:23 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi097-79636-53"}]: dispatch 2024-03-21T13:32:23.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:23 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4268387116' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi097-79636-53"}]: dispatch 2024-03-21T13:32:23.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:23 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi097-79636-53"}]: dispatch 2024-03-21T13:32:23.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:23 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4268387116' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi097-79636-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:23.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:23 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi097-79636-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:23.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:23 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:32:23.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:23 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:32:23.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:23 smithi097 ceph-mon[30040]: pgmap v365: 324 pgs: 324 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:32:23.871 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:32:23 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:32:23.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:23 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi097-79636-52"}]': finished 2024-03-21T13:32:23.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:23 smithi144 ceph-mon[33633]: osdmap e267: 8 total, 8 up, 8 in 2024-03-21T13:32:23.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:23 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/536866655' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi097-79618-42"}]: dispatch 2024-03-21T13:32:23.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:23 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi097-79618-42"}]: dispatch 2024-03-21T13:32:23.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:23 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4268387116' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi097-79636-53"}]: dispatch 2024-03-21T13:32:23.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:23 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi097-79636-53"}]: dispatch 2024-03-21T13:32:23.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:23 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4268387116' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi097-79636-53"}]: dispatch 2024-03-21T13:32:23.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:23 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi097-79636-53"}]: dispatch 2024-03-21T13:32:23.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:23 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4268387116' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi097-79636-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:23.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:23 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi097-79636-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:23.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:23 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:32:23.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:23 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:32:23.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:23 smithi144 ceph-mon[33633]: pgmap v365: 324 pgs: 324 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:32:24.405 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleWrite (7019 ms) 2024-03-21T13:32:24.405 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ RUN ] LibRadosAioEC.WaitForComplete 2024-03-21T13:32:24.405 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ OK ] LibRadosAioEC.WaitForComplete (6971 ms) 2024-03-21T13:32:24.405 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip 2024-03-21T13:32:24.405 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip (6992 ms) 2024-03-21T13:32:24.405 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip2 2024-03-21T13:32:24.405 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip2 (7036 ms) 2024-03-21T13:32:24.405 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripAppend 2024-03-21T13:32:24.405 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripAppend (7041 ms) 2024-03-21T13:32:24.405 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ RUN ] LibRadosAioEC.IsComplete 2024-03-21T13:32:24.405 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ OK ] LibRadosAioEC.IsComplete (7023 ms) 2024-03-21T13:32:24.405 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ RUN ] LibRadosAioEC.IsSafe 2024-03-21T13:32:24.405 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ OK ] LibRadosAioEC.IsSafe (7045 ms) 2024-03-21T13:32:24.406 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ RUN ] LibRadosAioEC.ReturnValue 2024-03-21T13:32:24.406 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ OK ] LibRadosAioEC.ReturnValue (6857 ms) 2024-03-21T13:32:24.406 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ RUN ] LibRadosAioEC.Flush 2024-03-21T13:32:24.406 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ OK ] LibRadosAioEC.Flush (7033 ms) 2024-03-21T13:32:24.406 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ RUN ] LibRadosAioEC.FlushAsync 2024-03-21T13:32:24.406 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ OK ] LibRadosAioEC.FlushAsync (7042 ms) 2024-03-21T13:32:24.406 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripWriteFull 2024-03-21T13:32:24.406 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripWriteFull (6936 ms) 2024-03-21T13:32:24.406 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStat 2024-03-21T13:32:24.406 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStat (7054 ms) 2024-03-21T13:32:24.406 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStatNS 2024-03-21T13:32:24.406 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStatNS (6949 ms) 2024-03-21T13:32:24.406 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ RUN ] LibRadosAioEC.StatRemove 2024-03-21T13:32:24.406 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ OK ] LibRadosAioEC.StatRemove (7052 ms) 2024-03-21T13:32:24.406 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ RUN ] LibRadosAioEC.ExecuteClass 2024-03-21T13:32:24.407 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ OK ] LibRadosAioEC.ExecuteClass (7026 ms) 2024-03-21T13:32:24.407 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ RUN ] LibRadosAioEC.MultiWrite 2024-03-21T13:32:24.407 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ OK ] LibRadosAioEC.MultiWrite (7056 ms) 2024-03-21T13:32:24.407 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [----------] 16 tests from LibRadosAioEC (112132 ms total) 2024-03-21T13:32:24.407 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: 2024-03-21T13:32:24.407 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [----------] Global test environment tear-down 2024-03-21T13:32:24.407 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [==========] 42 tests from 2 test suites ran. (204975 ms total) 2024-03-21T13:32:24.407 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio: [ PASSED ] 42 tests. 2024-03-21T13:32:24.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:24 smithi097 ceph-mon[39416]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:32:24.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:24 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi097-79618-42"}]': finished 2024-03-21T13:32:24.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:24 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi097-79636-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:32:24.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:24 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:32:24.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:24 smithi097 ceph-mon[39416]: osdmap e268: 8 total, 8 up, 8 in 2024-03-21T13:32:24.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:24 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/536866655' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi097-79618-42"}]: dispatch 2024-03-21T13:32:24.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:24 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4268387116' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi097-79636-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi097-79636-53"}]: dispatch 2024-03-21T13:32:24.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:24 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi097-79618-42"}]: dispatch 2024-03-21T13:32:24.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:24 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-35"}]: dispatch 2024-03-21T13:32:24.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:24 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi097-79636-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi097-79636-53"}]: dispatch 2024-03-21T13:32:24.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:24 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-35"}]: dispatch 2024-03-21T13:32:24.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:24 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:32:24.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:24 smithi097 ceph-mon[30040]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:32:24.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:24 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi097-79618-42"}]': finished 2024-03-21T13:32:24.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:24 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi097-79636-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:32:24.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:24 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:32:24.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:24 smithi097 ceph-mon[30040]: osdmap e268: 8 total, 8 up, 8 in 2024-03-21T13:32:24.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:24 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/536866655' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi097-79618-42"}]: dispatch 2024-03-21T13:32:24.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:24 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4268387116' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi097-79636-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi097-79636-53"}]: dispatch 2024-03-21T13:32:24.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:24 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi097-79618-42"}]: dispatch 2024-03-21T13:32:24.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:24 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-35"}]: dispatch 2024-03-21T13:32:24.806 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:24 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi097-79636-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi097-79636-53"}]: dispatch 2024-03-21T13:32:24.806 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:24 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-35"}]: dispatch 2024-03-21T13:32:24.806 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:24 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:32:24.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:24 smithi144 ceph-mon[33633]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:32:24.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:24 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi097-79618-42"}]': finished 2024-03-21T13:32:24.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:24 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi097-79636-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:32:24.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:24 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:32:24.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:24 smithi144 ceph-mon[33633]: osdmap e268: 8 total, 8 up, 8 in 2024-03-21T13:32:24.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:24 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/536866655' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi097-79618-42"}]: dispatch 2024-03-21T13:32:24.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:24 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4268387116' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi097-79636-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi097-79636-53"}]: dispatch 2024-03-21T13:32:24.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:24 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi097-79618-42"}]: dispatch 2024-03-21T13:32:24.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:24 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-35"}]: dispatch 2024-03-21T13:32:24.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:24 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi097-79636-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi097-79636-53"}]: dispatch 2024-03-21T13:32:24.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:24 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-35"}]: dispatch 2024-03-21T13:32:24.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:24 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:32:25.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:25 smithi097 ceph-mon[39416]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:32:25.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:25 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi097-79618-42"}]': finished 2024-03-21T13:32:25.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:25 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-35"}]': finished 2024-03-21T13:32:25.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:25 smithi097 ceph-mon[39416]: pgmap v368: 324 pgs: 324 active+clean; 4.6 MiB data, 594 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-03-21T13:32:25.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:25 smithi097 ceph-mon[39416]: osdmap e269: 8 total, 8 up, 8 in 2024-03-21T13:32:25.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:25 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:32:25.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:25 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:32:25.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:25 smithi097 ceph-mon[30040]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:32:25.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:25 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi097-79618-42"}]': finished 2024-03-21T13:32:25.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:25 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-35"}]': finished 2024-03-21T13:32:25.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:25 smithi097 ceph-mon[30040]: pgmap v368: 324 pgs: 324 active+clean; 4.6 MiB data, 594 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-03-21T13:32:25.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:25 smithi097 ceph-mon[30040]: osdmap e269: 8 total, 8 up, 8 in 2024-03-21T13:32:25.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:25 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:32:25.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:25 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:32:25.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:25 smithi144 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:32:25.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:25 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi097-79618-42"}]': finished 2024-03-21T13:32:25.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:25 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-35"}]': finished 2024-03-21T13:32:25.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:25 smithi144 ceph-mon[33633]: pgmap v368: 324 pgs: 324 active+clean; 4.6 MiB data, 594 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-03-21T13:32:25.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:25 smithi144 ceph-mon[33633]: osdmap e269: 8 total, 8 up, 8 in 2024-03-21T13:32:25.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:25 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:32:25.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:25 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:32:26.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:26 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi097-79636-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi097-79636-53"}]': finished 2024-03-21T13:32:26.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:26 smithi097 ceph-mon[39416]: osdmap e270: 8 total, 8 up, 8 in 2024-03-21T13:32:26.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:26 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi097-79636-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi097-79636-53"}]': finished 2024-03-21T13:32:26.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:26 smithi097 ceph-mon[30040]: osdmap e270: 8 total, 8 up, 8 in 2024-03-21T13:32:26.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:26 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi097-79636-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi097-79636-53"}]': finished 2024-03-21T13:32:26.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:26 smithi144 ceph-mon[33633]: osdmap e270: 8 total, 8 up, 8 in 2024-03-21T13:32:27.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:27 smithi097 ceph-mon[39416]: osdmap e271: 8 total, 8 up, 8 in 2024-03-21T13:32:27.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:27 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:32:27.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:27 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:32:27.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:27 smithi097 ceph-mon[39416]: pgmap v371: 332 pgs: 40 unknown, 292 active+clean; 4.6 MiB data, 594 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-21T13:32:27.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:27 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:32:27.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:27 smithi097 ceph-mon[39416]: osdmap e272: 8 total, 8 up, 8 in 2024-03-21T13:32:27.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:27 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:32:27.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:27 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4268387116' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi097-79636-53"}]: dispatch 2024-03-21T13:32:27.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:27 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:32:27.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:27 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi097-79636-53"}]: dispatch 2024-03-21T13:32:27.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:27 smithi097 ceph-mon[30040]: osdmap e271: 8 total, 8 up, 8 in 2024-03-21T13:32:27.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:27 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:32:27.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:27 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:32:27.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:27 smithi097 ceph-mon[30040]: pgmap v371: 332 pgs: 40 unknown, 292 active+clean; 4.6 MiB data, 594 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-21T13:32:27.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:27 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:32:27.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:27 smithi097 ceph-mon[30040]: osdmap e272: 8 total, 8 up, 8 in 2024-03-21T13:32:27.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:27 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:32:27.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:27 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4268387116' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi097-79636-53"}]: dispatch 2024-03-21T13:32:27.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:27 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:32:27.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:27 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi097-79636-53"}]: dispatch 2024-03-21T13:32:27.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:27 smithi144 ceph-mon[33633]: osdmap e271: 8 total, 8 up, 8 in 2024-03-21T13:32:27.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:27 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:32:27.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:27 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:32:27.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:27 smithi144 ceph-mon[33633]: pgmap v371: 332 pgs: 40 unknown, 292 active+clean; 4.6 MiB data, 594 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-21T13:32:27.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:27 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:32:27.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:27 smithi144 ceph-mon[33633]: osdmap e272: 8 total, 8 up, 8 in 2024-03-21T13:32:27.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:27 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:32:27.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:27 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4268387116' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi097-79636-53"}]: dispatch 2024-03-21T13:32:27.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:27 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:32:27.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:27 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi097-79636-53"}]: dispatch 2024-03-21T13:32:29.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:29 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-37", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:32:29.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:29 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi097-79636-53"}]': finished 2024-03-21T13:32:29.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:29 smithi097 ceph-mon[30040]: osdmap e273: 8 total, 8 up, 8 in 2024-03-21T13:32:29.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:29 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-37"}]: dispatch 2024-03-21T13:32:29.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:29 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4268387116' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi097-79636-53"}]: dispatch 2024-03-21T13:32:29.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:29 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-37"}]: dispatch 2024-03-21T13:32:29.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:29 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi097-79636-53"}]: dispatch 2024-03-21T13:32:29.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:29 smithi097 ceph-mon[30040]: pgmap v374: 324 pgs: 324 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-03-21T13:32:29.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:29 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-37", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:32:29.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:29 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi097-79636-53"}]': finished 2024-03-21T13:32:29.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:29 smithi097 ceph-mon[39416]: osdmap e273: 8 total, 8 up, 8 in 2024-03-21T13:32:29.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:29 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-37"}]: dispatch 2024-03-21T13:32:29.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:29 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4268387116' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi097-79636-53"}]: dispatch 2024-03-21T13:32:29.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:29 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-37"}]: dispatch 2024-03-21T13:32:29.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:29 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi097-79636-53"}]: dispatch 2024-03-21T13:32:29.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:29 smithi097 ceph-mon[39416]: pgmap v374: 324 pgs: 324 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-03-21T13:32:29.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:29 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-37", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:32:29.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:29 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi097-79636-53"}]': finished 2024-03-21T13:32:29.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:29 smithi144 ceph-mon[33633]: osdmap e273: 8 total, 8 up, 8 in 2024-03-21T13:32:29.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:29 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-37"}]: dispatch 2024-03-21T13:32:29.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:29 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4268387116' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi097-79636-53"}]: dispatch 2024-03-21T13:32:29.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:29 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-37"}]: dispatch 2024-03-21T13:32:29.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:29 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi097-79636-53"}]: dispatch 2024-03-21T13:32:29.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:29 smithi144 ceph-mon[33633]: pgmap v374: 324 pgs: 324 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-03-21T13:32:30.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:30 smithi097 ceph-mon[39416]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:32:30.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:30 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-37"}]': finished 2024-03-21T13:32:30.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:30 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi097-79636-53"}]': finished 2024-03-21T13:32:30.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:30 smithi097 ceph-mon[39416]: osdmap e274: 8 total, 8 up, 8 in 2024-03-21T13:32:30.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:30 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-37", "mode": "writeback"}]: dispatch 2024-03-21T13:32:30.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:30 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-37", "mode": "writeback"}]: dispatch 2024-03-21T13:32:30.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:30 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2834570801' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi097-79636-54"}]: dispatch 2024-03-21T13:32:30.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:30 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi097-79636-54"}]: dispatch 2024-03-21T13:32:30.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:30 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2834570801' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi097-79636-54"}]: dispatch 2024-03-21T13:32:30.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:30 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi097-79636-54"}]: dispatch 2024-03-21T13:32:30.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:30 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2834570801' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi097-79636-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:30.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:30 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi097-79636-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:30.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:30 smithi097 ceph-mon[30040]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:32:30.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:30 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-37"}]': finished 2024-03-21T13:32:30.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:30 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi097-79636-53"}]': finished 2024-03-21T13:32:30.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:30 smithi097 ceph-mon[30040]: osdmap e274: 8 total, 8 up, 8 in 2024-03-21T13:32:30.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:30 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-37", "mode": "writeback"}]: dispatch 2024-03-21T13:32:30.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:30 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-37", "mode": "writeback"}]: dispatch 2024-03-21T13:32:30.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:30 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2834570801' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi097-79636-54"}]: dispatch 2024-03-21T13:32:30.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:30 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi097-79636-54"}]: dispatch 2024-03-21T13:32:30.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:30 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2834570801' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi097-79636-54"}]: dispatch 2024-03-21T13:32:30.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:30 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi097-79636-54"}]: dispatch 2024-03-21T13:32:30.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:30 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2834570801' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi097-79636-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:30.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:30 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi097-79636-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:30.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:30 smithi144 ceph-mon[33633]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:32:30.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:30 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-37"}]': finished 2024-03-21T13:32:30.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:30 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi097-79636-53"}]': finished 2024-03-21T13:32:30.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:30 smithi144 ceph-mon[33633]: osdmap e274: 8 total, 8 up, 8 in 2024-03-21T13:32:30.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:30 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-37", "mode": "writeback"}]: dispatch 2024-03-21T13:32:30.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:30 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-37", "mode": "writeback"}]: dispatch 2024-03-21T13:32:30.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:30 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2834570801' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi097-79636-54"}]: dispatch 2024-03-21T13:32:30.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:30 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi097-79636-54"}]: dispatch 2024-03-21T13:32:30.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:30 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2834570801' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi097-79636-54"}]: dispatch 2024-03-21T13:32:30.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:30 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi097-79636-54"}]: dispatch 2024-03-21T13:32:30.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:30 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2834570801' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi097-79636-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:30.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:30 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi097-79636-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:31.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:31 smithi097 ceph-mon[39416]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:32:31.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:31 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-37", "mode": "writeback"}]': finished 2024-03-21T13:32:31.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:31 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi097-79636-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:32:31.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:31 smithi097 ceph-mon[39416]: osdmap e275: 8 total, 8 up, 8 in 2024-03-21T13:32:31.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:31 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2834570801' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi097-79636-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi097-79636-54"}]: dispatch 2024-03-21T13:32:31.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:31 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi097-79636-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi097-79636-54"}]: dispatch 2024-03-21T13:32:31.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:31 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:32:31.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:31 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:32:31.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:31 smithi097 ceph-mon[39416]: pgmap v377: 324 pgs: 324 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:32:31.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:31 smithi097 ceph-mon[30040]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:32:31.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:31 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-37", "mode": "writeback"}]': finished 2024-03-21T13:32:31.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:31 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi097-79636-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:32:31.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:31 smithi097 ceph-mon[30040]: osdmap e275: 8 total, 8 up, 8 in 2024-03-21T13:32:31.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:31 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2834570801' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi097-79636-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi097-79636-54"}]: dispatch 2024-03-21T13:32:31.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:31 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi097-79636-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi097-79636-54"}]: dispatch 2024-03-21T13:32:31.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:31 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:32:31.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:31 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:32:31.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:31 smithi097 ceph-mon[30040]: pgmap v377: 324 pgs: 324 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:32:31.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:31 smithi144 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:32:31.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:31 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-37", "mode": "writeback"}]': finished 2024-03-21T13:32:31.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:31 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi097-79636-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:32:31.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:31 smithi144 ceph-mon[33633]: osdmap e275: 8 total, 8 up, 8 in 2024-03-21T13:32:31.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:31 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2834570801' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi097-79636-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi097-79636-54"}]: dispatch 2024-03-21T13:32:31.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:31 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi097-79636-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi097-79636-54"}]: dispatch 2024-03-21T13:32:31.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:31 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:32:31.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:31 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:32:31.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:31 smithi144 ceph-mon[33633]: pgmap v377: 324 pgs: 324 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:32:32.286 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:32:31 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:32:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:32:32.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:32 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:32:32.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:32 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-37"}]: dispatch 2024-03-21T13:32:32.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:32 smithi097 ceph-mon[30040]: osdmap e276: 8 total, 8 up, 8 in 2024-03-21T13:32:32.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:32 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-37"}]: dispatch 2024-03-21T13:32:32.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:32 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:32:32.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:32 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-37"}]: dispatch 2024-03-21T13:32:32.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:32 smithi097 ceph-mon[39416]: osdmap e276: 8 total, 8 up, 8 in 2024-03-21T13:32:32.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:32 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-37"}]: dispatch 2024-03-21T13:32:32.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:32 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:32:32.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:32 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-37"}]: dispatch 2024-03-21T13:32:32.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:32 smithi144 ceph-mon[33633]: osdmap e276: 8 total, 8 up, 8 in 2024-03-21T13:32:32.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:32 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-37"}]: dispatch 2024-03-21T13:32:33.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:33 smithi097 ceph-mon[30040]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:32:33.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:33 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi097-79636-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi097-79636-54"}]': finished 2024-03-21T13:32:33.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:33 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-37"}]': finished 2024-03-21T13:32:33.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:33 smithi097 ceph-mon[30040]: osdmap e277: 8 total, 8 up, 8 in 2024-03-21T13:32:33.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:33 smithi097 ceph-mon[30040]: pgmap v380: 332 pgs: 8 unknown, 324 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:32:33.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:33 smithi097 ceph-mon[39416]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:32:33.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:33 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi097-79636-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi097-79636-54"}]': finished 2024-03-21T13:32:33.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:33 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-37"}]': finished 2024-03-21T13:32:33.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:33 smithi097 ceph-mon[39416]: osdmap e277: 8 total, 8 up, 8 in 2024-03-21T13:32:33.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:33 smithi097 ceph-mon[39416]: pgmap v380: 332 pgs: 8 unknown, 324 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:32:33.600 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:33 smithi144 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:32:33.600 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:33 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi097-79636-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi097-79636-54"}]': finished 2024-03-21T13:32:33.600 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:33 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-37"}]': finished 2024-03-21T13:32:33.600 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:33 smithi144 ceph-mon[33633]: osdmap e277: 8 total, 8 up, 8 in 2024-03-21T13:32:33.600 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:33 smithi144 ceph-mon[33633]: pgmap v380: 332 pgs: 8 unknown, 324 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:32:33.871 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:32:33 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:32:34.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:34 smithi097 ceph-mon[39416]: osdmap e278: 8 total, 8 up, 8 in 2024-03-21T13:32:34.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:34 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:32:34.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:34 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:32:34.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:34 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:32:34.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:34 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:32:34.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:34 smithi097 ceph-mon[30040]: osdmap e278: 8 total, 8 up, 8 in 2024-03-21T13:32:34.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:34 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:32:34.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:34 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:32:34.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:34 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:32:34.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:34 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:32:34.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:34 smithi144 ceph-mon[33633]: osdmap e278: 8 total, 8 up, 8 in 2024-03-21T13:32:34.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:34 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:32:34.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:34 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:32:34.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:34 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:32:34.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:34 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:32:35.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:35 smithi144 ceph-mon[33633]: osdmap e279: 8 total, 8 up, 8 in 2024-03-21T13:32:35.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:35 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2834570801' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi097-79636-54"}]: dispatch 2024-03-21T13:32:35.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:35 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:32:35.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:35 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi097-79636-54"}]: dispatch 2024-03-21T13:32:35.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:35 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:32:35.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:35 smithi144 ceph-mon[33633]: pgmap v383: 324 pgs: 32 unknown, 292 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:32:35.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:35 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:32:35.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:35 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:32:35.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:35 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:32:35.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:35 smithi097 ceph-mon[39416]: osdmap e279: 8 total, 8 up, 8 in 2024-03-21T13:32:35.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:35 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2834570801' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi097-79636-54"}]: dispatch 2024-03-21T13:32:35.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:35 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:32:35.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:35 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi097-79636-54"}]: dispatch 2024-03-21T13:32:35.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:35 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:32:35.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:35 smithi097 ceph-mon[39416]: pgmap v383: 324 pgs: 32 unknown, 292 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:32:35.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:35 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:32:35.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:35 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:32:35.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:35 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:32:35.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:35 smithi097 ceph-mon[30040]: osdmap e279: 8 total, 8 up, 8 in 2024-03-21T13:32:35.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:35 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2834570801' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi097-79636-54"}]: dispatch 2024-03-21T13:32:35.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:35 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:32:35.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:35 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi097-79636-54"}]: dispatch 2024-03-21T13:32:35.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:35 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:32:35.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:35 smithi097 ceph-mon[30040]: pgmap v383: 324 pgs: 32 unknown, 292 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:32:35.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:35 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:32:35.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:35 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:32:35.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:35 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:32:36.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:36 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi097-79636-54"}]': finished 2024-03-21T13:32:36.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:36 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:32:36.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:36 smithi144 ceph-mon[33633]: osdmap e280: 8 total, 8 up, 8 in 2024-03-21T13:32:36.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:36 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2834570801' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi097-79636-54"}]: dispatch 2024-03-21T13:32:36.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:36 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi097-79636-54"}]: dispatch 2024-03-21T13:32:36.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:36 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:32:36.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:36 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:32:36.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:36 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi097-79636-54"}]': finished 2024-03-21T13:32:36.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:36 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:32:36.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:36 smithi097 ceph-mon[39416]: osdmap e280: 8 total, 8 up, 8 in 2024-03-21T13:32:36.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:36 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2834570801' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi097-79636-54"}]: dispatch 2024-03-21T13:32:36.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:36 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi097-79636-54"}]: dispatch 2024-03-21T13:32:36.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:36 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:32:36.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:36 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:32:36.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:36 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi097-79636-54"}]': finished 2024-03-21T13:32:36.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:36 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:32:36.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:36 smithi097 ceph-mon[30040]: osdmap e280: 8 total, 8 up, 8 in 2024-03-21T13:32:36.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:36 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2834570801' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi097-79636-54"}]: dispatch 2024-03-21T13:32:36.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:36 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi097-79636-54"}]: dispatch 2024-03-21T13:32:36.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:36 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:32:36.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:36 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:32:37.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:37 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi097-79636-54"}]': finished 2024-03-21T13:32:37.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:37 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-39", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:32:37.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:37 smithi144 ceph-mon[33633]: osdmap e281: 8 total, 8 up, 8 in 2024-03-21T13:32:37.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:37 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-39"}]: dispatch 2024-03-21T13:32:37.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:37 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-39"}]: dispatch 2024-03-21T13:32:37.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:37 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1585712526' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi097-79636-55"}]: dispatch 2024-03-21T13:32:37.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:37 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi097-79636-55"}]: dispatch 2024-03-21T13:32:37.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:37 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1585712526' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi097-79636-55"}]: dispatch 2024-03-21T13:32:37.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:37 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi097-79636-55"}]: dispatch 2024-03-21T13:32:37.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:37 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1585712526' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi097-79636-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:37.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:37 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi097-79636-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:37.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:37 smithi144 ceph-mon[33633]: pgmap v386: 324 pgs: 32 unknown, 292 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:32:37.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:37 smithi144 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:32:37.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:37 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi097-79636-54"}]': finished 2024-03-21T13:32:37.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:37 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-39", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:32:37.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:37 smithi097 ceph-mon[39416]: osdmap e281: 8 total, 8 up, 8 in 2024-03-21T13:32:37.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:37 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-39"}]: dispatch 2024-03-21T13:32:37.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:37 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-39"}]: dispatch 2024-03-21T13:32:37.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:37 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1585712526' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi097-79636-55"}]: dispatch 2024-03-21T13:32:37.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:37 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi097-79636-55"}]: dispatch 2024-03-21T13:32:37.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:37 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1585712526' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi097-79636-55"}]: dispatch 2024-03-21T13:32:37.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:37 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi097-79636-55"}]: dispatch 2024-03-21T13:32:37.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:37 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1585712526' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi097-79636-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:37.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:37 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi097-79636-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:37.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:37 smithi097 ceph-mon[39416]: pgmap v386: 324 pgs: 32 unknown, 292 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:32:37.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:37 smithi097 ceph-mon[39416]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:32:37.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:37 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi097-79636-54"}]': finished 2024-03-21T13:32:37.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:37 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-39", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:32:37.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:37 smithi097 ceph-mon[30040]: osdmap e281: 8 total, 8 up, 8 in 2024-03-21T13:32:37.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:37 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-39"}]: dispatch 2024-03-21T13:32:37.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:37 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-39"}]: dispatch 2024-03-21T13:32:37.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:37 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1585712526' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi097-79636-55"}]: dispatch 2024-03-21T13:32:37.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:37 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi097-79636-55"}]: dispatch 2024-03-21T13:32:37.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:37 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1585712526' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi097-79636-55"}]: dispatch 2024-03-21T13:32:37.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:37 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi097-79636-55"}]: dispatch 2024-03-21T13:32:37.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:37 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1585712526' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi097-79636-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:37.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:37 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi097-79636-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:37.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:37 smithi097 ceph-mon[30040]: pgmap v386: 324 pgs: 32 unknown, 292 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:32:37.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:37 smithi097 ceph-mon[30040]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:32:38.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:38 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-39"}]': finished 2024-03-21T13:32:38.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:38 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi097-79636-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:32:38.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:38 smithi144 ceph-mon[33633]: osdmap e282: 8 total, 8 up, 8 in 2024-03-21T13:32:38.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:38 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-39", "mode": "writeback"}]: dispatch 2024-03-21T13:32:38.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:38 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1585712526' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi097-79636-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi097-79636-55"}]: dispatch 2024-03-21T13:32:38.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:38 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-39", "mode": "writeback"}]: dispatch 2024-03-21T13:32:38.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:38 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi097-79636-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi097-79636-55"}]: dispatch 2024-03-21T13:32:39.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:38 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-39"}]': finished 2024-03-21T13:32:39.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:38 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi097-79636-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:32:39.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:38 smithi097 ceph-mon[39416]: osdmap e282: 8 total, 8 up, 8 in 2024-03-21T13:32:39.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:38 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-39", "mode": "writeback"}]: dispatch 2024-03-21T13:32:39.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:38 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1585712526' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi097-79636-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi097-79636-55"}]: dispatch 2024-03-21T13:32:39.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:38 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-39", "mode": "writeback"}]: dispatch 2024-03-21T13:32:39.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:38 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi097-79636-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi097-79636-55"}]: dispatch 2024-03-21T13:32:39.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:38 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-39"}]': finished 2024-03-21T13:32:39.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:38 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi097-79636-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:32:39.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:38 smithi097 ceph-mon[30040]: osdmap e282: 8 total, 8 up, 8 in 2024-03-21T13:32:39.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:38 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-39", "mode": "writeback"}]: dispatch 2024-03-21T13:32:39.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:38 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1585712526' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi097-79636-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi097-79636-55"}]: dispatch 2024-03-21T13:32:39.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:38 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-39", "mode": "writeback"}]: dispatch 2024-03-21T13:32:39.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:38 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi097-79636-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi097-79636-55"}]: dispatch 2024-03-21T13:32:39.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:39 smithi144 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:32:39.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:39 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-39", "mode": "writeback"}]': finished 2024-03-21T13:32:39.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:39 smithi144 ceph-mon[33633]: pgmap v389: 324 pgs: 324 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:32:39.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:39 smithi144 ceph-mon[33633]: osdmap e283: 8 total, 8 up, 8 in 2024-03-21T13:32:39.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:39 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:32:39.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:39 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:32:39.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:39 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi097-79636-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi097-79636-55"}]': finished 2024-03-21T13:32:39.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:39 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:32:39.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:39 smithi144 ceph-mon[33633]: osdmap e284: 8 total, 8 up, 8 in 2024-03-21T13:32:39.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:39 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-39"}]: dispatch 2024-03-21T13:32:39.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:39 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-39"}]: dispatch 2024-03-21T13:32:40.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:39 smithi097 ceph-mon[39416]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:32:40.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:39 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-39", "mode": "writeback"}]': finished 2024-03-21T13:32:40.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:39 smithi097 ceph-mon[39416]: pgmap v389: 324 pgs: 324 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:32:40.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:39 smithi097 ceph-mon[39416]: osdmap e283: 8 total, 8 up, 8 in 2024-03-21T13:32:40.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:39 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:32:40.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:39 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:32:40.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:39 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi097-79636-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi097-79636-55"}]': finished 2024-03-21T13:32:40.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:39 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:32:40.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:39 smithi097 ceph-mon[39416]: osdmap e284: 8 total, 8 up, 8 in 2024-03-21T13:32:40.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:39 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-39"}]: dispatch 2024-03-21T13:32:40.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:39 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-39"}]: dispatch 2024-03-21T13:32:40.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:39 smithi097 ceph-mon[30040]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:32:40.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:39 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-39", "mode": "writeback"}]': finished 2024-03-21T13:32:40.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:39 smithi097 ceph-mon[30040]: pgmap v389: 324 pgs: 324 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:32:40.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:39 smithi097 ceph-mon[30040]: osdmap e283: 8 total, 8 up, 8 in 2024-03-21T13:32:40.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:39 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:32:40.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:39 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:32:40.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:39 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi097-79636-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi097-79636-55"}]': finished 2024-03-21T13:32:40.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:39 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:32:40.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:39 smithi097 ceph-mon[30040]: osdmap e284: 8 total, 8 up, 8 in 2024-03-21T13:32:40.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:39 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-39"}]: dispatch 2024-03-21T13:32:40.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:39 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-39"}]: dispatch 2024-03-21T13:32:41.344 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [==========] Running 77 tests from 4 test suites. 2024-03-21T13:32:41.344 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [----------] Global test environment set-up. 2024-03-21T13:32:41.344 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP 2024-03-21T13:32:41.344 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: seed 79906 2024-03-21T13:32:41.344 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.Dirty 2024-03-21T13:32:41.344 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTierPP.Dirty (415 ms) 2024-03-21T13:32:41.345 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.FlushWriteRaces 2024-03-21T13:32:41.345 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTierPP.FlushWriteRaces (16433 ms) 2024-03-21T13:32:41.345 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.HitSetNone 2024-03-21T13:32:41.345 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTierPP.HitSetNone (46 ms) 2024-03-21T13:32:41.345 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP (16894 ms total) 2024-03-21T13:32:41.345 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: 2024-03-21T13:32:41.345 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP 2024-03-21T13:32:41.345 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Overlay 2024-03-21T13:32:41.345 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Overlay (7069 ms) 2024-03-21T13:32:41.345 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Promote 2024-03-21T13:32:41.345 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Promote (8001 ms) 2024-03-21T13:32:41.345 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnap 2024-03-21T13:32:41.345 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnap (9998 ms) 2024-03-21T13:32:41.346 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapScrub 2024-03-21T13:32:41.346 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: my_snaps [3] 2024-03-21T13:32:41.346 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: my_snaps [4,3] 2024-03-21T13:32:41.346 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: my_snaps [5,4,3] 2024-03-21T13:32:41.346 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: my_snaps [6,5,4,3] 2024-03-21T13:32:41.346 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: promoting some heads 2024-03-21T13:32:41.346 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: promoting from clones for snap 6 2024-03-21T13:32:41.346 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: promoting from clones for snap 5 2024-03-21T13:32:41.346 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: promoting from clones for snap 4 2024-03-21T13:32:41.346 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: promoting from clones for snap 3 2024-03-21T13:32:41.346 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: waiting for scrubs... 2024-03-21T13:32:41.346 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: done waiting 2024-03-21T13:32:41.346 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapScrub (46048 ms) 2024-03-21T13:32:41.346 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapTrimRace 2024-03-21T13:32:41.347 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapTrimRace (9994 ms) 2024-03-21T13:32:41.347 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Whiteout 2024-03-21T13:32:41.347 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Whiteout (7992 ms) 2024-03-21T13:32:41.347 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate 2024-03-21T13:32:41.347 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate (8039 ms) 2024-03-21T13:32:41.347 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Evict 2024-03-21T13:32:41.347 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Evict (8034 ms) 2024-03-21T13:32:41.347 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap 2024-03-21T13:32:41.347 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap (10051 ms) 2024-03-21T13:32:41.347 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap2 2024-03-21T13:32:41.347 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap2 (8869 ms) 2024-03-21T13:32:41.347 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ListSnap 2024-03-21T13:32:41.347 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ListSnap (10073 ms) 2024-03-21T13:32:41.347 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace 2024-03-21T13:32:41.348 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace (12974 ms) 2024-03-21T13:32:41.348 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlush 2024-03-21T13:32:41.348 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlush (7952 ms) 2024-03-21T13:32:41.348 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Flush 2024-03-21T13:32:41.348 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Flush (8294 ms) 2024-03-21T13:32:41.348 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushSnap 2024-03-21T13:32:41.348 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushSnap (12841 ms) 2024-03-21T13:32:41.348 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushTryFlushRaces 2024-03-21T13:32:41.348 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushTryFlushRaces (7904 ms) 2024-03-21T13:32:41.348 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlushReadRace 2024-03-21T13:32:41.348 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlushReadRace (8037 ms) 2024-03-21T13:32:41.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:41 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:32:41.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:41 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:32:41.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:41 smithi097 ceph-mon[30040]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:32:41.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:41 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-39"}]': finished 2024-03-21T13:32:41.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:41 smithi097 ceph-mon[30040]: osdmap e285: 8 total, 8 up, 8 in 2024-03-21T13:32:41.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:41 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:32:41.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:41 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:32:41.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:41 smithi097 ceph-mon[39416]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:32:41.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:41 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-39"}]': finished 2024-03-21T13:32:41.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:41 smithi097 ceph-mon[39416]: osdmap e285: 8 total, 8 up, 8 in 2024-03-21T13:32:41.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:41 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:32:41.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:41 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:32:41.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:41 smithi144 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:32:41.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:41 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-39"}]': finished 2024-03-21T13:32:41.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:41 smithi144 ceph-mon[33633]: osdmap e285: 8 total, 8 up, 8 in 2024-03-21T13:32:42.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:32:41 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:32:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:32:42.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:42 smithi097 ceph-mon[30040]: pgmap v392: 332 pgs: 8 unknown, 324 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:32:42.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:42 smithi097 ceph-mon[30040]: osdmap e286: 8 total, 8 up, 8 in 2024-03-21T13:32:42.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:42 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1585712526' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi097-79636-55"}]: dispatch 2024-03-21T13:32:42.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:42 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi097-79636-55"}]: dispatch 2024-03-21T13:32:42.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:42 smithi097 ceph-mon[39416]: pgmap v392: 332 pgs: 8 unknown, 324 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:32:42.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:42 smithi097 ceph-mon[39416]: osdmap e286: 8 total, 8 up, 8 in 2024-03-21T13:32:42.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:42 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1585712526' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi097-79636-55"}]: dispatch 2024-03-21T13:32:42.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:42 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi097-79636-55"}]: dispatch 2024-03-21T13:32:42.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:42 smithi144 ceph-mon[33633]: pgmap v392: 332 pgs: 8 unknown, 324 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:32:42.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:42 smithi144 ceph-mon[33633]: osdmap e286: 8 total, 8 up, 8 in 2024-03-21T13:32:42.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:42 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1585712526' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi097-79636-55"}]: dispatch 2024-03-21T13:32:42.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:42 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi097-79636-55"}]: dispatch 2024-03-21T13:32:43.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:43 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi097-79636-55"}]': finished 2024-03-21T13:32:43.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:43 smithi144 ceph-mon[33633]: osdmap e287: 8 total, 8 up, 8 in 2024-03-21T13:32:43.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:43 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1585712526' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi097-79636-55"}]: dispatch 2024-03-21T13:32:43.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:43 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:32:43.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:43 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi097-79636-55"}]: dispatch 2024-03-21T13:32:43.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:43 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:32:43.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:43 smithi144 ceph-mon[33633]: pgmap v395: 324 pgs: 32 unknown, 292 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:32:43.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:43 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi097-79636-55"}]': finished 2024-03-21T13:32:43.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:43 smithi097 ceph-mon[30040]: osdmap e287: 8 total, 8 up, 8 in 2024-03-21T13:32:43.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:43 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1585712526' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi097-79636-55"}]: dispatch 2024-03-21T13:32:43.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:43 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:32:43.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:43 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi097-79636-55"}]: dispatch 2024-03-21T13:32:43.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:43 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:32:43.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:43 smithi097 ceph-mon[30040]: pgmap v395: 324 pgs: 32 unknown, 292 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:32:43.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:43 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi097-79636-55"}]': finished 2024-03-21T13:32:43.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:43 smithi097 ceph-mon[39416]: osdmap e287: 8 total, 8 up, 8 in 2024-03-21T13:32:43.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:43 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1585712526' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi097-79636-55"}]: dispatch 2024-03-21T13:32:43.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:43 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:32:43.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:43 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi097-79636-55"}]: dispatch 2024-03-21T13:32:43.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:43 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:32:43.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:43 smithi097 ceph-mon[39416]: pgmap v395: 324 pgs: 32 unknown, 292 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:32:43.871 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:32:43 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:32:44.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:44 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi097-79636-55"}]': finished 2024-03-21T13:32:44.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:44 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:32:44.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:44 smithi144 ceph-mon[33633]: osdmap e288: 8 total, 8 up, 8 in 2024-03-21T13:32:44.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:44 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:32:44.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:44 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:32:44.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:44 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3836142937' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi097-79636-56"}]: dispatch 2024-03-21T13:32:44.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:44 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi097-79636-56"}]: dispatch 2024-03-21T13:32:44.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:44 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3836142937' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi097-79636-56"}]: dispatch 2024-03-21T13:32:44.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:44 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi097-79636-56"}]: dispatch 2024-03-21T13:32:44.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:44 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3836142937' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi097-79636-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:44.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:44 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi097-79636-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:44.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:44 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:32:44.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:44 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi097-79636-55"}]': finished 2024-03-21T13:32:44.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:44 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:32:44.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:44 smithi097 ceph-mon[39416]: osdmap e288: 8 total, 8 up, 8 in 2024-03-21T13:32:44.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:44 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:32:44.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:44 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:32:44.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:44 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3836142937' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi097-79636-56"}]: dispatch 2024-03-21T13:32:44.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:44 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi097-79636-56"}]: dispatch 2024-03-21T13:32:44.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:44 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3836142937' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi097-79636-56"}]: dispatch 2024-03-21T13:32:44.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:44 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi097-79636-56"}]: dispatch 2024-03-21T13:32:44.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:44 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3836142937' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi097-79636-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:44.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:44 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi097-79636-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:44.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:44 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:32:44.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:44 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi097-79636-55"}]': finished 2024-03-21T13:32:44.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:44 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:32:44.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:44 smithi097 ceph-mon[30040]: osdmap e288: 8 total, 8 up, 8 in 2024-03-21T13:32:44.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:44 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:32:44.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:44 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:32:44.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:44 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3836142937' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi097-79636-56"}]: dispatch 2024-03-21T13:32:44.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:44 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi097-79636-56"}]: dispatch 2024-03-21T13:32:44.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:44 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3836142937' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi097-79636-56"}]: dispatch 2024-03-21T13:32:44.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:44 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi097-79636-56"}]: dispatch 2024-03-21T13:32:44.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:44 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3836142937' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi097-79636-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:44.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:44 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi097-79636-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:44.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:44 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:32:45.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:45 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-41", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:32:45.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:45 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi097-79636-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:32:45.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:45 smithi144 ceph-mon[33633]: osdmap e289: 8 total, 8 up, 8 in 2024-03-21T13:32:45.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:45 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-41","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:32:45.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:45 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-41","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:32:45.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:45 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3836142937' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi097-79636-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi097-79636-56"}]: dispatch 2024-03-21T13:32:45.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:45 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi097-79636-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi097-79636-56"}]: dispatch 2024-03-21T13:32:45.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:45 smithi144 ceph-mon[33633]: pgmap v398: 324 pgs: 1 active, 10 creating+activating, 16 creating+peering, 297 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-03-21T13:32:45.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:45 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-41","var": "hit_set_count","val": "2"}]': finished 2024-03-21T13:32:45.888 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetrted 0x55ca2a5bba70 2024-03-21T13:32:45.888 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bb5c0 2024-03-21T13:32:45.888 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bb710 2024-03-21T13:32:45.888 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bb820 2024-03-21T13:32:45.888 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bb950 2024-03-21T13:32:45.888 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bc070 2024-03-21T13:32:45.889 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bbbc0 2024-03-21T13:32:45.889 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bbd10 2024-03-21T13:32:45.889 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bbe20 2024-03-21T13:32:45.889 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bc1a0 2024-03-21T13:32:45.889 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bc670 2024-03-21T13:32:45.889 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bc7c0 2024-03-21T13:32:45.889 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bcda0 2024-03-21T13:32:45.889 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bc2b0 2024-03-21T13:32:45.889 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bc400 2024-03-21T13:32:45.889 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bc530 2024-03-21T13:32:45.889 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bbf50 2024-03-21T13:32:45.889 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bc8f0 2024-03-21T13:32:45.890 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bca60 2024-03-21T13:32:45.890 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bcb70 2024-03-21T13:32:45.890 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bced0 2024-03-21T13:32:45.890 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bd3a0 2024-03-21T13:32:45.890 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bd4f0 2024-03-21T13:32:45.890 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bcfe0 2024-03-21T13:32:45.890 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bd0f0 2024-03-21T13:32:45.890 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bd240 2024-03-21T13:32:45.890 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bd640 2024-03-21T13:32:45.890 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bdb10 2024-03-21T13:32:45.890 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bdc40 2024-03-21T13:32:45.890 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bd750 2024-03-21T13:32:45.891 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bd8a0 2024-03-21T13:32:45.891 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bd9f0 2024-03-21T13:32:45.891 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bdd50 2024-03-21T13:32:45.891 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5be220 2024-03-21T13:32:45.891 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5be370 2024-03-21T13:32:45.891 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b8780 2024-03-21T13:32:45.891 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5b88d0 2024-03-21T13:32:45.891 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5be930 2024-03-21T13:32:45.891 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5be480 2024-03-21T13:32:45.891 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5be5d0 2024-03-21T13:32:45.891 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5be720 2024-03-21T13:32:45.891 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bea60 2024-03-21T13:32:45.891 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bef30 2024-03-21T13:32:45.892 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bf040 2024-03-21T13:32:45.892 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5beb70 2024-03-21T13:32:45.892 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5beca0 2024-03-21T13:32:45.892 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bedd0 2024-03-21T13:32:45.892 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bf150 2024-03-21T13:32:45.892 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bf620 2024-03-21T13:32:45.892 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bf750 2024-03-21T13:32:45.892 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bde60 2024-03-21T13:32:45.892 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bdf90 2024-03-21T13:32:45.892 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5be0c0 2024-03-21T13:32:45.892 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bf880 2024-03-21T13:32:45.892 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bfd30 2024-03-21T13:32:45.892 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bfe80 2024-03-21T13:32:45.893 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c0480 2024-03-21T13:32:45.893 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bf990 2024-03-21T13:32:45.893 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bfae0 2024-03-21T13:32:45.893 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bfbf0 2024-03-21T13:32:45.893 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c05b0 2024-03-21T13:32:45.893 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bffd0 2024-03-21T13:32:45.893 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c0100 2024-03-21T13:32:45.893 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c0210 2024-03-21T13:32:45.893 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c0360 2024-03-21T13:32:45.893 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c0b90 2024-03-21T13:32:45.893 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c06e0 2024-03-21T13:32:45.893 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c0810 2024-03-21T13:32:45.894 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c0920 2024-03-21T13:32:45.894 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c0a70 2024-03-21T13:32:45.894 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c1190 2024-03-21T13:32:45.894 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c0ce0 2024-03-21T13:32:45.894 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c0e10 2024-03-21T13:32:45.894 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c0f20 2024-03-21T13:32:45.894 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c1070 2024-03-21T13:32:45.894 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c1790 2024-03-21T13:32:45.894 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c12e0 2024-03-21T13:32:45.894 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c1430 2024-03-21T13:32:45.894 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c1540 2024-03-21T13:32:45.895 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c1670 2024-03-21T13:32:45.895 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c1d90 2024-03-21T13:32:45.895 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c18e0 2024-03-21T13:32:46.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:45 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-41", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:32:46.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:45 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi097-79636-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:32:46.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:45 smithi097 ceph-mon[30040]: osdmap e289: 8 total, 8 up, 8 in 2024-03-21T13:32:46.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:45 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-41","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:32:46.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:45 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-41","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:32:46.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:45 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3836142937' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi097-79636-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi097-79636-56"}]: dispatch 2024-03-21T13:32:46.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:45 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi097-79636-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi097-79636-56"}]: dispatch 2024-03-21T13:32:46.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:45 smithi097 ceph-mon[30040]: pgmap v398: 324 pgs: 1 active, 10 creating+activating, 16 creating+peering, 297 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-03-21T13:32:46.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:45 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-41","var": "hit_set_count","val": "2"}]': finished 2024-03-21T13:32:46.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:45 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-41", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:32:46.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:45 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi097-79636-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:32:46.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:45 smithi097 ceph-mon[39416]: osdmap e289: 8 total, 8 up, 8 in 2024-03-21T13:32:46.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:45 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-41","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:32:46.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:45 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-41","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:32:46.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:45 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3836142937' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi097-79636-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi097-79636-56"}]: dispatch 2024-03-21T13:32:46.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:45 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi097-79636-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi097-79636-56"}]: dispatch 2024-03-21T13:32:46.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:45 smithi097 ceph-mon[39416]: pgmap v398: 324 pgs: 1 active, 10 creating+activating, 16 creating+peering, 297 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-03-21T13:32:46.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:45 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-41","var": "hit_set_count","val": "2"}]': finished 2024-03-21T13:32:46.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:46 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-41","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:32:46.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:46 smithi144 ceph-mon[33633]: osdmap e290: 8 total, 8 up, 8 in 2024-03-21T13:32:46.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:46 smithi144 ceph-mon[33633]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:32:46.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:46 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-41","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:32:46.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:46 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi097-79636-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi097-79636-56"}]': finished 2024-03-21T13:32:46.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:46 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-41","var": "hit_set_period","val": "600"}]': finished 2024-03-21T13:32:46.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:46 smithi144 ceph-mon[33633]: osdmap e291: 8 total, 8 up, 8 in 2024-03-21T13:32:46.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:46 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-21T13:32:46.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:46 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-21T13:32:47.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:46 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-41","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:32:47.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:46 smithi097 ceph-mon[39416]: osdmap e290: 8 total, 8 up, 8 in 2024-03-21T13:32:47.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:46 smithi097 ceph-mon[39416]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:32:47.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:46 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-41","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:32:47.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:46 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi097-79636-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi097-79636-56"}]': finished 2024-03-21T13:32:47.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:46 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-41","var": "hit_set_period","val": "600"}]': finished 2024-03-21T13:32:47.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:46 smithi097 ceph-mon[39416]: osdmap e291: 8 total, 8 up, 8 in 2024-03-21T13:32:47.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:46 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-21T13:32:47.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:46 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-21T13:32:47.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:46 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-41","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:32:47.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:46 smithi097 ceph-mon[30040]: osdmap e290: 8 total, 8 up, 8 in 2024-03-21T13:32:47.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:46 smithi097 ceph-mon[30040]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:32:47.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:46 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-41","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:32:47.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:46 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi097-79636-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi097-79636-56"}]': finished 2024-03-21T13:32:47.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:46 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-41","var": "hit_set_period","val": "600"}]': finished 2024-03-21T13:32:47.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:46 smithi097 ceph-mon[30040]: osdmap e291: 8 total, 8 up, 8 in 2024-03-21T13:32:47.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:46 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-21T13:32:47.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:46 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-21T13:32:47.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:47 smithi144 ceph-mon[33633]: pgmap v401: 332 pgs: 8 unknown, 1 active, 10 creating+activating, 16 creating+peering, 297 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-03-21T13:32:47.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:47 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-21T13:32:47.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:47 smithi144 ceph-mon[33633]: osdmap e292: 8 total, 8 up, 8 in 2024-03-21T13:32:48.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:47 smithi097 ceph-mon[30040]: pgmap v401: 332 pgs: 8 unknown, 1 active, 10 creating+activating, 16 creating+peering, 297 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-03-21T13:32:48.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:47 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-21T13:32:48.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:47 smithi097 ceph-mon[30040]: osdmap e292: 8 total, 8 up, 8 in 2024-03-21T13:32:48.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:47 smithi097 ceph-mon[39416]: pgmap v401: 332 pgs: 8 unknown, 1 active, 10 creating+activating, 16 creating+peering, 297 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-03-21T13:32:48.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:47 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-21T13:32:48.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:47 smithi097 ceph-mon[39416]: osdmap e292: 8 total, 8 up, 8 in 2024-03-21T13:32:49.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:49 smithi144 ceph-mon[33633]: osdmap e293: 8 total, 8 up, 8 in 2024-03-21T13:32:49.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:49 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3836142937' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi097-79636-56"}]: dispatch 2024-03-21T13:32:49.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:49 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi097-79636-56"}]: dispatch 2024-03-21T13:32:49.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:49 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:32:49.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:49 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:32:49.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:49 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-41"}]: dispatch 2024-03-21T13:32:49.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:49 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-41"}]: dispatch 2024-03-21T13:32:49.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:49 smithi144 ceph-mon[33633]: pgmap v404: 324 pgs: 324 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:32:49.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:49 smithi097 ceph-mon[30040]: osdmap e293: 8 total, 8 up, 8 in 2024-03-21T13:32:49.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:49 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3836142937' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi097-79636-56"}]: dispatch 2024-03-21T13:32:49.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:49 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi097-79636-56"}]: dispatch 2024-03-21T13:32:49.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:49 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:32:49.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:49 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:32:49.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:49 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-41"}]: dispatch 2024-03-21T13:32:49.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:49 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-41"}]: dispatch 2024-03-21T13:32:49.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:49 smithi097 ceph-mon[30040]: pgmap v404: 324 pgs: 324 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:32:49.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:49 smithi097 ceph-mon[39416]: osdmap e293: 8 total, 8 up, 8 in 2024-03-21T13:32:49.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:49 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3836142937' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi097-79636-56"}]: dispatch 2024-03-21T13:32:49.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:49 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi097-79636-56"}]: dispatch 2024-03-21T13:32:49.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:49 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:32:49.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:49 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:32:49.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:49 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-41"}]: dispatch 2024-03-21T13:32:49.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:49 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-41"}]: dispatch 2024-03-21T13:32:49.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:49 smithi097 ceph-mon[39416]: pgmap v404: 324 pgs: 324 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:32:50.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:50 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi097-79636-56"}]': finished 2024-03-21T13:32:50.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:50 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-41"}]': finished 2024-03-21T13:32:50.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:50 smithi097 ceph-mon[39416]: osdmap e294: 8 total, 8 up, 8 in 2024-03-21T13:32:50.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:50 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3836142937' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi097-79636-56"}]: dispatch 2024-03-21T13:32:50.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:50 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi097-79636-56"}]: dispatch 2024-03-21T13:32:50.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:50 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi097-79636-56"}]': finished 2024-03-21T13:32:50.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:50 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-41"}]': finished 2024-03-21T13:32:50.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:50 smithi097 ceph-mon[30040]: osdmap e294: 8 total, 8 up, 8 in 2024-03-21T13:32:50.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:50 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3836142937' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi097-79636-56"}]: dispatch 2024-03-21T13:32:50.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:50 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi097-79636-56"}]: dispatch 2024-03-21T13:32:50.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:50 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi097-79636-56"}]': finished 2024-03-21T13:32:50.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:50 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-41"}]': finished 2024-03-21T13:32:50.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:50 smithi144 ceph-mon[33633]: osdmap e294: 8 total, 8 up, 8 in 2024-03-21T13:32:50.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:50 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3836142937' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi097-79636-56"}]: dispatch 2024-03-21T13:32:50.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:50 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi097-79636-56"}]: dispatch 2024-03-21T13:32:51.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:51 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi097-79636-56"}]': finished 2024-03-21T13:32:51.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:51 smithi097 ceph-mon[39416]: osdmap e295: 8 total, 8 up, 8 in 2024-03-21T13:32:51.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:51 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1802561187' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi097-79636-57"}]: dispatch 2024-03-21T13:32:51.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:51 smithi097 ceph-mon[39416]: pgmap v407: 292 pgs: 292 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:32:51.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:51 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi097-79636-57"}]: dispatch 2024-03-21T13:32:51.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:51 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1802561187' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi097-79636-57"}]: dispatch 2024-03-21T13:32:51.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:51 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi097-79636-57"}]: dispatch 2024-03-21T13:32:51.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:51 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1802561187' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi097-79636-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:51.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:51 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi097-79636-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:51.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:51 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi097-79636-56"}]': finished 2024-03-21T13:32:51.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:51 smithi097 ceph-mon[30040]: osdmap e295: 8 total, 8 up, 8 in 2024-03-21T13:32:51.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:51 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1802561187' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi097-79636-57"}]: dispatch 2024-03-21T13:32:51.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:51 smithi097 ceph-mon[30040]: pgmap v407: 292 pgs: 292 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:32:51.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:51 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi097-79636-57"}]: dispatch 2024-03-21T13:32:51.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:51 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1802561187' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi097-79636-57"}]: dispatch 2024-03-21T13:32:51.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:51 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi097-79636-57"}]: dispatch 2024-03-21T13:32:51.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:51 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1802561187' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi097-79636-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:51.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:51 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi097-79636-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:51.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:51 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi097-79636-56"}]': finished 2024-03-21T13:32:51.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:51 smithi144 ceph-mon[33633]: osdmap e295: 8 total, 8 up, 8 in 2024-03-21T13:32:51.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:51 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1802561187' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi097-79636-57"}]: dispatch 2024-03-21T13:32:51.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:51 smithi144 ceph-mon[33633]: pgmap v407: 292 pgs: 292 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:32:51.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:51 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi097-79636-57"}]: dispatch 2024-03-21T13:32:51.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:51 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1802561187' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi097-79636-57"}]: dispatch 2024-03-21T13:32:51.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:51 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi097-79636-57"}]: dispatch 2024-03-21T13:32:51.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:51 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1802561187' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi097-79636-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:51.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:51 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi097-79636-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:52.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:32:51 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:32:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:32:53.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:52 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi097-79636-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:32:53.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:52 smithi097 ceph-mon[39416]: osdmap e296: 8 total, 8 up, 8 in 2024-03-21T13:32:53.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:52 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1802561187' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi097-79636-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi097-79636-57"}]: dispatch 2024-03-21T13:32:53.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:52 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:32:53.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:52 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi097-79636-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi097-79636-57"}]: dispatch 2024-03-21T13:32:53.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:52 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:32:53.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:52 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi097-79636-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:32:53.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:52 smithi097 ceph-mon[30040]: osdmap e296: 8 total, 8 up, 8 in 2024-03-21T13:32:53.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:52 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1802561187' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi097-79636-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi097-79636-57"}]: dispatch 2024-03-21T13:32:53.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:52 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:32:53.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:52 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi097-79636-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi097-79636-57"}]: dispatch 2024-03-21T13:32:53.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:52 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:32:53.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:52 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi097-79636-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:32:53.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:52 smithi144 ceph-mon[33633]: osdmap e296: 8 total, 8 up, 8 in 2024-03-21T13:32:53.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:52 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1802561187' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi097-79636-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi097-79636-57"}]: dispatch 2024-03-21T13:32:53.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:52 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:32:53.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:52 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi097-79636-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi097-79636-57"}]: dispatch 2024-03-21T13:32:53.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:52 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:32:53.871 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:32:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:32:53.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:53 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:32:53.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:53 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi097-79906-6","var": "pg_num","format": "json"}]: dispatch 2024-03-21T13:32:53.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:53 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:32:53.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:53 smithi144 ceph-mon[33633]: pgmap v410: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:32:53.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:53 smithi144 ceph-mon[33633]: osdmap e297: 8 total, 8 up, 8 in 2024-03-21T13:32:53.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:53 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:32:53.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:53 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi097-79636-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi097-79636-57"}]': finished 2024-03-21T13:32:53.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:53 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-43", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:32:53.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:53 smithi144 ceph-mon[33633]: osdmap e298: 8 total, 8 up, 8 in 2024-03-21T13:32:53.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:53 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-43","var": "hit_set_count","val": "8"}]: dispatch 2024-03-21T13:32:53.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:53 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-43","var": "hit_set_count","val": "8"}]: dispatch 2024-03-21T13:32:54.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:53 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:32:54.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:53 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi097-79906-6","var": "pg_num","format": "json"}]: dispatch 2024-03-21T13:32:54.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:53 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:32:54.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:53 smithi097 ceph-mon[39416]: pgmap v410: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:32:54.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:53 smithi097 ceph-mon[39416]: osdmap e297: 8 total, 8 up, 8 in 2024-03-21T13:32:54.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:53 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:32:54.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:53 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi097-79636-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi097-79636-57"}]': finished 2024-03-21T13:32:54.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:53 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-43", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:32:54.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:53 smithi097 ceph-mon[39416]: osdmap e298: 8 total, 8 up, 8 in 2024-03-21T13:32:54.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:53 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-43","var": "hit_set_count","val": "8"}]: dispatch 2024-03-21T13:32:54.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:53 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-43","var": "hit_set_count","val": "8"}]: dispatch 2024-03-21T13:32:54.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:53 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:32:54.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:53 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi097-79906-6","var": "pg_num","format": "json"}]: dispatch 2024-03-21T13:32:54.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:53 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:32:54.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:53 smithi097 ceph-mon[30040]: pgmap v410: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:32:54.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:53 smithi097 ceph-mon[30040]: osdmap e297: 8 total, 8 up, 8 in 2024-03-21T13:32:54.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:53 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:32:54.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:53 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi097-79636-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi097-79636-57"}]': finished 2024-03-21T13:32:54.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:53 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-43", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:32:54.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:53 smithi097 ceph-mon[30040]: osdmap e298: 8 total, 8 up, 8 in 2024-03-21T13:32:54.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:53 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-43","var": "hit_set_count","val": "8"}]: dispatch 2024-03-21T13:32:54.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:53 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-43","var": "hit_set_count","val": "8"}]: dispatch 2024-03-21T13:32:55.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:54 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:32:55.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:54 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-43","var": "hit_set_count","val": "8"}]': finished 2024-03-21T13:32:55.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:54 smithi097 ceph-mon[39416]: osdmap e299: 8 total, 8 up, 8 in 2024-03-21T13:32:55.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:54 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-43","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:32:55.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:54 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-43","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:32:55.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:54 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:32:55.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:54 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-43","var": "hit_set_count","val": "8"}]': finished 2024-03-21T13:32:55.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:54 smithi097 ceph-mon[30040]: osdmap e299: 8 total, 8 up, 8 in 2024-03-21T13:32:55.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:54 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-43","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:32:55.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:54 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-43","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:32:55.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:54 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:32:55.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:54 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-43","var": "hit_set_count","val": "8"}]': finished 2024-03-21T13:32:55.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:54 smithi144 ceph-mon[33633]: osdmap e299: 8 total, 8 up, 8 in 2024-03-21T13:32:55.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:54 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-43","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:32:55.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:54 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-43","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:32:56.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:56 smithi097 ceph-mon[39416]: pgmap v413: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:32:56.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:56 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:32:56.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:56 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:32:56.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:56 smithi097 ceph-mon[39416]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:32:56.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:56 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-43","var": "hit_set_period","val": "600"}]': finished 2024-03-21T13:32:56.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:56 smithi097 ceph-mon[39416]: osdmap e300: 8 total, 8 up, 8 in 2024-03-21T13:32:56.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:56 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-03-21T13:32:56.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:56 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-03-21T13:32:56.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:56 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1802561187' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi097-79636-57"}]: dispatch 2024-03-21T13:32:56.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:56 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi097-79636-57"}]: dispatch 2024-03-21T13:32:56.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:56 smithi097 ceph-mon[30040]: pgmap v413: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:32:56.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:56 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:32:56.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:56 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:32:56.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:56 smithi097 ceph-mon[30040]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:32:56.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:56 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-43","var": "hit_set_period","val": "600"}]': finished 2024-03-21T13:32:56.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:56 smithi097 ceph-mon[30040]: osdmap e300: 8 total, 8 up, 8 in 2024-03-21T13:32:56.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:56 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-03-21T13:32:56.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:56 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-03-21T13:32:56.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:56 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1802561187' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi097-79636-57"}]: dispatch 2024-03-21T13:32:56.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:56 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi097-79636-57"}]: dispatch 2024-03-21T13:32:56.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:56 smithi144 ceph-mon[33633]: pgmap v413: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:32:56.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:56 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:32:56.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:56 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:32:56.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:56 smithi144 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:32:56.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:56 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-43","var": "hit_set_period","val": "600"}]': finished 2024-03-21T13:32:56.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:56 smithi144 ceph-mon[33633]: osdmap e300: 8 total, 8 up, 8 in 2024-03-21T13:32:56.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:56 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-03-21T13:32:56.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:56 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-03-21T13:32:56.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:56 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1802561187' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi097-79636-57"}]: dispatch 2024-03-21T13:32:56.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:56 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi097-79636-57"}]: dispatch 2024-03-21T13:32:56.656 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc:Read 2024-03-21T13:32:56.656 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: hmm, no HitSet yet 2024-03-21T13:32:56.656 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: ok, hit_set contains 268:602f83fe:::foo:head 2024-03-21T13:32:56.656 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetRead (9037 ms) 2024-03-21T13:32:56.656 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetWrite 2024-03-21T13:32:56.656 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: pg_num = 32 2024-03-21T13:32:56.656 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: pg 0 ls 1711027977,0 2024-03-21T13:32:56.656 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: pg 1 ls 1711027977,0 2024-03-21T13:32:56.656 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: pg 2 ls 1711027977,0 2024-03-21T13:32:56.656 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: pg 3 ls 1711027977,0 2024-03-21T13:32:56.656 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: pg 4 ls 1711027977,0 2024-03-21T13:32:56.656 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: pg 5 ls 1711027977,0 2024-03-21T13:32:56.656 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: pg 6 ls 1711027977,0 2024-03-21T13:32:56.657 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: pg 7 ls 1711027977,0 2024-03-21T13:32:56.657 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: pg 8 ls 1711027977,0 2024-03-21T13:32:56.657 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: pg 9 ls 1711027977,0 2024-03-21T13:32:56.657 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: pg 10 ls 1711027977,0 2024-03-21T13:32:56.657 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: pg 11 ls 1711027977,0 2024-03-21T13:32:56.657 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: pg 12 ls 1711027977,0 2024-03-21T13:32:56.657 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: pg 13 ls 1711027977,0 2024-03-21T13:32:56.657 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: pg 14 ls 1711027977,0 2024-03-21T13:32:56.657 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: pg 15 ls 1711027977,0 2024-03-21T13:32:56.657 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: pg 16 ls 1711027977,0 2024-03-21T13:32:56.657 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: pg 17 ls 1711027977,0 2024-03-21T13:32:56.657 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: pg 18 ls 1711027977,0 2024-03-21T13:32:56.657 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: pg 19 ls 1711027977,0 2024-03-21T13:32:56.657 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: pg 20 ls 1711027977,0 2024-03-21T13:32:56.657 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: pg 21 ls 1711027977,0 2024-03-21T13:32:56.658 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: pg 22 ls 1711027977,0 2024-03-21T13:32:56.658 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: pg 23 ls 1711027977,0 2024-03-21T13:32:56.658 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: pg 24 ls 1711027977,0 2024-03-21T13:32:56.658 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: pg 25 ls 1711027977,0 2024-03-21T13:32:56.658 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: pg 26 ls 1711027977,0 2024-03-21T13:32:56.658 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: pg 27 ls 1711027977,0 2024-03-21T13:32:56.658 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: pg 28 ls 1711027977,0 2024-03-21T13:32:56.658 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: pg 29 ls 1711027977,0 2024-03-21T13:32:56.658 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: pg 30 ls 1711027977,0 2024-03-21T13:32:56.658 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: pg 31 ls 1711027977,0 2024-03-21T13:32:56.658 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: pg_num = 32 2024-03-21T13:32:56.658 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:6cac518f:::0:head 2024-03-21T13:32:56.658 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:02547ec2:::1:head 2024-03-21T13:32:56.659 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:f905c69b:::2:head 2024-03-21T13:32:56.659 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:cfc208b3:::3:head 2024-03-21T13:32:56.659 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:d83876eb:::4:head 2024-03-21T13:32:56.659 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:b29083e3:::5:head 2024-03-21T13:32:56.659 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:c4fdafeb:::6:head 2024-03-21T13:32:56.659 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:5c6b0b28:::7:head 2024-03-21T13:32:56.659 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:bd63b0f1:::8:head 2024-03-21T13:32:56.659 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:e960b815:::9:head 2024-03-21T13:32:56.659 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:52ea6a34:::10:head 2024-03-21T13:32:56.659 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:89d3ae78:::11:head 2024-03-21T13:32:56.659 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:de5d7c5f:::12:head 2024-03-21T13:32:56.659 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:566253c9:::13:head 2024-03-21T13:32:56.659 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:62a1935d:::14:head 2024-03-21T13:32:56.659 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:863748b0:::15:head 2024-03-21T13:32:56.660 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:3958e169:::16:head 2024-03-21T13:32:56.660 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:4d4dabf9:::17:head 2024-03-21T13:32:56.660 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:8391935d:::18:head 2024-03-21T13:32:56.660 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:28883081:::19:head 2024-03-21T13:32:56.660 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:69259c59:::20:head 2024-03-21T13:32:56.660 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:4bdb80b7:::21:head 2024-03-21T13:32:56.660 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:a11c5d71:::22:head 2024-03-21T13:32:56.660 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:271af37b:::23:head 2024-03-21T13:32:56.660 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:95b121be:::24:head 2024-03-21T13:32:56.660 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:58d1031b:::25:head 2024-03-21T13:32:56.660 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:0a050783:::26:head 2024-03-21T13:32:56.660 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:c709704c:::27:head 2024-03-21T13:32:56.660 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:cbe56eaf:::28:head 2024-03-21T13:32:56.660 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:86b4b162:::29:head 2024-03-21T13:32:56.660 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:70d89383:::30:head 2024-03-21T13:32:56.661 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:dd450c7c:::31:head 2024-03-21T13:32:56.661 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:6d5729b1:::32:head 2024-03-21T13:32:56.661 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:c388f3fb:::33:head 2024-03-21T13:32:56.661 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:56cfea31:::34:head 2024-03-21T13:32:56.661 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:9dbc1bf7:::35:head 2024-03-21T13:32:56.661 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:40b74ccd:::36:head 2024-03-21T13:32:56.661 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:4d5aaf42:::37:head 2024-03-21T13:32:56.661 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:920f362c:::38:head 2024-03-21T13:32:56.661 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:6cc53222:::39:head 2024-03-21T13:32:56.661 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:9cad833f:::40:head 2024-03-21T13:32:56.661 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:1ea84d41:::41:head 2024-03-21T13:32:56.661 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:c4480ef6:::42:head 2024-03-21T13:32:56.661 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:a694361e:::43:head 2024-03-21T13:32:56.661 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:d1bd33e9:::44:head 2024-03-21T13:32:56.661 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:ddc2cd5d:::45:head 2024-03-21T13:32:56.662 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:2b782207:::46:head 2024-03-21T13:32:56.662 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:7b187fca:::47:head 2024-03-21T13:32:56.662 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:90ecdf6f:::48:head 2024-03-21T13:32:56.662 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:a5ed95fe:::49:head 2024-03-21T13:32:56.662 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:ea0eaa55:::50:head 2024-03-21T13:32:56.662 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:f33ef17b:::51:head 2024-03-21T13:32:56.662 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:a0d1b2f6:::52:head 2024-03-21T13:32:56.662 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:60c5229e:::53:head 2024-03-21T13:32:56.662 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:edcbc575:::54:head 2024-03-21T13:32:56.662 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:102cf253:::55:head 2024-03-21T13:32:56.662 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:efb7fb0b:::56:head 2024-03-21T13:32:56.662 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:50d0a326:::57:head 2024-03-21T13:32:56.662 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:d4dc5daf:::58:head 2024-03-21T13:32:56.662 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:3a130462:::59:head 2024-03-21T13:32:56.662 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:ec87ed71:::60:head 2024-03-21T13:32:56.663 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:d5bc9454:::61:head 2024-03-21T13:32:56.663 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:3ddfe313:::62:head 2024-03-21T13:32:56.663 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:7c2816b9:::63:head 2024-03-21T13:32:56.663 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:47e00e4d:::64:head 2024-03-21T13:32:56.663 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:c6410c18:::65:head 2024-03-21T13:32:56.663 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:b48ed237:::66:head 2024-03-21T13:32:56.663 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:cd63ad31:::67:head 2024-03-21T13:32:56.663 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:b179e92b:::68:head 2024-03-21T13:32:56.663 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:0d9f741a:::69:head 2024-03-21T13:32:56.663 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:6d3352ae:::70:head 2024-03-21T13:32:56.663 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:c6d5c19e:::71:head 2024-03-21T13:32:56.663 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:bc4729c3:::72:head 2024-03-21T13:32:56.663 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:77e930b9:::73:head 2024-03-21T13:32:56.663 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:0abeecfd:::74:head 2024-03-21T13:32:56.663 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:b7c37e15:::75:head 2024-03-21T13:32:56.663 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:b6378398:::76:head 2024-03-21T13:32:56.664 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:02bd68de:::77:head 2024-03-21T13:32:56.664 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:cc795d2d:::78:head 2024-03-21T13:32:56.664 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:630d4fea:::79:head 2024-03-21T13:32:56.664 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:e0d29ef5:::80:head 2024-03-21T13:32:56.664 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:fd6f13d2:::81:head 2024-03-21T13:32:56.664 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:606461d5:::82:head 2024-03-21T13:32:56.664 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:eadbdc43:::83:head 2024-03-21T13:32:56.664 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:8761d0bb:::84:head 2024-03-21T13:32:56.664 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:9ef0186f:::85:head 2024-03-21T13:32:56.664 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:e0d41294:::86:head 2024-03-21T13:32:56.664 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:961de695:::87:head 2024-03-21T13:32:56.664 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:1423148f:::88:head 2024-03-21T13:32:56.664 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:633a8fa2:::89:head 2024-03-21T13:32:56.664 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:a8653809:::90:head 2024-03-21T13:32:56.664 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:3dac8b33:::91:head 2024-03-21T13:32:56.665 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:35aad435:::92:head 2024-03-21T13:32:56.665 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:f6dcc343:::93:head 2024-03-21T13:32:56.665 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:dbbdad87:::94:head 2024-03-21T13:32:56.665 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:1cb48ce0:::95:head 2024-03-21T13:32:56.665 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:03cd461c:::96:head 2024-03-21T13:32:56.665 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:17a4ea99:::97:head 2024-03-21T13:32:56.665 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:9993c9a7:::98:head 2024-03-21T13:32:56.665 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:6394211c:::99:head 2024-03-21T13:32:56.665 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:94c7ae57:::100:head 2024-03-21T13:32:56.665 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:6fdee5bb:::101:head 2024-03-21T13:32:56.665 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:9a477fd1:::102:head 2024-03-21T13:32:56.665 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:eb850916:::103:head 2024-03-21T13:32:56.665 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:affc56b9:::104:head 2024-03-21T13:32:56.665 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:b42dc814:::105:head 2024-03-21T13:32:56.665 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:f319f8f0:::106:head 2024-03-21T13:32:56.666 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:9a40b9de:::107:head 2024-03-21T13:32:56.666 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:8b524f28:::108:head 2024-03-21T13:32:56.666 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:e3de589f:::109:head 2024-03-21T13:32:56.666 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:90f90a5b:::110:head 2024-03-21T13:32:56.666 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:a7b4f1d7:::111:head 2024-03-21T13:32:56.666 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:af51766e:::112:head 2024-03-21T13:32:56.666 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:b6f90bd1:::113:head 2024-03-21T13:32:56.666 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:e0261208:::114:head 2024-03-21T13:32:56.666 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:c9569ef7:::115:head 2024-03-21T13:32:56.666 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:61bebe50:::116:head 2024-03-21T13:32:56.666 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:fe93412b:::117:head 2024-03-21T13:32:56.666 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:d3d38bee:::118:head 2024-03-21T13:32:56.666 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:3100ba0c:::119:head 2024-03-21T13:32:56.666 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:d0560ada:::120:head 2024-03-21T13:32:56.666 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:f0ea8b35:::121:head 2024-03-21T13:32:56.666 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:766f231a:::122:head 2024-03-21T13:32:56.667 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:a07a2582:::123:head 2024-03-21T13:32:56.667 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:bd7c6b3a:::124:head 2024-03-21T13:32:56.667 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:fb2ddaff:::125:head 2024-03-21T13:32:56.667 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:4408e1fe:::126:head 2024-03-21T13:32:56.667 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:ee1df7a7:::127:head 2024-03-21T13:32:56.667 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:c3002909:::128:head 2024-03-21T13:32:56.667 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:4f48ffa9:::129:head 2024-03-21T13:32:56.667 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:edf38733:::130:head 2024-03-21T13:32:56.667 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:c08425c0:::131:head 2024-03-21T13:32:56.667 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:5f902d98:::132:head 2024-03-21T13:32:56.667 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:41ea2c93:::133:head 2024-03-21T13:32:56.667 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:813cee13:::134:head 2024-03-21T13:32:56.667 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:0131818d:::135:head 2024-03-21T13:32:56.667 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:26ba5a85:::136:head 2024-03-21T13:32:56.667 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:381b8a5a:::137:head 2024-03-21T13:32:56.667 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:28797e47:::138:head 2024-03-21T13:32:56.667 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:bfca7f22:::139:head 2024-03-21T13:32:56.668 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:36807075:::140:head 2024-03-21T13:32:56.668 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:80b03975:::141:head 2024-03-21T13:32:56.668 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:5c15709b:::142:head 2024-03-21T13:32:56.668 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:f39ea15e:::143:head 2024-03-21T13:32:56.668 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:ea992956:::144:head 2024-03-21T13:32:56.668 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:48887b1c:::145:head 2024-03-21T13:32:56.668 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:9f24a9dd:::146:head 2024-03-21T13:32:56.668 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:987f100b:::147:head 2024-03-21T13:32:56.668 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:d2dd3581:::148:head 2024-03-21T13:32:56.668 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:7fed1808:::149:head 2024-03-21T13:32:56.668 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:c80b70e9:::150:head 2024-03-21T13:32:56.668 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:85ed90f9:::151:head 2024-03-21T13:32:56.668 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:36428b24:::152:head 2024-03-21T13:32:56.668 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:d044c34a:::153:head 2024-03-21T13:32:56.668 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:7c18bf58:::154:head 2024-03-21T13:32:56.669 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:d1c21232:::155:head 2024-03-21T13:32:56.669 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:a7a3c575:::156:head 2024-03-21T13:32:56.669 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:87da0633:::157:head 2024-03-21T13:32:56.669 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:d5ac3822:::158:head 2024-03-21T13:32:56.669 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:3f20522d:::159:head 2024-03-21T13:32:56.669 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:6ca26563:::160:head 2024-03-21T13:32:56.669 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:532ce135:::161:head 2024-03-21T13:32:56.669 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:c78863e6:::162:head 2024-03-21T13:32:57.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:57 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-03-21T13:32:57.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:57 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi097-79636-57"}]': finished 2024-03-21T13:32:57.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:57 smithi097 ceph-mon[30040]: osdmap e301: 8 total, 8 up, 8 in 2024-03-21T13:32:57.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:57 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1802561187' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi097-79636-57"}]: dispatch 2024-03-21T13:32:57.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:57 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi097-79636-57"}]: dispatch 2024-03-21T13:32:57.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:57 smithi097 ceph-mon[30040]: pgmap v416: 324 pgs: 324 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:32:57.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:57 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi097-79906-43","var": "pg_num","format": "json"}]: dispatch 2024-03-21T13:32:57.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:57 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:32:57.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:57 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:32:57.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:57 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-43"}]: dispatch 2024-03-21T13:32:57.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:57 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-43"}]: dispatch 2024-03-21T13:32:57.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:57 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-03-21T13:32:57.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:57 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi097-79636-57"}]': finished 2024-03-21T13:32:57.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:57 smithi097 ceph-mon[39416]: osdmap e301: 8 total, 8 up, 8 in 2024-03-21T13:32:57.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:57 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1802561187' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi097-79636-57"}]: dispatch 2024-03-21T13:32:57.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:57 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi097-79636-57"}]: dispatch 2024-03-21T13:32:57.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:57 smithi097 ceph-mon[39416]: pgmap v416: 324 pgs: 324 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:32:57.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:57 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi097-79906-43","var": "pg_num","format": "json"}]: dispatch 2024-03-21T13:32:57.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:57 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:32:57.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:57 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:32:57.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:57 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-43"}]: dispatch 2024-03-21T13:32:57.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:57 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-43"}]: dispatch 2024-03-21T13:32:57.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:57 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-03-21T13:32:57.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:57 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi097-79636-57"}]': finished 2024-03-21T13:32:57.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:57 smithi144 ceph-mon[33633]: osdmap e301: 8 total, 8 up, 8 in 2024-03-21T13:32:57.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:57 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1802561187' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi097-79636-57"}]: dispatch 2024-03-21T13:32:57.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:57 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi097-79636-57"}]: dispatch 2024-03-21T13:32:57.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:57 smithi144 ceph-mon[33633]: pgmap v416: 324 pgs: 324 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:32:57.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:57 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi097-79906-43","var": "pg_num","format": "json"}]: dispatch 2024-03-21T13:32:57.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:57 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:32:57.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:57 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:32:57.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:57 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-43"}]: dispatch 2024-03-21T13:32:57.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:57 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-43"}]: dispatch 2024-03-21T13:32:58.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:58 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi097-79636-57"}]': finished 2024-03-21T13:32:58.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:58 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-43"}]': finished 2024-03-21T13:32:58.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:58 smithi097 ceph-mon[39416]: osdmap e302: 8 total, 8 up, 8 in 2024-03-21T13:32:58.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:58 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/600012106' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi097-79636-58"}]: dispatch 2024-03-21T13:32:58.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:58 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi097-79636-58"}]: dispatch 2024-03-21T13:32:58.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:58 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/600012106' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi097-79636-58"}]: dispatch 2024-03-21T13:32:58.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:58 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi097-79636-58"}]: dispatch 2024-03-21T13:32:58.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:58 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/600012106' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi097-79636-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:58.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:58 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi097-79636-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:58.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:58 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi097-79636-57"}]': finished 2024-03-21T13:32:58.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:58 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-43"}]': finished 2024-03-21T13:32:58.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:58 smithi097 ceph-mon[30040]: osdmap e302: 8 total, 8 up, 8 in 2024-03-21T13:32:58.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:58 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/600012106' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi097-79636-58"}]: dispatch 2024-03-21T13:32:58.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:58 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi097-79636-58"}]: dispatch 2024-03-21T13:32:58.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:58 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/600012106' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi097-79636-58"}]: dispatch 2024-03-21T13:32:58.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:58 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi097-79636-58"}]: dispatch 2024-03-21T13:32:58.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:58 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/600012106' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi097-79636-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:58.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:58 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi097-79636-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:58.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:58 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi097-79636-57"}]': finished 2024-03-21T13:32:58.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:58 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-43"}]': finished 2024-03-21T13:32:58.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:58 smithi144 ceph-mon[33633]: osdmap e302: 8 total, 8 up, 8 in 2024-03-21T13:32:58.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:58 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/600012106' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi097-79636-58"}]: dispatch 2024-03-21T13:32:58.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:58 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi097-79636-58"}]: dispatch 2024-03-21T13:32:58.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:58 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/600012106' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi097-79636-58"}]: dispatch 2024-03-21T13:32:58.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:58 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi097-79636-58"}]: dispatch 2024-03-21T13:32:58.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:58 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/600012106' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi097-79636-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:58.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:58 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi097-79636-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:32:59.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:59 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi097-79636-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:32:59.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:59 smithi097 ceph-mon[39416]: osdmap e303: 8 total, 8 up, 8 in 2024-03-21T13:32:59.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:59 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/600012106' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi097-79636-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi097-79636-58"}]: dispatch 2024-03-21T13:32:59.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:59 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi097-79636-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi097-79636-58"}]: dispatch 2024-03-21T13:32:59.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:32:59 smithi097 ceph-mon[39416]: pgmap v419: 292 pgs: 292 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:32:59.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:59 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi097-79636-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:32:59.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:59 smithi097 ceph-mon[30040]: osdmap e303: 8 total, 8 up, 8 in 2024-03-21T13:32:59.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:59 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/600012106' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi097-79636-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi097-79636-58"}]: dispatch 2024-03-21T13:32:59.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:59 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi097-79636-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi097-79636-58"}]: dispatch 2024-03-21T13:32:59.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:32:59 smithi097 ceph-mon[30040]: pgmap v419: 292 pgs: 292 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:32:59.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:59 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi097-79636-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:32:59.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:59 smithi144 ceph-mon[33633]: osdmap e303: 8 total, 8 up, 8 in 2024-03-21T13:32:59.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:59 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/600012106' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi097-79636-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi097-79636-58"}]: dispatch 2024-03-21T13:32:59.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:59 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi097-79636-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi097-79636-58"}]: dispatch 2024-03-21T13:32:59.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:32:59 smithi144 ceph-mon[33633]: pgmap v419: 292 pgs: 292 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:33:00.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:00 smithi097 ceph-mon[39416]: osdmap e304: 8 total, 8 up, 8 in 2024-03-21T13:33:00.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:00 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:33:00.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:00 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:33:00.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:00 smithi097 ceph-mon[30040]: osdmap e304: 8 total, 8 up, 8 in 2024-03-21T13:33:00.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:00 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:33:00.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:00 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:33:00.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:00 smithi144 ceph-mon[33633]: osdmap e304: 8 total, 8 up, 8 in 2024-03-21T13:33:00.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:00 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:33:00.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:00 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:33:01.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:01 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi097-79636-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi097-79636-58"}]': finished 2024-03-21T13:33:01.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:01 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:33:01.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:01 smithi097 ceph-mon[39416]: osdmap e305: 8 total, 8 up, 8 in 2024-03-21T13:33:01.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:01 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:33:01.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:01 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:33:01.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:01 smithi097 ceph-mon[39416]: pgmap v422: 332 pgs: 40 unknown, 292 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:33:01.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:01 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi097-79636-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi097-79636-58"}]': finished 2024-03-21T13:33:01.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:01 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:33:01.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:01 smithi097 ceph-mon[30040]: osdmap e305: 8 total, 8 up, 8 in 2024-03-21T13:33:01.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:01 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:33:01.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:01 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:33:01.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:01 smithi097 ceph-mon[30040]: pgmap v422: 332 pgs: 40 unknown, 292 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:33:01.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:01 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi097-79636-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi097-79636-58"}]': finished 2024-03-21T13:33:01.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:01 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:33:01.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:01 smithi144 ceph-mon[33633]: osdmap e305: 8 total, 8 up, 8 in 2024-03-21T13:33:01.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:01 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:33:01.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:01 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:33:01.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:01 smithi144 ceph-mon[33633]: pgmap v422: 332 pgs: 40 unknown, 292 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:33:02.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:33:01 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:33:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:33:02.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:02 smithi097 ceph-mon[39416]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:33:02.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:02 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-45", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:33:02.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:02 smithi097 ceph-mon[39416]: osdmap e306: 8 total, 8 up, 8 in 2024-03-21T13:33:02.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:02 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-45","var": "hit_set_count","val": "3"}]: dispatch 2024-03-21T13:33:02.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:02 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-45","var": "hit_set_count","val": "3"}]: dispatch 2024-03-21T13:33:02.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:02 smithi097 ceph-mon[30040]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:33:02.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:02 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-45", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:33:02.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:02 smithi097 ceph-mon[30040]: osdmap e306: 8 total, 8 up, 8 in 2024-03-21T13:33:02.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:02 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-45","var": "hit_set_count","val": "3"}]: dispatch 2024-03-21T13:33:02.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:02 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-45","var": "hit_set_count","val": "3"}]: dispatch 2024-03-21T13:33:02.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:02 smithi144 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:33:02.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:02 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-45", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:33:02.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:02 smithi144 ceph-mon[33633]: osdmap e306: 8 total, 8 up, 8 in 2024-03-21T13:33:02.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:02 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-45","var": "hit_set_count","val": "3"}]: dispatch 2024-03-21T13:33:02.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:02 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-45","var": "hit_set_count","val": "3"}]: dispatch 2024-03-21T13:33:03.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:03 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-45","var": "hit_set_count","val": "3"}]': finished 2024-03-21T13:33:03.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:03 smithi097 ceph-mon[39416]: osdmap e307: 8 total, 8 up, 8 in 2024-03-21T13:33:03.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:03 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-45","var": "hit_set_period","val": "3"}]: dispatch 2024-03-21T13:33:03.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:03 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-45","var": "hit_set_period","val": "3"}]: dispatch 2024-03-21T13:33:03.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:03 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/600012106' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi097-79636-58"}]: dispatch 2024-03-21T13:33:03.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:03 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi097-79636-58"}]: dispatch 2024-03-21T13:33:03.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:03 smithi097 ceph-mon[39416]: pgmap v425: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:33:03.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:03 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-45","var": "hit_set_count","val": "3"}]': finished 2024-03-21T13:33:03.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:03 smithi097 ceph-mon[30040]: osdmap e307: 8 total, 8 up, 8 in 2024-03-21T13:33:03.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:03 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-45","var": "hit_set_period","val": "3"}]: dispatch 2024-03-21T13:33:03.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:03 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-45","var": "hit_set_period","val": "3"}]: dispatch 2024-03-21T13:33:03.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:03 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/600012106' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi097-79636-58"}]: dispatch 2024-03-21T13:33:03.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:03 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi097-79636-58"}]: dispatch 2024-03-21T13:33:03.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:03 smithi097 ceph-mon[30040]: pgmap v425: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:33:03.871 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:33:03 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:33:03.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:03 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-45","var": "hit_set_count","val": "3"}]': finished 2024-03-21T13:33:03.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:03 smithi144 ceph-mon[33633]: osdmap e307: 8 total, 8 up, 8 in 2024-03-21T13:33:03.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:03 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-45","var": "hit_set_period","val": "3"}]: dispatch 2024-03-21T13:33:03.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:03 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-45","var": "hit_set_period","val": "3"}]: dispatch 2024-03-21T13:33:03.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:03 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/600012106' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi097-79636-58"}]: dispatch 2024-03-21T13:33:03.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:03 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi097-79636-58"}]: dispatch 2024-03-21T13:33:03.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:03 smithi144 ceph-mon[33633]: pgmap v425: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:33:04.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:04 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-45","var": "hit_set_period","val": "3"}]': finished 2024-03-21T13:33:04.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:04 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi097-79636-58"}]': finished 2024-03-21T13:33:04.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:04 smithi097 ceph-mon[39416]: osdmap e308: 8 total, 8 up, 8 in 2024-03-21T13:33:04.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:04 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:33:04.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:04 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/600012106' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi097-79636-58"}]: dispatch 2024-03-21T13:33:04.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:04 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:33:04.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:04 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi097-79636-58"}]: dispatch 2024-03-21T13:33:04.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:04 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:33:04.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:04 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-45","var": "hit_set_period","val": "3"}]': finished 2024-03-21T13:33:04.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:04 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi097-79636-58"}]': finished 2024-03-21T13:33:04.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:04 smithi097 ceph-mon[30040]: osdmap e308: 8 total, 8 up, 8 in 2024-03-21T13:33:04.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:04 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:33:04.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:04 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/600012106' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi097-79636-58"}]: dispatch 2024-03-21T13:33:04.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:04 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:33:04.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:04 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi097-79636-58"}]: dispatch 2024-03-21T13:33:04.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:04 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:33:04.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:04 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-45","var": "hit_set_period","val": "3"}]': finished 2024-03-21T13:33:04.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:04 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi097-79636-58"}]': finished 2024-03-21T13:33:04.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:04 smithi144 ceph-mon[33633]: osdmap e308: 8 total, 8 up, 8 in 2024-03-21T13:33:04.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:04 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:33:04.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:04 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/600012106' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi097-79636-58"}]: dispatch 2024-03-21T13:33:04.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:04 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:33:04.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:04 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi097-79636-58"}]: dispatch 2024-03-21T13:33:04.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:04 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:33:05.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:05 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-45","var": "hit_set_type","val": "bloom"}]': finished 2024-03-21T13:33:05.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:05 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi097-79636-58"}]': finished 2024-03-21T13:33:05.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:05 smithi097 ceph-mon[39416]: osdmap e309: 8 total, 8 up, 8 in 2024-03-21T13:33:05.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:05 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-03-21T13:33:05.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:05 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-03-21T13:33:05.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:05 smithi097 ceph-mon[39416]: pgmap v428: 324 pgs: 324 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:33:05.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:05 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1332783429' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi097-79636-59"}]: dispatch 2024-03-21T13:33:05.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:05 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1332783429' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi097-79636-59"}]: dispatch 2024-03-21T13:33:05.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:05 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1332783429' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi097-79636-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:33:05.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:05 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-45","var": "hit_set_type","val": "bloom"}]': finished 2024-03-21T13:33:05.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:05 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi097-79636-58"}]': finished 2024-03-21T13:33:05.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:05 smithi097 ceph-mon[30040]: osdmap e309: 8 total, 8 up, 8 in 2024-03-21T13:33:05.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:05 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-03-21T13:33:05.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:05 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-03-21T13:33:05.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:05 smithi097 ceph-mon[30040]: pgmap v428: 324 pgs: 324 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:33:05.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:05 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1332783429' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi097-79636-59"}]: dispatch 2024-03-21T13:33:05.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:05 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1332783429' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi097-79636-59"}]: dispatch 2024-03-21T13:33:05.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:05 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1332783429' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi097-79636-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:33:05.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:05 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-45","var": "hit_set_type","val": "bloom"}]': finished 2024-03-21T13:33:05.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:05 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi097-79636-58"}]': finished 2024-03-21T13:33:05.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:05 smithi144 ceph-mon[33633]: osdmap e309: 8 total, 8 up, 8 in 2024-03-21T13:33:05.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:05 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-03-21T13:33:05.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:05 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-03-21T13:33:05.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:05 smithi144 ceph-mon[33633]: pgmap v428: 324 pgs: 324 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:33:05.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:05 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1332783429' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi097-79636-59"}]: dispatch 2024-03-21T13:33:05.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:05 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1332783429' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi097-79636-59"}]: dispatch 2024-03-21T13:33:05.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:05 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1332783429' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi097-79636-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:33:06.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:06 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-03-21T13:33:06.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:06 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1332783429' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi097-79636-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:33:06.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:06 smithi097 ceph-mon[39416]: osdmap e310: 8 total, 8 up, 8 in 2024-03-21T13:33:06.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:06 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1332783429' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi097-79636-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi097-79636-59"}]: dispatch 2024-03-21T13:33:06.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:06 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-03-21T13:33:06.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:06 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1332783429' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi097-79636-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:33:06.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:06 smithi097 ceph-mon[30040]: osdmap e310: 8 total, 8 up, 8 in 2024-03-21T13:33:06.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:06 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1332783429' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi097-79636-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi097-79636-59"}]: dispatch 2024-03-21T13:33:06.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:06 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-03-21T13:33:06.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:06 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1332783429' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi097-79636-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:33:06.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:06 smithi144 ceph-mon[33633]: osdmap e310: 8 total, 8 up, 8 in 2024-03-21T13:33:06.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:06 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1332783429' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi097-79636-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi097-79636-59"}]: dispatch 2024-03-21T13:33:07.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:07 smithi097 ceph-mon[39416]: pgmap v430: 324 pgs: 324 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:33:07.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:07 smithi097 ceph-mon[39416]: osdmap e311: 8 total, 8 up, 8 in 2024-03-21T13:33:07.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:07 smithi097 ceph-mon[39416]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:33:07.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:07 smithi097 ceph-mon[30040]: pgmap v430: 324 pgs: 324 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:33:07.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:07 smithi097 ceph-mon[30040]: osdmap e311: 8 total, 8 up, 8 in 2024-03-21T13:33:07.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:07 smithi097 ceph-mon[30040]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:33:07.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:07 smithi144 ceph-mon[33633]: pgmap v430: 324 pgs: 324 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:33:07.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:07 smithi144 ceph-mon[33633]: osdmap e311: 8 total, 8 up, 8 in 2024-03-21T13:33:07.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:07 smithi144 ceph-mon[33633]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:33:08.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:08 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1332783429' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi097-79636-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi097-79636-59"}]': finished 2024-03-21T13:33:08.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:08 smithi097 ceph-mon[39416]: osdmap e312: 8 total, 8 up, 8 in 2024-03-21T13:33:08.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:08 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1332783429' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi097-79636-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi097-79636-59"}]': finished 2024-03-21T13:33:08.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:08 smithi097 ceph-mon[30040]: osdmap e312: 8 total, 8 up, 8 in 2024-03-21T13:33:08.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:08 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1332783429' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi097-79636-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi097-79636-59"}]': finished 2024-03-21T13:33:08.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:08 smithi144 ceph-mon[33633]: osdmap e312: 8 total, 8 up, 8 in 2024-03-21T13:33:09.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:09 smithi097 ceph-mon[39416]: pgmap v433: 332 pgs: 8 creating+peering, 324 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 765 B/s wr, 1 op/s 2024-03-21T13:33:09.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:09 smithi097 ceph-mon[39416]: osdmap e313: 8 total, 8 up, 8 in 2024-03-21T13:33:09.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:09 smithi097 ceph-mon[30040]: pgmap v433: 332 pgs: 8 creating+peering, 324 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 765 B/s wr, 1 op/s 2024-03-21T13:33:09.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:09 smithi097 ceph-mon[30040]: osdmap e313: 8 total, 8 up, 8 in 2024-03-21T13:33:09.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:09 smithi144 ceph-mon[33633]: pgmap v433: 332 pgs: 8 creating+peering, 324 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 765 B/s wr, 1 op/s 2024-03-21T13:33:09.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:09 smithi144 ceph-mon[33633]: osdmap e313: 8 total, 8 up, 8 in 2024-03-21T13:33:10.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:10 smithi097 ceph-mon[39416]: osdmap e314: 8 total, 8 up, 8 in 2024-03-21T13:33:10.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:10 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1332783429' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi097-79636-59"}]: dispatch 2024-03-21T13:33:10.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:10 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:33:10.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:10 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1332783429' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi097-79636-59"}]': finished 2024-03-21T13:33:10.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:10 smithi097 ceph-mon[30040]: osdmap e314: 8 total, 8 up, 8 in 2024-03-21T13:33:10.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:10 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1332783429' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi097-79636-59"}]: dispatch 2024-03-21T13:33:10.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:10 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:33:10.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:10 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1332783429' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi097-79636-59"}]': finished 2024-03-21T13:33:10.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:10 smithi144 ceph-mon[33633]: osdmap e314: 8 total, 8 up, 8 in 2024-03-21T13:33:10.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:10 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1332783429' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi097-79636-59"}]: dispatch 2024-03-21T13:33:10.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:10 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:33:10.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:10 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1332783429' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi097-79636-59"}]': finished 2024-03-21T13:33:11.506 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:7462ddf6:::1o.RoundTripAppendPP (3247 ms) 2024-03-21T13:33:11.507 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAio.RacingRemovePP 2024-03-21T13:33:11.507 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAio.RacingRemovePP (2788 ms) 2024-03-21T13:33:11.507 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP 2024-03-21T13:33:11.507 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP (2954 ms) 2024-03-21T13:33:11.507 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP2 2024-03-21T13:33:11.507 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP2 (3018 ms) 2024-03-21T13:33:11.507 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolEIOFlag 2024-03-21T13:33:11.507 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: setting pool EIO 2024-03-21T13:33:11.507 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: max_success 99, min_failed 100 2024-03-21T13:33:11.507 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolEIOFlag (4035 ms) 2024-03-21T13:33:11.507 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiReads 2024-03-21T13:33:11.507 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiReads (3005 ms) 2024-03-21T13:33:11.507 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio (124876 ms total) 2024-03-21T13:33:11.507 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: 2024-03-21T13:33:11.507 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP 2024-03-21T13:33:11.508 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.ReadIntoBufferlist 2024-03-21T13:33:11.508 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAioPP.ReadIntoBufferlist (3034 ms) 2024-03-21T13:33:11.508 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.XattrsRoundTripPP 2024-03-21T13:33:11.508 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAioPP.XattrsRoundTripPP (9269 ms) 2024-03-21T13:33:11.508 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RmXattrPP 2024-03-21T13:33:11.508 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RmXattrPP (14654 ms) 2024-03-21T13:33:11.508 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RemoveTestPP 2024-03-21T13:33:11.508 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RemoveTestPP (3009 ms) 2024-03-21T13:33:11.508 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP (29966 ms total) 2024-03-21T13:33:11.508 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: 2024-03-21T13:33:11.508 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP 2024-03-21T13:33:11.508 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-03-21T13:33:11.508 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosIoPP.XattrListPP (3260 ms) 2024-03-21T13:33:11.508 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP (3260 ms total) 2024-03-21T13:33:11.508 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: 2024-03-21T13:33:11.508 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC 2024-03-21T13:33:11.509 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleWritePP 2024-03-21T13:33:11.509 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleWritePP (13749 ms) 2024-03-21T13:33:11.509 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.WaitForSafePP 2024-03-21T13:33:11.509 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAioEC.WaitForSafePP (6984 ms) 2024-03-21T13:33:11.509 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP 2024-03-21T13:33:11.509 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP (7022 ms) 2024-03-21T13:33:11.509 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP2 2024-03-21T13:33:11.509 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP2 (7037 ms) 2024-03-21T13:33:11.509 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP3 2024-03-21T13:33:11.509 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP3 (3014 ms) 2024-03-21T13:33:11.509 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripSparseReadPP 2024-03-21T13:33:11.509 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripSparseReadPP (7037 ms) 2024-03-21T13:33:11.509 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripAppendPP 2024-03-21T13:33:11.509 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripAppendPP (6905 ms) 2024-03-21T13:33:11.509 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsCompletePP 2024-03-21T13:33:11.509 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsCompletePP (7043 ms) 2024-03-21T13:33:11.510 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsSafePP 2024-03-21T13:33:11.510 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsSafePP (7029 ms) 2024-03-21T13:33:11.510 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ReturnValuePP 2024-03-21T13:33:11.510 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ReturnValuePP (7037 ms) 2024-03-21T13:33:11.510 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushPP 2024-03-21T13:33:11.510 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushPP (7031 ms) 2024-03-21T13:33:11.510 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushAsyncPP 2024-03-21T13:33:11.510 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushAsyncPP (7047 ms) 2024-03-21T13:33:11.510 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP 2024-03-21T13:33:11.510 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP (7038 ms) 2024-03-21T13:33:11.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:11 smithi097 ceph-mon[39416]: pgmap v436: 324 pgs: 324 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-03-21T13:33:11.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:11 smithi097 ceph-mon[39416]: osdmap e315: 8 total, 8 up, 8 in 2024-03-21T13:33:11.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:11 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1332783429' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi097-79636-59"}]: dispatch 2024-03-21T13:33:11.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:11 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1332783429' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi097-79636-59"}]': finished 2024-03-21T13:33:11.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:11 smithi097 ceph-mon[30040]: pgmap v436: 324 pgs: 324 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-03-21T13:33:11.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:11 smithi097 ceph-mon[30040]: osdmap e315: 8 total, 8 up, 8 in 2024-03-21T13:33:11.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:11 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1332783429' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi097-79636-59"}]: dispatch 2024-03-21T13:33:11.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:11 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1332783429' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi097-79636-59"}]': finished 2024-03-21T13:33:11.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:11 smithi144 ceph-mon[33633]: pgmap v436: 324 pgs: 324 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-03-21T13:33:11.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:11 smithi144 ceph-mon[33633]: osdmap e315: 8 total, 8 up, 8 in 2024-03-21T13:33:11.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:11 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1332783429' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi097-79636-59"}]: dispatch 2024-03-21T13:33:11.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:11 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1332783429' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi097-79636-59"}]': finished 2024-03-21T13:33:12.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:33:11 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:33:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:33:12.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:12 smithi097 ceph-mon[39416]: osdmap e316: 8 total, 8 up, 8 in 2024-03-21T13:33:12.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:12 smithi097 ceph-mon[30040]: osdmap e316: 8 total, 8 up, 8 in 2024-03-21T13:33:12.870 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:12 smithi144 ceph-mon[33633]: osdmap e316: 8 total, 8 up, 8 in 2024-03-21T13:33:13.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:13 smithi097 ceph-mon[39416]: pgmap v439: 324 pgs: 324 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:33:13.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:13 smithi097 ceph-mon[39416]: osdmap e317: 8 total, 8 up, 8 in 2024-03-21T13:33:13.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:13 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1828898140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi097-79636-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:33:13.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:13 smithi097 ceph-mon[30040]: pgmap v439: 324 pgs: 324 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:33:13.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:13 smithi097 ceph-mon[30040]: osdmap e317: 8 total, 8 up, 8 in 2024-03-21T13:33:13.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:13 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1828898140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi097-79636-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:33:13.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:13 smithi144 ceph-mon[33633]: pgmap v439: 324 pgs: 324 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:33:13.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:13 smithi144 ceph-mon[33633]: osdmap e317: 8 total, 8 up, 8 in 2024-03-21T13:33:13.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:13 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1828898140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi097-79636-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:33:13.871 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:33:13 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:33:14.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:14 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1828898140' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi097-79636-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:33:14.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:14 smithi097 ceph-mon[39416]: osdmap e318: 8 total, 8 up, 8 in 2024-03-21T13:33:14.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:14 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:33:14.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:14 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1828898140' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi097-79636-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:33:14.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:14 smithi097 ceph-mon[30040]: osdmap e318: 8 total, 8 up, 8 in 2024-03-21T13:33:14.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:14 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:33:14.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:14 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1828898140' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi097-79636-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:33:14.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:14 smithi144 ceph-mon[33633]: osdmap e318: 8 total, 8 up, 8 in 2024-03-21T13:33:14.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:14 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:33:15.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:15 smithi097 ceph-mon[39416]: pgmap v442: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-03-21T13:33:15.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:15 smithi097 ceph-mon[39416]: osdmap e319: 8 total, 8 up, 8 in 2024-03-21T13:33:15.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:15 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2849646535' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi097-79636-61"}]: dispatch 2024-03-21T13:33:15.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:15 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi097-79636-61"}]: dispatch 2024-03-21T13:33:15.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:15 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2849646535' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi097-79636-61"}]: dispatch 2024-03-21T13:33:15.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:15 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi097-79636-61"}]: dispatch 2024-03-21T13:33:15.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:15 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2849646535' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi097-79636-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:33:15.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:15 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi097-79636-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:33:15.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:15 smithi097 ceph-mon[30040]: pgmap v442: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-03-21T13:33:15.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:15 smithi097 ceph-mon[30040]: osdmap e319: 8 total, 8 up, 8 in 2024-03-21T13:33:15.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:15 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2849646535' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi097-79636-61"}]: dispatch 2024-03-21T13:33:15.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:15 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi097-79636-61"}]: dispatch 2024-03-21T13:33:15.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:15 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2849646535' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi097-79636-61"}]: dispatch 2024-03-21T13:33:15.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:15 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi097-79636-61"}]: dispatch 2024-03-21T13:33:15.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:15 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2849646535' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi097-79636-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:33:15.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:15 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi097-79636-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:33:15.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:15 smithi144 ceph-mon[33633]: pgmap v442: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-03-21T13:33:15.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:15 smithi144 ceph-mon[33633]: osdmap e319: 8 total, 8 up, 8 in 2024-03-21T13:33:15.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:15 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2849646535' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi097-79636-61"}]: dispatch 2024-03-21T13:33:15.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:15 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi097-79636-61"}]: dispatch 2024-03-21T13:33:15.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:15 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2849646535' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi097-79636-61"}]: dispatch 2024-03-21T13:33:15.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:15 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi097-79636-61"}]: dispatch 2024-03-21T13:33:15.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:15 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2849646535' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi097-79636-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:33:15.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:15 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi097-79636-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:33:16.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:16 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi097-79636-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:33:16.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:16 smithi097 ceph-mon[39416]: osdmap e320: 8 total, 8 up, 8 in 2024-03-21T13:33:16.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:16 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2849646535' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi097-79636-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi097-79636-61"}]: dispatch 2024-03-21T13:33:16.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:16 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi097-79636-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi097-79636-61"}]: dispatch 2024-03-21T13:33:16.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:16 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi097-79636-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:33:16.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:16 smithi097 ceph-mon[30040]: osdmap e320: 8 total, 8 up, 8 in 2024-03-21T13:33:16.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:16 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2849646535' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi097-79636-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi097-79636-61"}]: dispatch 2024-03-21T13:33:16.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:16 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi097-79636-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi097-79636-61"}]: dispatch 2024-03-21T13:33:16.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:16 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi097-79636-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:33:16.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:16 smithi144 ceph-mon[33633]: osdmap e320: 8 total, 8 up, 8 in 2024-03-21T13:33:16.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:16 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2849646535' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi097-79636-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi097-79636-61"}]: dispatch 2024-03-21T13:33:16.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:16 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi097-79636-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi097-79636-61"}]: dispatch 2024-03-21T13:33:17.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:17 smithi097 ceph-mon[39416]: pgmap v445: 324 pgs: 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-03-21T13:33:17.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:17 smithi097 ceph-mon[39416]: osdmap e321: 8 total, 8 up, 8 in 2024-03-21T13:33:17.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:17 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:33:17.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:17 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:33:17.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:17 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-45"}]: dispatch 2024-03-21T13:33:17.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:17 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-45"}]: dispatch 2024-03-21T13:33:17.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:17 smithi097 ceph-mon[30040]: pgmap v445: 324 pgs: 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-03-21T13:33:17.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:17 smithi097 ceph-mon[30040]: osdmap e321: 8 total, 8 up, 8 in 2024-03-21T13:33:17.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:17 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:33:17.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:17 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:33:17.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:17 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-45"}]: dispatch 2024-03-21T13:33:17.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:17 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-45"}]: dispatch 2024-03-21T13:33:17.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:17 smithi144 ceph-mon[33633]: pgmap v445: 324 pgs: 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-03-21T13:33:17.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:17 smithi144 ceph-mon[33633]: osdmap e321: 8 total, 8 up, 8 in 2024-03-21T13:33:17.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:17 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:33:17.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:17 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:33:17.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:17 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-45"}]: dispatch 2024-03-21T13:33:17.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:17 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-45"}]: dispatch 2024-03-21T13:33:18.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:18 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi097-79636-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi097-79636-61"}]': finished 2024-03-21T13:33:18.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:18 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-45"}]': finished 2024-03-21T13:33:18.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:18 smithi144 ceph-mon[33633]: osdmap e322: 8 total, 8 up, 8 in 2024-03-21T13:33:19.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:18 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi097-79636-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi097-79636-61"}]': finished 2024-03-21T13:33:19.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:18 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-45"}]': finished 2024-03-21T13:33:19.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:18 smithi097 ceph-mon[39416]: osdmap e322: 8 total, 8 up, 8 in 2024-03-21T13:33:19.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:18 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi097-79636-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi097-79636-61"}]': finished 2024-03-21T13:33:19.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:18 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-45"}]': finished 2024-03-21T13:33:19.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:18 smithi097 ceph-mon[30040]: osdmap e322: 8 total, 8 up, 8 in 2024-03-21T13:33:19.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:19 smithi144 ceph-mon[33633]: pgmap v448: 332 pgs: 8 creating+peering, 324 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-21T13:33:19.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:19 smithi144 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:33:19.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:19 smithi144 ceph-mon[33633]: osdmap e323: 8 total, 8 up, 8 in 2024-03-21T13:33:20.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:19 smithi097 ceph-mon[39416]: pgmap v448: 332 pgs: 8 creating+peering, 324 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-21T13:33:20.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:19 smithi097 ceph-mon[39416]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:33:20.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:19 smithi097 ceph-mon[39416]: osdmap e323: 8 total, 8 up, 8 in 2024-03-21T13:33:20.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:19 smithi097 ceph-mon[30040]: pgmap v448: 332 pgs: 8 creating+peering, 324 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-21T13:33:20.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:19 smithi097 ceph-mon[30040]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:33:20.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:19 smithi097 ceph-mon[30040]: osdmap e323: 8 total, 8 up, 8 in 2024-03-21T13:33:20.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:20 smithi144 ceph-mon[33633]: osdmap e324: 8 total, 8 up, 8 in 2024-03-21T13:33:20.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:20 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2849646535' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi097-79636-61"}]: dispatch 2024-03-21T13:33:20.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:20 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi097-79636-61"}]: dispatch 2024-03-21T13:33:20.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:20 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:33:20.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:20 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:33:21.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:20 smithi097 ceph-mon[39416]: osdmap e324: 8 total, 8 up, 8 in 2024-03-21T13:33:21.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:20 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2849646535' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi097-79636-61"}]: dispatch 2024-03-21T13:33:21.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:20 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi097-79636-61"}]: dispatch 2024-03-21T13:33:21.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:20 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:33:21.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:20 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:33:21.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:20 smithi097 ceph-mon[30040]: osdmap e324: 8 total, 8 up, 8 in 2024-03-21T13:33:21.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:20 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2849646535' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi097-79636-61"}]: dispatch 2024-03-21T13:33:21.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:20 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi097-79636-61"}]: dispatch 2024-03-21T13:33:21.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:20 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:33:21.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:20 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:33:21.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:21 smithi144 ceph-mon[33633]: pgmap v451: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:33:21.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:21 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi097-79636-61"}]': finished 2024-03-21T13:33:21.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:21 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:33:21.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:21 smithi144 ceph-mon[33633]: osdmap e325: 8 total, 8 up, 8 in 2024-03-21T13:33:21.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:21 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2849646535' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi097-79636-61"}]: dispatch 2024-03-21T13:33:21.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:21 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi097-79636-61"}]: dispatch 2024-03-21T13:33:21.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:21 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:33:21.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:21 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:33:21.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:21 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi097-79636-61"}]': finished 2024-03-21T13:33:21.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:21 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-47", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:33:21.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:21 smithi144 ceph-mon[33633]: osdmap e326: 8 total, 8 up, 8 in 2024-03-21T13:33:21.949 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:21 smithi097 ceph-mon[39416]: pgmap v451: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:33:21.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:21 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi097-79636-61"}]': finished 2024-03-21T13:33:21.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:21 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:33:21.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:21 smithi097 ceph-mon[39416]: osdmap e325: 8 total, 8 up, 8 in 2024-03-21T13:33:21.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:21 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2849646535' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi097-79636-61"}]: dispatch 2024-03-21T13:33:21.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:21 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi097-79636-61"}]: dispatch 2024-03-21T13:33:21.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:21 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:33:21.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:21 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:33:21.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:21 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi097-79636-61"}]': finished 2024-03-21T13:33:21.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:21 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-47", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:33:21.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:21 smithi097 ceph-mon[39416]: osdmap e326: 8 total, 8 up, 8 in 2024-03-21T13:33:21.950 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:21 smithi097 ceph-mon[30040]: pgmap v451: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:33:21.950 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:21 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi097-79636-61"}]': finished 2024-03-21T13:33:21.951 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:21 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:33:21.951 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:21 smithi097 ceph-mon[30040]: osdmap e325: 8 total, 8 up, 8 in 2024-03-21T13:33:21.951 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:21 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2849646535' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi097-79636-61"}]: dispatch 2024-03-21T13:33:21.951 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:21 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi097-79636-61"}]: dispatch 2024-03-21T13:33:21.951 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:21 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:33:21.951 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:21 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:33:21.951 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:21 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi097-79636-61"}]': finished 2024-03-21T13:33:21.951 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:21 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-47", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:33:21.951 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:21 smithi097 ceph-mon[30040]: osdmap e326: 8 total, 8 up, 8 in 2024-03-21T13:33:22.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:33:21 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:33:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:33:22.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:22 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-47"}]: dispatch 2024-03-21T13:33:22.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:22 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-47"}]: dispatch 2024-03-21T13:33:22.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:22 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4229220967' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi097-79636-62"}]: dispatch 2024-03-21T13:33:22.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:22 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4229220967' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi097-79636-62"}]: dispatch 2024-03-21T13:33:22.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:22 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4229220967' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi097-79636-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:33:22.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:22 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-47"}]': finished 2024-03-21T13:33:22.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:22 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4229220967' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi097-79636-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:33:23.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:22 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-47"}]: dispatch 2024-03-21T13:33:23.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:22 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-47"}]: dispatch 2024-03-21T13:33:23.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:22 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4229220967' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi097-79636-62"}]: dispatch 2024-03-21T13:33:23.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:22 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4229220967' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi097-79636-62"}]: dispatch 2024-03-21T13:33:23.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:22 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4229220967' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi097-79636-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:33:23.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:22 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-47"}]': finished 2024-03-21T13:33:23.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:22 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4229220967' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi097-79636-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:33:23.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:22 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-47"}]: dispatch 2024-03-21T13:33:23.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:22 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-47"}]: dispatch 2024-03-21T13:33:23.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:22 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4229220967' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi097-79636-62"}]: dispatch 2024-03-21T13:33:23.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:22 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4229220967' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi097-79636-62"}]: dispatch 2024-03-21T13:33:23.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:22 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4229220967' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi097-79636-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:33:23.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:22 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-47"}]': finished 2024-03-21T13:33:23.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:22 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4229220967' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi097-79636-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:33:24.121 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:33:23 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:33:24.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:23 smithi144 ceph-mon[33633]: pgmap v454: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:33:24.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:23 smithi144 ceph-mon[33633]: osdmap e327: 8 total, 8 up, 8 in 2024-03-21T13:33:24.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:23 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4229220967' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi097-79636-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi097-79636-62"}]: dispatch 2024-03-21T13:33:24.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:23 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-47", "mode": "writeback"}]: dispatch 2024-03-21T13:33:24.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:23 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-47", "mode": "writeback"}]: dispatch 2024-03-21T13:33:24.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:23 smithi144 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:33:24.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:23 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-47", "mode": "writeback"}]': finished 2024-03-21T13:33:24.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:23 smithi097 ceph-mon[39416]: pgmap v454: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:33:24.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:23 smithi097 ceph-mon[39416]: osdmap e327: 8 total, 8 up, 8 in 2024-03-21T13:33:24.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:23 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4229220967' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi097-79636-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi097-79636-62"}]: dispatch 2024-03-21T13:33:24.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:23 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-47", "mode": "writeback"}]: dispatch 2024-03-21T13:33:24.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:23 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-47", "mode": "writeback"}]: dispatch 2024-03-21T13:33:24.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:23 smithi097 ceph-mon[39416]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:33:24.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:23 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-47", "mode": "writeback"}]': finished 2024-03-21T13:33:24.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:23 smithi097 ceph-mon[30040]: pgmap v454: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:33:24.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:23 smithi097 ceph-mon[30040]: osdmap e327: 8 total, 8 up, 8 in 2024-03-21T13:33:24.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:23 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4229220967' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi097-79636-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi097-79636-62"}]: dispatch 2024-03-21T13:33:24.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:23 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-47", "mode": "writeback"}]: dispatch 2024-03-21T13:33:24.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:23 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-47", "mode": "writeback"}]: dispatch 2024-03-21T13:33:24.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:23 smithi097 ceph-mon[30040]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:33:24.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:23 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-47", "mode": "writeback"}]': finished 2024-03-21T13:33:25.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:24 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:33:25.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:24 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:33:25.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:24 smithi144 ceph-mon[33633]: osdmap e328: 8 total, 8 up, 8 in 2024-03-21T13:33:25.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:24 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:33:25.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:24 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4229220967' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi097-79636-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi097-79636-62"}]': finished 2024-03-21T13:33:25.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:24 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "hit_set_count","val": "2"}]': finished 2024-03-21T13:33:25.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:24 smithi144 ceph-mon[33633]: osdmap e329: 8 total, 8 up, 8 in 2024-03-21T13:33:25.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:24 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:33:25.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:24 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:33:25.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:24 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:33:25.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:24 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:33:25.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:24 smithi097 ceph-mon[39416]: osdmap e328: 8 total, 8 up, 8 in 2024-03-21T13:33:25.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:24 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:33:25.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:24 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4229220967' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi097-79636-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi097-79636-62"}]': finished 2024-03-21T13:33:25.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:24 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "hit_set_count","val": "2"}]': finished 2024-03-21T13:33:25.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:24 smithi097 ceph-mon[39416]: osdmap e329: 8 total, 8 up, 8 in 2024-03-21T13:33:25.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:24 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:33:25.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:24 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:33:25.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:24 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:33:25.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:24 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:33:25.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:24 smithi097 ceph-mon[30040]: osdmap e328: 8 total, 8 up, 8 in 2024-03-21T13:33:25.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:24 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:33:25.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:24 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4229220967' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi097-79636-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi097-79636-62"}]': finished 2024-03-21T13:33:25.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:24 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "hit_set_count","val": "2"}]': finished 2024-03-21T13:33:25.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:24 smithi097 ceph-mon[30040]: osdmap e329: 8 total, 8 up, 8 in 2024-03-21T13:33:25.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:24 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:33:25.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:24 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:33:26.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:26 smithi097 ceph-mon[39416]: pgmap v457: 324 pgs: 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-03-21T13:33:26.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:26 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:33:26.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:26 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:33:26.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:26 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "hit_set_period","val": "600"}]': finished 2024-03-21T13:33:26.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:26 smithi097 ceph-mon[39416]: osdmap e330: 8 total, 8 up, 8 in 2024-03-21T13:33:26.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:26 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:33:26.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:26 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:33:26.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:26 smithi097 ceph-mon[30040]: pgmap v457: 324 pgs: 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-03-21T13:33:26.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:26 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:33:26.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:26 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:33:26.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:26 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "hit_set_period","val": "600"}]': finished 2024-03-21T13:33:26.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:26 smithi097 ceph-mon[30040]: osdmap e330: 8 total, 8 up, 8 in 2024-03-21T13:33:26.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:26 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:33:26.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:26 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:33:26.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:26 smithi144 ceph-mon[33633]: pgmap v457: 324 pgs: 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-03-21T13:33:26.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:26 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:33:26.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:26 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:33:26.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:26 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "hit_set_period","val": "600"}]': finished 2024-03-21T13:33:26.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:26 smithi144 ceph-mon[33633]: osdmap e330: 8 total, 8 up, 8 in 2024-03-21T13:33:26.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:26 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:33:26.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:26 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:33:27.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:27 smithi097 ceph-mon[39416]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:33:27.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:27 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "hit_set_type","val": "bloom"}]': finished 2024-03-21T13:33:27.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:27 smithi097 ceph-mon[39416]: osdmap e331: 8 total, 8 up, 8 in 2024-03-21T13:33:27.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:27 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-21T13:33:27.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:27 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-21T13:33:27.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:27 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4229220967' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi097-79636-62"}]: dispatch 2024-03-21T13:33:27.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:27 smithi097 ceph-mon[30040]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:33:27.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:27 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "hit_set_type","val": "bloom"}]': finished 2024-03-21T13:33:27.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:27 smithi097 ceph-mon[30040]: osdmap e331: 8 total, 8 up, 8 in 2024-03-21T13:33:27.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:27 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-21T13:33:27.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:27 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-21T13:33:27.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:27 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4229220967' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi097-79636-62"}]: dispatch 2024-03-21T13:33:27.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:27 smithi144 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:33:27.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:27 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "hit_set_type","val": "bloom"}]': finished 2024-03-21T13:33:27.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:27 smithi144 ceph-mon[33633]: osdmap e331: 8 total, 8 up, 8 in 2024-03-21T13:33:27.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:27 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-21T13:33:27.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:27 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-21T13:33:27.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:27 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4229220967' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi097-79636-62"}]: dispatch 2024-03-21T13:33:28.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:28 smithi097 ceph-mon[39416]: pgmap v460: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-03-21T13:33:28.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:28 smithi097 ceph-mon[39416]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:33:28.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:28 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-21T13:33:28.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:28 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4229220967' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi097-79636-62"}]': finished 2024-03-21T13:33:28.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:28 smithi097 ceph-mon[39416]: osdmap e332: 8 total, 8 up, 8 in 2024-03-21T13:33:28.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:28 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4229220967' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi097-79636-62"}]: dispatch 2024-03-21T13:33:28.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:28 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-03-21T13:33:28.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:28 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-03-21T13:33:28.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:28 smithi097 ceph-mon[30040]: pgmap v460: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-03-21T13:33:28.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:28 smithi097 ceph-mon[30040]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:33:28.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:28 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-21T13:33:28.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:28 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4229220967' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi097-79636-62"}]': finished 2024-03-21T13:33:28.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:28 smithi097 ceph-mon[30040]: osdmap e332: 8 total, 8 up, 8 in 2024-03-21T13:33:28.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:28 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4229220967' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi097-79636-62"}]: dispatch 2024-03-21T13:33:28.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:28 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-03-21T13:33:28.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:28 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-03-21T13:33:28.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:28 smithi144 ceph-mon[33633]: pgmap v460: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-03-21T13:33:28.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:28 smithi144 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:33:28.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:28 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-21T13:33:28.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:28 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4229220967' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi097-79636-62"}]': finished 2024-03-21T13:33:28.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:28 smithi144 ceph-mon[33633]: osdmap e332: 8 total, 8 up, 8 in 2024-03-21T13:33:28.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:28 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4229220967' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi097-79636-62"}]: dispatch 2024-03-21T13:33:28.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:28 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-03-21T13:33:28.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:28 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-03-21T13:33:29.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:29 smithi144 ceph-mon[33633]: pgmap v463: 324 pgs: 324 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:33:29.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:29 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4229220967' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi097-79636-62"}]': finished 2024-03-21T13:33:29.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:29 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-03-21T13:33:29.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:29 smithi144 ceph-mon[33633]: osdmap e333: 8 total, 8 up, 8 in 2024-03-21T13:33:29.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:29 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-03-21T13:33:29.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:29 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-03-21T13:33:29.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:29 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/703233033' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi097-79636-63"}]: dispatch 2024-03-21T13:33:29.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:29 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi097-79636-63"}]: dispatch 2024-03-21T13:33:29.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:29 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/703233033' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi097-79636-63"}]: dispatch 2024-03-21T13:33:29.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:29 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi097-79636-63"}]: dispatch 2024-03-21T13:33:29.873 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:29 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/703233033' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi097-79636-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:33:29.873 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:29 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi097-79636-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:33:30.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:29 smithi097 ceph-mon[39416]: pgmap v463: 324 pgs: 324 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:33:30.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:29 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4229220967' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi097-79636-62"}]': finished 2024-03-21T13:33:30.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:29 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-03-21T13:33:30.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:29 smithi097 ceph-mon[39416]: osdmap e333: 8 total, 8 up, 8 in 2024-03-21T13:33:30.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:29 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-03-21T13:33:30.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:29 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-03-21T13:33:30.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:29 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/703233033' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi097-79636-63"}]: dispatch 2024-03-21T13:33:30.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:29 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi097-79636-63"}]: dispatch 2024-03-21T13:33:30.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:29 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/703233033' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi097-79636-63"}]: dispatch 2024-03-21T13:33:30.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:29 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi097-79636-63"}]: dispatch 2024-03-21T13:33:30.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:29 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/703233033' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi097-79636-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:33:30.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:29 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi097-79636-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:33:30.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:29 smithi097 ceph-mon[30040]: pgmap v463: 324 pgs: 324 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:33:30.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:29 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4229220967' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi097-79636-62"}]': finished 2024-03-21T13:33:30.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:29 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-03-21T13:33:30.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:29 smithi097 ceph-mon[30040]: osdmap e333: 8 total, 8 up, 8 in 2024-03-21T13:33:30.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:29 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-03-21T13:33:30.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:29 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-03-21T13:33:30.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:29 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/703233033' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi097-79636-63"}]: dispatch 2024-03-21T13:33:30.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:29 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi097-79636-63"}]: dispatch 2024-03-21T13:33:30.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:29 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/703233033' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi097-79636-63"}]: dispatch 2024-03-21T13:33:30.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:29 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi097-79636-63"}]: dispatch 2024-03-21T13:33:30.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:29 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/703233033' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi097-79636-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:33:30.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:29 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi097-79636-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:33:30.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:30 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-03-21T13:33:30.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:30 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi097-79636-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:33:30.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:30 smithi144 ceph-mon[33633]: osdmap e334: 8 total, 8 up, 8 in 2024-03-21T13:33:30.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:30 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/703233033' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi097-79636-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi097-79636-63"}]: dispatch 2024-03-21T13:33:30.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:30 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi097-79636-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi097-79636-63"}]: dispatch 2024-03-21T13:33:30.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:30 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:33:30.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:30 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:33:31.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:30 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-03-21T13:33:31.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:30 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi097-79636-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:33:31.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:30 smithi097 ceph-mon[39416]: osdmap e334: 8 total, 8 up, 8 in 2024-03-21T13:33:31.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:30 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/703233033' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi097-79636-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi097-79636-63"}]: dispatch 2024-03-21T13:33:31.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:30 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi097-79636-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi097-79636-63"}]: dispatch 2024-03-21T13:33:31.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:30 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:33:31.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:30 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:33:31.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:30 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-03-21T13:33:31.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:30 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi097-79636-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:33:31.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:30 smithi097 ceph-mon[30040]: osdmap e334: 8 total, 8 up, 8 in 2024-03-21T13:33:31.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:30 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/703233033' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi097-79636-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi097-79636-63"}]: dispatch 2024-03-21T13:33:31.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:30 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi097-79636-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi097-79636-63"}]: dispatch 2024-03-21T13:33:31.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:30 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:33:31.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:30 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:33:31.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:31 smithi144 ceph-mon[33633]: pgmap v466: 324 pgs: 324 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:33:31.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:31 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:33:31.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:31 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-47"}]: dispatch 2024-03-21T13:33:31.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:31 smithi144 ceph-mon[33633]: osdmap e335: 8 total, 8 up, 8 in 2024-03-21T13:33:31.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:31 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-47"}]: dispatch 2024-03-21T13:33:31.949 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:31 smithi097 ceph-mon[39416]: pgmap v466: 324 pgs: 324 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:33:31.949 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:31 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:33:31.949 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:31 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-47"}]: dispatch 2024-03-21T13:33:31.949 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:31 smithi097 ceph-mon[39416]: osdmap e335: 8 total, 8 up, 8 in 2024-03-21T13:33:31.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:31 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-47"}]: dispatch 2024-03-21T13:33:31.950 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:31 smithi097 ceph-mon[30040]: pgmap v466: 324 pgs: 324 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:33:31.950 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:31 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:33:31.950 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:31 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-47"}]: dispatch 2024-03-21T13:33:31.950 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:31 smithi097 ceph-mon[30040]: osdmap e335: 8 total, 8 up, 8 in 2024-03-21T13:33:31.950 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:31 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-47"}]: dispatch 2024-03-21T13:33:32.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:33:31 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:33:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:33:33.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:32 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi097-79636-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi097-79636-63"}]': finished 2024-03-21T13:33:33.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:32 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-47"}]': finished 2024-03-21T13:33:33.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:32 smithi097 ceph-mon[39416]: osdmap e336: 8 total, 8 up, 8 in 2024-03-21T13:33:33.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:32 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:33:33.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:32 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:33:33.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:32 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-47"}]: dispatch 2024-03-21T13:33:33.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:32 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-47"}]: dispatch 2024-03-21T13:33:33.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:32 smithi097 ceph-mon[39416]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:33:33.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:32 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi097-79636-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi097-79636-63"}]': finished 2024-03-21T13:33:33.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:32 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-47"}]': finished 2024-03-21T13:33:33.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:32 smithi097 ceph-mon[30040]: osdmap e336: 8 total, 8 up, 8 in 2024-03-21T13:33:33.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:32 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:33:33.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:32 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:33:33.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:32 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-47"}]: dispatch 2024-03-21T13:33:33.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:32 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-47"}]: dispatch 2024-03-21T13:33:33.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:32 smithi097 ceph-mon[30040]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:33:33.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:32 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi097-79636-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi097-79636-63"}]': finished 2024-03-21T13:33:33.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:32 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-47"}]': finished 2024-03-21T13:33:33.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:32 smithi144 ceph-mon[33633]: osdmap e336: 8 total, 8 up, 8 in 2024-03-21T13:33:33.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:32 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:33:33.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:32 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:33:33.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:32 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-47"}]: dispatch 2024-03-21T13:33:33.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:32 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-47"}]: dispatch 2024-03-21T13:33:33.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:32 smithi144 ceph-mon[33633]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:33:34.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:33 smithi097 ceph-mon[39416]: pgmap v469: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:33:34.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:33 smithi097 ceph-mon[39416]: osdmap e337: 8 total, 8 up, 8 in 2024-03-21T13:33:34.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:33 smithi097 ceph-mon[30040]: pgmap v469: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:33:34.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:33 smithi097 ceph-mon[30040]: osdmap e337: 8 total, 8 up, 8 in 2024-03-21T13:33:34.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:33 smithi144 ceph-mon[33633]: pgmap v469: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:33:34.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:33 smithi144 ceph-mon[33633]: osdmap e337: 8 total, 8 up, 8 in 2024-03-21T13:33:34.121 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:33:33 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:33:35.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:34 smithi097 ceph-mon[39416]: osdmap e338: 8 total, 8 up, 8 in 2024-03-21T13:33:35.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:34 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:33:35.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:34 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/703233033' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi097-79636-63"}]: dispatch 2024-03-21T13:33:35.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:34 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:33:35.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:34 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi097-79636-63"}]: dispatch 2024-03-21T13:33:35.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:34 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:33:35.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:34 smithi097 ceph-mon[30040]: osdmap e338: 8 total, 8 up, 8 in 2024-03-21T13:33:35.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:34 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:33:35.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:34 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/703233033' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi097-79636-63"}]: dispatch 2024-03-21T13:33:35.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:34 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:33:35.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:34 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi097-79636-63"}]: dispatch 2024-03-21T13:33:35.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:34 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:33:35.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:34 smithi144 ceph-mon[33633]: osdmap e338: 8 total, 8 up, 8 in 2024-03-21T13:33:35.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:34 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:33:35.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:34 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/703233033' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi097-79636-63"}]: dispatch 2024-03-21T13:33:35.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:34 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:33:35.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:34 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi097-79636-63"}]: dispatch 2024-03-21T13:33:35.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:34 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:33:36.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:35 smithi097 ceph-mon[39416]: pgmap v472: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-21T13:33:36.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:35 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:33:36.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:35 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi097-79636-63"}]': finished 2024-03-21T13:33:36.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:35 smithi097 ceph-mon[39416]: osdmap e339: 8 total, 8 up, 8 in 2024-03-21T13:33:36.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:35 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/703233033' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi097-79636-63"}]: dispatch 2024-03-21T13:33:36.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:35 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi097-79636-63"}]: dispatch 2024-03-21T13:33:36.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:35 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:33:36.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:35 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:33:36.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:35 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:33:36.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:35 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:33:36.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:35 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:33:36.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:35 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:33:36.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:35 smithi097 ceph-mon[30040]: pgmap v472: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-21T13:33:36.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:35 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:33:36.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:35 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi097-79636-63"}]': finished 2024-03-21T13:33:36.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:35 smithi097 ceph-mon[30040]: osdmap e339: 8 total, 8 up, 8 in 2024-03-21T13:33:36.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:35 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/703233033' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi097-79636-63"}]: dispatch 2024-03-21T13:33:36.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:35 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi097-79636-63"}]: dispatch 2024-03-21T13:33:36.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:35 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:33:36.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:35 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:33:36.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:35 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:33:36.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:35 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:33:36.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:35 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:33:36.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:35 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:33:36.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:35 smithi144 ceph-mon[33633]: pgmap v472: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-21T13:33:36.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:35 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:33:36.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:35 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi097-79636-63"}]': finished 2024-03-21T13:33:36.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:35 smithi144 ceph-mon[33633]: osdmap e339: 8 total, 8 up, 8 in 2024-03-21T13:33:36.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:35 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/703233033' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi097-79636-63"}]: dispatch 2024-03-21T13:33:36.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:35 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi097-79636-63"}]: dispatch 2024-03-21T13:33:36.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:35 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:33:36.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:35 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:33:36.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:35 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:33:36.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:35 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:33:36.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:35 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:33:36.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:35 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:33:37.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:36 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi097-79636-63"}]': finished 2024-03-21T13:33:37.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:36 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-49", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:33:37.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:36 smithi097 ceph-mon[39416]: osdmap e340: 8 total, 8 up, 8 in 2024-03-21T13:33:37.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:36 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-49"}]: dispatch 2024-03-21T13:33:37.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:36 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-49"}]: dispatch 2024-03-21T13:33:37.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:36 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3520340963' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi097-79636-64"}]: dispatch 2024-03-21T13:33:37.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:36 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3520340963' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi097-79636-64"}]: dispatch 2024-03-21T13:33:37.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:36 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3520340963' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi097-79636-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:33:37.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:36 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi097-79636-63"}]': finished 2024-03-21T13:33:37.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:36 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-49", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:33:37.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:36 smithi097 ceph-mon[30040]: osdmap e340: 8 total, 8 up, 8 in 2024-03-21T13:33:37.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:36 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-49"}]: dispatch 2024-03-21T13:33:37.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:36 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-49"}]: dispatch 2024-03-21T13:33:37.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:36 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3520340963' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi097-79636-64"}]: dispatch 2024-03-21T13:33:37.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:36 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3520340963' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi097-79636-64"}]: dispatch 2024-03-21T13:33:37.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:36 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3520340963' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi097-79636-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:33:37.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:36 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi097-79636-63"}]': finished 2024-03-21T13:33:37.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:36 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-49", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:33:37.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:36 smithi144 ceph-mon[33633]: osdmap e340: 8 total, 8 up, 8 in 2024-03-21T13:33:37.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:36 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-49"}]: dispatch 2024-03-21T13:33:37.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:36 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-49"}]: dispatch 2024-03-21T13:33:37.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:36 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3520340963' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi097-79636-64"}]: dispatch 2024-03-21T13:33:37.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:36 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3520340963' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi097-79636-64"}]: dispatch 2024-03-21T13:33:37.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:36 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3520340963' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi097-79636-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:33:38.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:37 smithi097 ceph-mon[30040]: pgmap v475: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-21T13:33:38.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:37 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-49"}]': finished 2024-03-21T13:33:38.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:37 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3520340963' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi097-79636-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:33:38.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:37 smithi097 ceph-mon[30040]: osdmap e341: 8 total, 8 up, 8 in 2024-03-21T13:33:38.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:37 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-49", "mode": "readproxy"}]: dispatch 2024-03-21T13:33:38.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:37 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3520340963' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi097-79636-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi097-79636-64"}]: dispatch 2024-03-21T13:33:38.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:37 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-49", "mode": "readproxy"}]: dispatch 2024-03-21T13:33:38.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:37 smithi097 ceph-mon[39416]: pgmap v475: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-21T13:33:38.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:37 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-49"}]': finished 2024-03-21T13:33:38.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:37 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3520340963' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi097-79636-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:33:38.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:37 smithi097 ceph-mon[39416]: osdmap e341: 8 total, 8 up, 8 in 2024-03-21T13:33:38.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:37 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-49", "mode": "readproxy"}]: dispatch 2024-03-21T13:33:38.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:37 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3520340963' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi097-79636-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi097-79636-64"}]: dispatch 2024-03-21T13:33:38.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:37 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-49", "mode": "readproxy"}]: dispatch 2024-03-21T13:33:38.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:37 smithi144 ceph-mon[33633]: pgmap v475: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-21T13:33:38.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:37 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-49"}]': finished 2024-03-21T13:33:38.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:37 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3520340963' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi097-79636-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:33:38.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:37 smithi144 ceph-mon[33633]: osdmap e341: 8 total, 8 up, 8 in 2024-03-21T13:33:38.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:37 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-49", "mode": "readproxy"}]: dispatch 2024-03-21T13:33:38.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:37 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3520340963' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi097-79636-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi097-79636-64"}]: dispatch 2024-03-21T13:33:38.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:37 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-49", "mode": "readproxy"}]: dispatch 2024-03-21T13:33:39.204 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:38 smithi097 ceph-mon[39416]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:33:39.204 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:38 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-49", "mode": "readproxy"}]': finished 2024-03-21T13:33:39.204 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:38 smithi097 ceph-mon[39416]: osdmap e342: 8 total, 8 up, 8 in 2024-03-21T13:33:39.204 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:38 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3520340963' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi097-79636-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi097-79636-64"}]': finished 2024-03-21T13:33:39.205 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:38 smithi097 ceph-mon[39416]: osdmap e343: 8 total, 8 up, 8 in 2024-03-21T13:33:39.205 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:38 smithi097 ceph-mon[30040]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:33:39.205 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:38 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-49", "mode": "readproxy"}]': finished 2024-03-21T13:33:39.205 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:38 smithi097 ceph-mon[30040]: osdmap e342: 8 total, 8 up, 8 in 2024-03-21T13:33:39.205 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:38 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3520340963' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi097-79636-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi097-79636-64"}]': finished 2024-03-21T13:33:39.205 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:38 smithi097 ceph-mon[30040]: osdmap e343: 8 total, 8 up, 8 in 2024-03-21T13:33:39.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:38 smithi144 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:33:39.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:38 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-49", "mode": "readproxy"}]': finished 2024-03-21T13:33:39.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:38 smithi144 ceph-mon[33633]: osdmap e342: 8 total, 8 up, 8 in 2024-03-21T13:33:39.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:38 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3520340963' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi097-79636-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi097-79636-64"}]': finished 2024-03-21T13:33:39.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:38 smithi144 ceph-mon[33633]: osdmap e343: 8 total, 8 up, 8 in 2024-03-21T13:33:40.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:39 smithi097 ceph-mon[39416]: pgmap v478: 324 pgs: 324 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:33:40.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:39 smithi097 ceph-mon[39416]: osdmap e344: 8 total, 8 up, 8 in 2024-03-21T13:33:40.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:39 smithi097 ceph-mon[30040]: pgmap v478: 324 pgs: 324 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:33:40.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:39 smithi097 ceph-mon[30040]: osdmap e344: 8 total, 8 up, 8 in 2024-03-21T13:33:40.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:39 smithi144 ceph-mon[33633]: pgmap v478: 324 pgs: 324 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:33:40.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:39 smithi144 ceph-mon[33633]: osdmap e344: 8 total, 8 up, 8 in 2024-03-21T13:33:41.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:40 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:33:41.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:40 smithi097 ceph-mon[39416]: osdmap e345: 8 total, 8 up, 8 in 2024-03-21T13:33:41.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:40 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3520340963' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi097-79636-64"}]: dispatch 2024-03-21T13:33:41.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:40 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:33:41.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:40 smithi097 ceph-mon[30040]: osdmap e345: 8 total, 8 up, 8 in 2024-03-21T13:33:41.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:40 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3520340963' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi097-79636-64"}]: dispatch 2024-03-21T13:33:41.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:40 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:33:41.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:40 smithi144 ceph-mon[33633]: osdmap e345: 8 total, 8 up, 8 in 2024-03-21T13:33:41.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:40 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3520340963' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi097-79636-64"}]: dispatch 2024-03-21T13:33:42.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:33:41 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:33:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:33:42.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:41 smithi097 ceph-mon[39416]: pgmap v481: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:33:42.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:41 smithi097 ceph-mon[39416]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:33:42.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:41 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3520340963' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi097-79636-64"}]': finished 2024-03-21T13:33:42.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:41 smithi097 ceph-mon[39416]: osdmap e346: 8 total, 8 up, 8 in 2024-03-21T13:33:42.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:41 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3520340963' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi097-79636-64"}]: dispatch 2024-03-21T13:33:42.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:41 smithi097 ceph-mon[30040]: pgmap v481: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:33:42.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:41 smithi097 ceph-mon[30040]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:33:42.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:41 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3520340963' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi097-79636-64"}]': finished 2024-03-21T13:33:42.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:41 smithi097 ceph-mon[30040]: osdmap e346: 8 total, 8 up, 8 in 2024-03-21T13:33:42.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:41 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3520340963' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi097-79636-64"}]: dispatch 2024-03-21T13:33:42.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:41 smithi144 ceph-mon[33633]: pgmap v481: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:33:42.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:41 smithi144 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:33:42.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:41 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3520340963' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi097-79636-64"}]': finished 2024-03-21T13:33:42.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:41 smithi144 ceph-mon[33633]: osdmap e346: 8 total, 8 up, 8 in 2024-03-21T13:33:42.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:41 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3520340963' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi097-79636-64"}]: dispatch 2024-03-21T13:33:44.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:43 smithi097 ceph-mon[30040]: pgmap v484: 324 pgs: 324 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:33:44.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:43 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3520340963' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi097-79636-64"}]': finished 2024-03-21T13:33:44.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:43 smithi097 ceph-mon[30040]: osdmap e347: 8 total, 8 up, 8 in 2024-03-21T13:33:44.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:43 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3108214073' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi097-79636-65"}]: dispatch 2024-03-21T13:33:44.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:43 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi097-79636-65"}]: dispatch 2024-03-21T13:33:44.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:43 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3108214073' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi097-79636-65"}]: dispatch 2024-03-21T13:33:44.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:43 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi097-79636-65"}]: dispatch 2024-03-21T13:33:44.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:43 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3108214073' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi097-79636-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:33:44.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:43 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi097-79636-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:33:44.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:43 smithi097 ceph-mon[39416]: pgmap v484: 324 pgs: 324 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:33:44.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:43 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3520340963' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi097-79636-64"}]': finished 2024-03-21T13:33:44.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:43 smithi097 ceph-mon[39416]: osdmap e347: 8 total, 8 up, 8 in 2024-03-21T13:33:44.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:43 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3108214073' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi097-79636-65"}]: dispatch 2024-03-21T13:33:44.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:43 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi097-79636-65"}]: dispatch 2024-03-21T13:33:44.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:43 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3108214073' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi097-79636-65"}]: dispatch 2024-03-21T13:33:44.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:43 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi097-79636-65"}]: dispatch 2024-03-21T13:33:44.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:43 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3108214073' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi097-79636-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:33:44.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:43 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi097-79636-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:33:44.121 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:33:43 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:33:44.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:43 smithi144 ceph-mon[33633]: pgmap v484: 324 pgs: 324 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:33:44.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:43 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3520340963' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi097-79636-64"}]': finished 2024-03-21T13:33:44.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:43 smithi144 ceph-mon[33633]: osdmap e347: 8 total, 8 up, 8 in 2024-03-21T13:33:44.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:43 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3108214073' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi097-79636-65"}]: dispatch 2024-03-21T13:33:44.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:43 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi097-79636-65"}]: dispatch 2024-03-21T13:33:44.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:43 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3108214073' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi097-79636-65"}]: dispatch 2024-03-21T13:33:44.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:43 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi097-79636-65"}]: dispatch 2024-03-21T13:33:44.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:43 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3108214073' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi097-79636-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:33:44.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:43 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi097-79636-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:33:45.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:44 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:33:45.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:44 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi097-79636-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:33:45.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:44 smithi097 ceph-mon[30040]: osdmap e348: 8 total, 8 up, 8 in 2024-03-21T13:33:45.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:44 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3108214073' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi097-79636-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi097-79636-65"}]: dispatch 2024-03-21T13:33:45.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:44 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi097-79636-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi097-79636-65"}]: dispatch 2024-03-21T13:33:45.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:44 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:33:45.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:44 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi097-79636-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:33:45.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:44 smithi097 ceph-mon[39416]: osdmap e348: 8 total, 8 up, 8 in 2024-03-21T13:33:45.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:44 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3108214073' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi097-79636-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi097-79636-65"}]: dispatch 2024-03-21T13:33:45.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:44 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi097-79636-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi097-79636-65"}]: dispatch 2024-03-21T13:33:45.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:44 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:33:45.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:44 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi097-79636-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:33:45.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:44 smithi144 ceph-mon[33633]: osdmap e348: 8 total, 8 up, 8 in 2024-03-21T13:33:45.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:44 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3108214073' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi097-79636-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi097-79636-65"}]: dispatch 2024-03-21T13:33:45.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:44 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi097-79636-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi097-79636-65"}]: dispatch 2024-03-21T13:33:46.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:45 smithi097 ceph-mon[39416]: pgmap v487: 324 pgs: 324 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:33:46.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:45 smithi097 ceph-mon[39416]: osdmap e349: 8 total, 8 up, 8 in 2024-03-21T13:33:46.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:45 smithi097 ceph-mon[30040]: pgmap v487: 324 pgs: 324 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:33:46.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:45 smithi097 ceph-mon[30040]: osdmap e349: 8 total, 8 up, 8 in 2024-03-21T13:33:46.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:45 smithi144 ceph-mon[33633]: pgmap v487: 324 pgs: 324 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:33:46.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:45 smithi144 ceph-mon[33633]: osdmap e349: 8 total, 8 up, 8 in 2024-03-21T13:33:47.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:46 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi097-79636-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi097-79636-65"}]': finished 2024-03-21T13:33:47.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:46 smithi097 ceph-mon[39416]: osdmap e350: 8 total, 8 up, 8 in 2024-03-21T13:33:47.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:46 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi097-79636-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi097-79636-65"}]': finished 2024-03-21T13:33:47.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:46 smithi097 ceph-mon[30040]: osdmap e350: 8 total, 8 up, 8 in 2024-03-21T13:33:47.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:46 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi097-79636-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi097-79636-65"}]': finished 2024-03-21T13:33:47.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:46 smithi144 ceph-mon[33633]: osdmap e350: 8 total, 8 up, 8 in 2024-03-21T13:33:48.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:47 smithi097 ceph-mon[39416]: pgmap v490: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:33:48.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:47 smithi097 ceph-mon[39416]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:33:48.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:47 smithi097 ceph-mon[39416]: osdmap e351: 8 total, 8 up, 8 in 2024-03-21T13:33:48.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:47 smithi097 ceph-mon[30040]: pgmap v490: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:33:48.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:47 smithi097 ceph-mon[30040]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:33:48.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:47 smithi097 ceph-mon[30040]: osdmap e351: 8 total, 8 up, 8 in 2024-03-21T13:33:48.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:47 smithi144 ceph-mon[33633]: pgmap v490: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:33:48.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:47 smithi144 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:33:48.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:47 smithi144 ceph-mon[33633]: osdmap e351: 8 total, 8 up, 8 in 2024-03-21T13:33:49.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:48 smithi097 ceph-mon[39416]: osdmap e352: 8 total, 8 up, 8 in 2024-03-21T13:33:49.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:48 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3108214073' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi097-79636-65"}]: dispatch 2024-03-21T13:33:49.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:48 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi097-79636-65"}]: dispatch 2024-03-21T13:33:49.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:48 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:33:49.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:48 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:33:49.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:48 smithi097 ceph-mon[30040]: osdmap e352: 8 total, 8 up, 8 in 2024-03-21T13:33:49.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:48 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3108214073' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi097-79636-65"}]: dispatch 2024-03-21T13:33:49.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:48 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi097-79636-65"}]: dispatch 2024-03-21T13:33:49.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:48 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:33:49.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:48 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:33:49.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:48 smithi144 ceph-mon[33633]: osdmap e352: 8 total, 8 up, 8 in 2024-03-21T13:33:49.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:48 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3108214073' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi097-79636-65"}]: dispatch 2024-03-21T13:33:49.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:48 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi097-79636-65"}]: dispatch 2024-03-21T13:33:49.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:48 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:33:49.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:48 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:33:50.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:49 smithi097 ceph-mon[39416]: pgmap v493: 324 pgs: 324 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:33:50.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:49 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi097-79636-65"}]': finished 2024-03-21T13:33:50.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:49 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:33:50.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:49 smithi097 ceph-mon[39416]: osdmap e353: 8 total, 8 up, 8 in 2024-03-21T13:33:50.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:49 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3108214073' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi097-79636-65"}]: dispatch 2024-03-21T13:33:50.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:49 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-49"}]: dispatch 2024-03-21T13:33:50.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:49 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi097-79636-65"}]: dispatch 2024-03-21T13:33:50.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:49 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-49"}]: dispatch 2024-03-21T13:33:50.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:49 smithi097 ceph-mon[30040]: pgmap v493: 324 pgs: 324 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:33:50.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:49 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi097-79636-65"}]': finished 2024-03-21T13:33:50.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:49 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:33:50.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:49 smithi097 ceph-mon[30040]: osdmap e353: 8 total, 8 up, 8 in 2024-03-21T13:33:50.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:49 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3108214073' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi097-79636-65"}]: dispatch 2024-03-21T13:33:50.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:49 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-49"}]: dispatch 2024-03-21T13:33:50.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:49 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi097-79636-65"}]: dispatch 2024-03-21T13:33:50.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:49 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-49"}]: dispatch 2024-03-21T13:33:50.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:49 smithi144 ceph-mon[33633]: pgmap v493: 324 pgs: 324 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:33:50.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:49 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi097-79636-65"}]': finished 2024-03-21T13:33:50.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:49 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:33:50.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:49 smithi144 ceph-mon[33633]: osdmap e353: 8 total, 8 up, 8 in 2024-03-21T13:33:50.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:49 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3108214073' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi097-79636-65"}]: dispatch 2024-03-21T13:33:50.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:49 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-49"}]: dispatch 2024-03-21T13:33:50.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:49 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi097-79636-65"}]: dispatch 2024-03-21T13:33:50.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:49 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-49"}]: dispatch 2024-03-21T13:33:50.741 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP263:head 2024-03-21T13:33:50.742 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:5d165639:::164:head 2024-03-21T13:33:50.742 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:f43765fc:::165:head 2024-03-21T13:33:50.742 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:b4c720e9:::166:head 2024-03-21T13:33:50.742 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:e694b040:::167:head 2024-03-21T13:33:50.742 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:afa38db2:::168:head 2024-03-21T13:33:50.742 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:77ba9f53:::169:head 2024-03-21T13:33:50.742 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:87495034:::170:head 2024-03-21T13:33:50.742 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:7c96bf0e:::171:head 2024-03-21T13:33:50.742 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:dbe346cc:::172:head 2024-03-21T13:33:50.742 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:e943ec24:::173:head 2024-03-21T13:33:50.742 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:f97a9c0c:::174:head 2024-03-21T13:33:50.742 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:6f26e74d:::175:head 2024-03-21T13:33:50.743 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:4f95e106:::176:head 2024-03-21T13:33:50.743 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:0e6f2f8f:::177:head 2024-03-21T13:33:50.743 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:05db05f1:::178:head 2024-03-21T13:33:50.743 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:38a78d66:::179:head 2024-03-21T13:33:50.743 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:d095610b:::180:head 2024-03-21T13:33:50.743 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:a1a9d709:::181:head 2024-03-21T13:33:50.743 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:1e5d39db:::182:head 2024-03-21T13:33:50.743 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:f7df4fb9:::183:head 2024-03-21T13:33:50.743 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:03a7f161:::184:head 2024-03-21T13:33:50.743 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:ba70721e:::185:head 2024-03-21T13:33:50.743 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:28e5662d:::186:head 2024-03-21T13:33:50.743 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:973d52de:::187:head 2024-03-21T13:33:50.743 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:4303eb1c:::188:head 2024-03-21T13:33:50.743 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:b990b48e:::189:head 2024-03-21T13:33:50.744 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:29b8165b:::190:head 2024-03-21T13:33:50.744 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:3547f197:::191:head 2024-03-21T13:33:50.744 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:7e260936:::192:head 2024-03-21T13:33:50.744 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:1abec7b1:::193:head 2024-03-21T13:33:50.744 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:10fdda93:::194:head 2024-03-21T13:33:50.744 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:15817eea:::195:head 2024-03-21T13:33:50.744 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:770bab57:::196:head 2024-03-21T13:33:50.744 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:ed9e13e7:::197:head 2024-03-21T13:33:50.744 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:71471a8f:::198:head 2024-03-21T13:33:50.744 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: checking for 270:10fb1d02:::199:head 2024-03-21T13:33:50.744 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetWrite (8045 ms) 2024-03-21T13:33:50.744 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetTrim 2024-03-21T13:33:50.744 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: got ls 1711027986,0 2024-03-21T13:33:50.744 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: first is 1711027986 2024-03-21T13:33:50.744 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: got ls 1711027986,0 2024-03-21T13:33:50.744 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: got ls 1711027986,0 2024-03-21T13:33:50.745 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: got ls 1711027986,1711027988,1711027989,0 2024-03-21T13:33:50.745 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: got ls 1711027986,1711027988,1711027989,0 2024-03-21T13:33:50.745 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: got ls 1711027986,1711027988,1711027989,0 2024-03-21T13:33:50.745 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: got ls 1711027986,1711027988,1711027989,1711027991,1711027992,0 2024-03-21T13:33:50.745 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: got ls 1711027986,1711027988,1711027989,1711027991,1711027992,0 2024-03-21T13:33:50.745 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: got ls 1711027986,1711027988,1711027989,1711027991,1711027992,0 2024-03-21T13:33:50.745 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: got ls 1711027986,1711027988,1711027989,1711027991,1711027992,1711027994,1711027995,0 2024-03-21T13:33:50.745 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: got ls 1711027986,1711027988,1711027989,1711027991,1711027992,1711027994,1711027995,0 2024-03-21T13:33:50.745 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: got ls 1711027986,1711027988,1711027989,1711027991,1711027992,1711027994,1711027995,0 2024-03-21T13:33:50.745 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: got ls 1711027989,1711027991,1711027992,1711027994,1711027995,1711027997,1711027998,0 2024-03-21T13:33:50.745 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: first now 1711027989, trimmed 2024-03-21T13:33:50.745 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetTrim (20148 ms) 2024-03-21T13:33:50.745 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteOn2ndRead 2024-03-21T13:33:50.745 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: foo0 2024-03-21T13:33:50.746 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-03-21T13:33:50.746 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteOn2ndRead (14065 ms) 2024-03-21T13:33:50.746 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ProxyRead 2024-03-21T13:33:51.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:50 smithi097 ceph-mon[39416]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:33:51.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:50 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi097-79636-65"}]': finished 2024-03-21T13:33:51.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:50 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-49"}]': finished 2024-03-21T13:33:51.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:50 smithi097 ceph-mon[39416]: osdmap e354: 8 total, 8 up, 8 in 2024-03-21T13:33:51.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:50 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3715338659' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi097-79636-66"}]: dispatch 2024-03-21T13:33:51.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:50 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3715338659' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi097-79636-66"}]: dispatch 2024-03-21T13:33:51.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:50 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3715338659' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi097-79636-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:33:51.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:50 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:33:51.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:50 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:33:51.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:50 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-49"}]: dispatch 2024-03-21T13:33:51.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:50 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-49"}]: dispatch 2024-03-21T13:33:51.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:50 smithi097 ceph-mon[30040]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:33:51.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:50 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi097-79636-65"}]': finished 2024-03-21T13:33:51.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:50 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-49"}]': finished 2024-03-21T13:33:51.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:50 smithi097 ceph-mon[30040]: osdmap e354: 8 total, 8 up, 8 in 2024-03-21T13:33:51.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:50 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3715338659' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi097-79636-66"}]: dispatch 2024-03-21T13:33:51.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:50 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3715338659' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi097-79636-66"}]: dispatch 2024-03-21T13:33:51.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:50 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3715338659' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi097-79636-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:33:51.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:50 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:33:51.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:50 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:33:51.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:50 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-49"}]: dispatch 2024-03-21T13:33:51.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:50 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-49"}]: dispatch 2024-03-21T13:33:51.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:50 smithi144 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:33:51.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:50 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi097-79636-65"}]': finished 2024-03-21T13:33:51.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:50 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-49"}]': finished 2024-03-21T13:33:51.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:50 smithi144 ceph-mon[33633]: osdmap e354: 8 total, 8 up, 8 in 2024-03-21T13:33:51.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:50 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3715338659' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi097-79636-66"}]: dispatch 2024-03-21T13:33:51.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:50 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3715338659' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi097-79636-66"}]: dispatch 2024-03-21T13:33:51.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:50 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3715338659' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi097-79636-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:33:51.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:50 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:33:51.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:50 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:33:51.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:50 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-49"}]: dispatch 2024-03-21T13:33:51.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:50 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-49"}]: dispatch 2024-03-21T13:33:52.053 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:33:51 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:33:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:33:52.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:51 smithi097 ceph-mon[30040]: pgmap v496: 324 pgs: 324 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:33:52.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:51 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3715338659' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi097-79636-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:33:52.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:51 smithi097 ceph-mon[30040]: osdmap e355: 8 total, 8 up, 8 in 2024-03-21T13:33:52.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:51 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3715338659' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi097-79636-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi097-79636-66"}]: dispatch 2024-03-21T13:33:52.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:51 smithi097 ceph-mon[39416]: pgmap v496: 324 pgs: 324 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:33:52.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:51 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3715338659' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi097-79636-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:33:52.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:51 smithi097 ceph-mon[39416]: osdmap e355: 8 total, 8 up, 8 in 2024-03-21T13:33:52.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:51 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3715338659' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi097-79636-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi097-79636-66"}]: dispatch 2024-03-21T13:33:52.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:51 smithi144 ceph-mon[33633]: pgmap v496: 324 pgs: 324 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:33:52.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:51 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3715338659' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi097-79636-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:33:52.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:51 smithi144 ceph-mon[33633]: osdmap e355: 8 total, 8 up, 8 in 2024-03-21T13:33:52.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:51 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3715338659' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi097-79636-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi097-79636-66"}]: dispatch 2024-03-21T13:33:53.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:52 smithi097 ceph-mon[39416]: osdmap e356: 8 total, 8 up, 8 in 2024-03-21T13:33:53.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:52 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:33:53.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:52 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:33:53.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:52 smithi097 ceph-mon[39416]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:33:53.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:52 smithi097 ceph-mon[30040]: osdmap e356: 8 total, 8 up, 8 in 2024-03-21T13:33:53.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:52 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:33:53.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:52 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:33:53.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:52 smithi097 ceph-mon[30040]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:33:53.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:52 smithi144 ceph-mon[33633]: osdmap e356: 8 total, 8 up, 8 in 2024-03-21T13:33:53.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:52 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:33:53.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:52 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:33:53.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:52 smithi144 ceph-mon[33633]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:33:54.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:53 smithi097 ceph-mon[39416]: pgmap v499: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:33:54.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:53 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3715338659' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi097-79636-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi097-79636-66"}]': finished 2024-03-21T13:33:54.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:53 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:33:54.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:53 smithi097 ceph-mon[39416]: osdmap e357: 8 total, 8 up, 8 in 2024-03-21T13:33:54.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:53 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:33:54.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:53 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:33:54.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:53 smithi097 ceph-mon[30040]: pgmap v499: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:33:54.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:53 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3715338659' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi097-79636-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi097-79636-66"}]': finished 2024-03-21T13:33:54.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:53 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:33:54.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:53 smithi097 ceph-mon[30040]: osdmap e357: 8 total, 8 up, 8 in 2024-03-21T13:33:54.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:53 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:33:54.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:53 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:33:54.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:53 smithi144 ceph-mon[33633]: pgmap v499: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:33:54.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:53 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3715338659' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi097-79636-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi097-79636-66"}]': finished 2024-03-21T13:33:54.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:53 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:33:54.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:53 smithi144 ceph-mon[33633]: osdmap e357: 8 total, 8 up, 8 in 2024-03-21T13:33:54.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:53 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:33:54.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:53 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:33:54.121 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:33:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:33:55.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:54 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:33:55.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:54 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-51", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:33:55.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:54 smithi097 ceph-mon[39416]: osdmap e358: 8 total, 8 up, 8 in 2024-03-21T13:33:55.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:54 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-51"}]: dispatch 2024-03-21T13:33:55.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:54 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-51"}]: dispatch 2024-03-21T13:33:55.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:54 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:33:55.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:54 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-51", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:33:55.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:54 smithi097 ceph-mon[30040]: osdmap e358: 8 total, 8 up, 8 in 2024-03-21T13:33:55.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:54 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-51"}]: dispatch 2024-03-21T13:33:55.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:54 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-51"}]: dispatch 2024-03-21T13:33:55.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:54 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:33:55.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:54 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-51", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:33:55.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:54 smithi144 ceph-mon[33633]: osdmap e358: 8 total, 8 up, 8 in 2024-03-21T13:33:55.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:54 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-51"}]: dispatch 2024-03-21T13:33:55.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:54 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-51"}]: dispatch 2024-03-21T13:33:56.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:55 smithi097 ceph-mon[39416]: pgmap v502: 332 pgs: 8 creating+peering, 324 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-21T13:33:56.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:55 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-51"}]': finished 2024-03-21T13:33:56.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:55 smithi097 ceph-mon[39416]: osdmap e359: 8 total, 8 up, 8 in 2024-03-21T13:33:56.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:55 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-51", "mode": "writeback"}]: dispatch 2024-03-21T13:33:56.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:55 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3715338659' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi097-79636-66"}]: dispatch 2024-03-21T13:33:56.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:55 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-51", "mode": "writeback"}]: dispatch 2024-03-21T13:33:56.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:55 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:33:56.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:55 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:33:56.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:55 smithi097 ceph-mon[30040]: pgmap v502: 332 pgs: 8 creating+peering, 324 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-21T13:33:56.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:55 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-51"}]': finished 2024-03-21T13:33:56.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:55 smithi097 ceph-mon[30040]: osdmap e359: 8 total, 8 up, 8 in 2024-03-21T13:33:56.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:55 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-51", "mode": "writeback"}]: dispatch 2024-03-21T13:33:56.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:55 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3715338659' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi097-79636-66"}]: dispatch 2024-03-21T13:33:56.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:55 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-51", "mode": "writeback"}]: dispatch 2024-03-21T13:33:56.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:55 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:33:56.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:55 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:33:56.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:55 smithi144 ceph-mon[33633]: pgmap v502: 332 pgs: 8 creating+peering, 324 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-21T13:33:56.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:55 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-51"}]': finished 2024-03-21T13:33:56.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:55 smithi144 ceph-mon[33633]: osdmap e359: 8 total, 8 up, 8 in 2024-03-21T13:33:56.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:55 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-51", "mode": "writeback"}]: dispatch 2024-03-21T13:33:56.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:55 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3715338659' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi097-79636-66"}]: dispatch 2024-03-21T13:33:56.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:55 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-51", "mode": "writeback"}]: dispatch 2024-03-21T13:33:56.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:55 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:33:56.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:55 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:33:56.779 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ 2024-03-21T13:33:56.781 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP2 (3037 ms) 2024-03-21T13:33:56.781 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPP 2024-03-21T13:33:56.781 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPP (7038 ms) 2024-03-21T13:33:56.781 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPPNS 2024-03-21T13:33:56.781 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPPNS (7019 ms) 2024-03-21T13:33:56.781 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.StatRemovePP 2024-03-21T13:33:56.781 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAioEC.StatRemovePP (7055 ms) 2024-03-21T13:33:56.781 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ExecuteClassPP 2024-03-21T13:33:56.781 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ExecuteClassPP (7018 ms) 2024-03-21T13:33:56.781 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.OmapPP 2024-03-21T13:33:56.781 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAioEC.OmapPP (7063 ms) 2024-03-21T13:33:56.781 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.MultiWritePP 2024-03-21T13:33:56.782 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ OK ] LibRadosAioEC.MultiWritePP (7038 ms) 2024-03-21T13:33:56.782 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC (139241 ms total) 2024-03-21T13:33:56.782 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: 2024-03-21T13:33:56.782 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [----------] Global test environment tear-down 2024-03-21T13:33:56.782 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [==========] 57 tests from 4 test suites ran. (297343 ms total) 2024-03-21T13:33:56.782 INFO:tasks.workunit.client.0.smithi097.stdout: api_aio_pp: [ PASSED ] 57 tests. 2024-03-21T13:33:57.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:56 smithi097 ceph-mon[39416]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:33:57.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:56 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3715338659' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi097-79636-66"}]': finished 2024-03-21T13:33:57.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:56 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-51", "mode": "writeback"}]': finished 2024-03-21T13:33:57.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:56 smithi097 ceph-mon[39416]: osdmap e360: 8 total, 8 up, 8 in 2024-03-21T13:33:57.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:56 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3715338659' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi097-79636-66"}]: dispatch 2024-03-21T13:33:57.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:56 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-51","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:33:57.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:56 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-51","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:33:57.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:56 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3715338659' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi097-79636-66"}]': finished 2024-03-21T13:33:57.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:56 smithi097 ceph-mon[30040]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:33:57.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:56 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3715338659' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi097-79636-66"}]': finished 2024-03-21T13:33:57.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:56 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-51", "mode": "writeback"}]': finished 2024-03-21T13:33:57.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:56 smithi097 ceph-mon[30040]: osdmap e360: 8 total, 8 up, 8 in 2024-03-21T13:33:57.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:56 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3715338659' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi097-79636-66"}]: dispatch 2024-03-21T13:33:57.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:56 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-51","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:33:57.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:56 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-51","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:33:57.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:56 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3715338659' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi097-79636-66"}]': finished 2024-03-21T13:33:57.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:56 smithi144 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:33:57.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:56 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3715338659' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi097-79636-66"}]': finished 2024-03-21T13:33:57.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:56 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-51", "mode": "writeback"}]': finished 2024-03-21T13:33:57.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:56 smithi144 ceph-mon[33633]: osdmap e360: 8 total, 8 up, 8 in 2024-03-21T13:33:57.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:56 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3715338659' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi097-79636-66"}]: dispatch 2024-03-21T13:33:57.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:56 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-51","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:33:57.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:56 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-51","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:33:57.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:56 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3715338659' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi097-79636-66"}]': finished 2024-03-21T13:33:58.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:57 smithi097 ceph-mon[30040]: pgmap v505: 324 pgs: 324 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-21T13:33:58.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:57 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-51","var": "hit_set_count","val": "2"}]': finished 2024-03-21T13:33:58.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:57 smithi097 ceph-mon[30040]: osdmap e361: 8 total, 8 up, 8 in 2024-03-21T13:33:58.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:57 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-51","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:33:58.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:57 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-51","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:33:58.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:57 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-51","var": "hit_set_period","val": "600"}]': finished 2024-03-21T13:33:58.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:57 smithi097 ceph-mon[30040]: osdmap e362: 8 total, 8 up, 8 in 2024-03-21T13:33:58.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:57 smithi097 ceph-mon[39416]: pgmap v505: 324 pgs: 324 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-21T13:33:58.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:57 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-51","var": "hit_set_count","val": "2"}]': finished 2024-03-21T13:33:58.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:57 smithi097 ceph-mon[39416]: osdmap e361: 8 total, 8 up, 8 in 2024-03-21T13:33:58.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:57 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-51","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:33:58.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:57 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-51","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:33:58.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:57 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-51","var": "hit_set_period","val": "600"}]': finished 2024-03-21T13:33:58.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:57 smithi097 ceph-mon[39416]: osdmap e362: 8 total, 8 up, 8 in 2024-03-21T13:33:58.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:57 smithi144 ceph-mon[33633]: pgmap v505: 324 pgs: 324 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-21T13:33:58.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:57 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-51","var": "hit_set_count","val": "2"}]': finished 2024-03-21T13:33:58.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:57 smithi144 ceph-mon[33633]: osdmap e361: 8 total, 8 up, 8 in 2024-03-21T13:33:58.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:57 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-51","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:33:58.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:57 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-51","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:33:58.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:57 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-51","var": "hit_set_period","val": "600"}]': finished 2024-03-21T13:33:58.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:57 smithi144 ceph-mon[33633]: osdmap e362: 8 total, 8 up, 8 in 2024-03-21T13:33:59.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:58 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:33:59.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:58 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:33:59.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:58 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:33:59.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:58 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:33:59.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:58 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:33:59.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:58 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:34:00.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:59 smithi097 ceph-mon[39416]: pgmap v508: 324 pgs: 324 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-21T13:34:00.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:59 smithi097 ceph-mon[39416]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:34:00.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:59 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-51","var": "hit_set_type","val": "bloom"}]': finished 2024-03-21T13:34:00.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:59 smithi097 ceph-mon[39416]: osdmap e363: 8 total, 8 up, 8 in 2024-03-21T13:34:00.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:59 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-21T13:34:00.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:33:59 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-21T13:34:00.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:59 smithi097 ceph-mon[30040]: pgmap v508: 324 pgs: 324 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-21T13:34:00.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:59 smithi097 ceph-mon[30040]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:34:00.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:59 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-51","var": "hit_set_type","val": "bloom"}]': finished 2024-03-21T13:34:00.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:59 smithi097 ceph-mon[30040]: osdmap e363: 8 total, 8 up, 8 in 2024-03-21T13:34:00.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:59 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-21T13:34:00.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:33:59 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-21T13:34:00.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:59 smithi144 ceph-mon[33633]: pgmap v508: 324 pgs: 324 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-21T13:34:00.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:59 smithi144 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:34:00.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:59 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-51","var": "hit_set_type","val": "bloom"}]': finished 2024-03-21T13:34:00.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:59 smithi144 ceph-mon[33633]: osdmap e363: 8 total, 8 up, 8 in 2024-03-21T13:34:00.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:59 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-21T13:34:00.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:33:59 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-21T13:34:01.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:00 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-21T13:34:01.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:00 smithi097 ceph-mon[39416]: osdmap e364: 8 total, 8 up, 8 in 2024-03-21T13:34:01.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:00 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-51","var": "target_max_objects","val": "1"}]: dispatch 2024-03-21T13:34:01.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:00 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-51","var": "target_max_objects","val": "1"}]: dispatch 2024-03-21T13:34:01.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:00 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-21T13:34:01.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:00 smithi097 ceph-mon[30040]: osdmap e364: 8 total, 8 up, 8 in 2024-03-21T13:34:01.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:00 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-51","var": "target_max_objects","val": "1"}]: dispatch 2024-03-21T13:34:01.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:00 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-51","var": "target_max_objects","val": "1"}]: dispatch 2024-03-21T13:34:01.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:00 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-21T13:34:01.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:00 smithi144 ceph-mon[33633]: osdmap e364: 8 total, 8 up, 8 in 2024-03-21T13:34:01.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:00 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-51","var": "target_max_objects","val": "1"}]: dispatch 2024-03-21T13:34:01.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:00 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-51","var": "target_max_objects","val": "1"}]: dispatch 2024-03-21T13:34:02.053 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:34:01 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:34:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:34:02.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:01 smithi097 ceph-mon[30040]: pgmap v511: 324 pgs: 324 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-21T13:34:02.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:01 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-51","var": "target_max_objects","val": "1"}]': finished 2024-03-21T13:34:02.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:01 smithi097 ceph-mon[30040]: osdmap e365: 8 total, 8 up, 8 in 2024-03-21T13:34:02.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:01 smithi097 ceph-mon[39416]: pgmap v511: 324 pgs: 324 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-21T13:34:02.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:01 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-51","var": "target_max_objects","val": "1"}]': finished 2024-03-21T13:34:02.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:01 smithi097 ceph-mon[39416]: osdmap e365: 8 total, 8 up, 8 in 2024-03-21T13:34:02.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:01 smithi144 ceph-mon[33633]: pgmap v511: 324 pgs: 324 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-21T13:34:02.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:01 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-51","var": "target_max_objects","val": "1"}]': finished 2024-03-21T13:34:02.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:01 smithi144 ceph-mon[33633]: osdmap e365: 8 total, 8 up, 8 in 2024-03-21T13:34:03.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:02 smithi144 ceph-mon[33633]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-03-21T13:34:03.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:02 smithi097 ceph-mon[30040]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-03-21T13:34:03.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:02 smithi097 ceph-mon[39416]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-03-21T13:34:04.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:03 smithi144 ceph-mon[33633]: pgmap v513: 324 pgs: 324 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-21T13:34:04.121 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:34:03 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:34:04.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:03 smithi097 ceph-mon[30040]: pgmap v513: 324 pgs: 324 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-21T13:34:04.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:03 smithi097 ceph-mon[39416]: pgmap v513: 324 pgs: 324 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-21T13:34:05.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:04 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:34:05.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:04 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:34:05.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:04 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:34:06.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:05 smithi144 ceph-mon[33633]: pgmap v514: 324 pgs: 324 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:34:06.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:05 smithi097 ceph-mon[39416]: pgmap v514: 324 pgs: 324 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:34:06.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:05 smithi097 ceph-mon[30040]: pgmap v514: 324 pgs: 324 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:34:07.904 INFO:tasks.workunit.client.0.smithi097.stdout: started 0x55ca2a5c1a10 2024-03-21T13:34:07.904 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c1b20 2024-03-21T13:34:07.904 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c1c70 2024-03-21T13:34:07.905 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c2390 2024-03-21T13:34:07.905 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c1ee0 2024-03-21T13:34:07.905 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c2030 2024-03-21T13:34:07.905 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c2180 2024-03-21T13:34:07.905 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c24e0 2024-03-21T13:34:07.905 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c29b0 2024-03-21T13:34:07.905 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c2ae0 2024-03-21T13:34:07.905 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c25f0 2024-03-21T13:34:07.905 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c2740 2024-03-21T13:34:07.905 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c2870 2024-03-21T13:34:07.905 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c2c10 2024-03-21T13:34:07.905 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c30e0 2024-03-21T13:34:07.905 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c3210 2024-03-21T13:34:07.905 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c2d20 2024-03-21T13:34:07.905 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c2e70 2024-03-21T13:34:07.905 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c2fc0 2024-03-21T13:34:07.906 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c3320 2024-03-21T13:34:07.906 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c3810 2024-03-21T13:34:07.906 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c3940 2024-03-21T13:34:07.906 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c3f00 2024-03-21T13:34:07.906 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c3470 2024-03-21T13:34:07.906 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c35c0 2024-03-21T13:34:07.906 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c3a50 2024-03-21T13:34:07.906 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c3b80 2024-03-21T13:34:07.906 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c3cb0 2024-03-21T13:34:07.906 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c3de0 2024-03-21T13:34:07.906 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c4500 2024-03-21T13:34:07.906 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c4050 2024-03-21T13:34:07.906 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c4180 2024-03-21T13:34:07.906 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c4290 2024-03-21T13:34:07.906 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c43e0 2024-03-21T13:34:07.907 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c4ae0 2024-03-21T13:34:07.907 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c4630 2024-03-21T13:34:07.907 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bf280 2024-03-21T13:34:07.907 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bf390 2024-03-21T13:34:07.907 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5bf4e0 2024-03-21T13:34:07.907 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c50c0 2024-03-21T13:34:07.907 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c4c10 2024-03-21T13:34:07.907 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c4740 2024-03-21T13:34:07.907 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c4850 2024-03-21T13:34:07.907 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c49a0 2024-03-21T13:34:07.907 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c56c0 2024-03-21T13:34:07.907 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c5210 2024-03-21T13:34:07.907 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c5340 2024-03-21T13:34:07.907 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c5450 2024-03-21T13:34:07.907 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c5580 2024-03-21T13:34:07.907 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c5cc0 2024-03-21T13:34:07.907 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c5810 2024-03-21T13:34:07.908 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c5960 2024-03-21T13:34:07.908 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c5a90 2024-03-21T13:34:07.908 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c5e10 2024-03-21T13:34:07.908 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c6300 2024-03-21T13:34:07.908 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c6450 2024-03-21T13:34:07.908 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c5f40 2024-03-21T13:34:07.908 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c6090 2024-03-21T13:34:07.908 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c61e0 2024-03-21T13:34:07.908 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c6580 2024-03-21T13:34:07.908 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c6a50 2024-03-21T13:34:07.908 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c6b80 2024-03-21T13:34:07.908 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c6690 2024-03-21T13:34:07.908 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c67e0 2024-03-21T13:34:07.908 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c6910 2024-03-21T13:34:07.908 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c6c90 2024-03-21T13:34:07.908 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c7160 2024-03-21T13:34:07.909 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c72d0 2024-03-21T13:34:07.909 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c78d0 2024-03-21T13:34:07.909 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c6dc0 2024-03-21T13:34:07.909 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c6f10 2024-03-21T13:34:07.909 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c7040 2024-03-21T13:34:07.909 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c7a20 2024-03-21T13:34:07.909 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c7420 2024-03-21T13:34:07.909 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c7570 2024-03-21T13:34:07.909 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c76a0 2024-03-21T13:34:07.909 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c77b0 2024-03-21T13:34:07.909 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c7fe0 2024-03-21T13:34:07.909 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c7b30 2024-03-21T13:34:07.909 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c7ca0 2024-03-21T13:34:07.909 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c7df0 2024-03-21T13:34:07.909 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c8130 2024-03-21T13:34:08.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:07 smithi144 ceph-mon[33633]: pgmap v515: 324 pgs: 324 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 664 B/s rd, 0 B/s wr, 0 op/s 2024-03-21T13:34:08.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:07 smithi097 ceph-mon[39416]: pgmap v515: 324 pgs: 324 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 664 B/s rd, 0 B/s wr, 0 op/s 2024-03-21T13:34:08.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:07 smithi097 ceph-mon[30040]: pgmap v515: 324 pgs: 324 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 664 B/s rd, 0 B/s wr, 0 op/s 2024-03-21T13:34:10.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:09 smithi144 ceph-mon[33633]: pgmap v516: 324 pgs: 324 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:34:10.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:09 smithi097 ceph-mon[39416]: pgmap v516: 324 pgs: 324 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:34:10.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:09 smithi097 ceph-mon[30040]: pgmap v516: 324 pgs: 324 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:34:11.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:11 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:34:11.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:11 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:34:11.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:11 smithi097 ceph-mon[39416]: pgmap v517: 324 pgs: 324 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:34:11.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:11 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:34:11.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:11 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:34:11.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:11 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:34:11.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:11 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:34:11.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:11 smithi097 ceph-mon[30040]: pgmap v517: 324 pgs: 324 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:34:11.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:11 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:34:11.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:11 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:34:11.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:11 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:34:11.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:11 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:34:11.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:11 smithi144 ceph-mon[33633]: pgmap v517: 324 pgs: 324 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:34:11.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:11 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:34:11.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:11 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:34:12.286 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:34:11 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:34:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:34:12.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:12 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:34:12.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:12 smithi097 ceph-mon[30040]: osdmap e366: 8 total, 8 up, 8 in 2024-03-21T13:34:12.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:12 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-51"}]: dispatch 2024-03-21T13:34:12.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:12 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-51"}]: dispatch 2024-03-21T13:34:12.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:12 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:34:12.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:12 smithi097 ceph-mon[39416]: osdmap e366: 8 total, 8 up, 8 in 2024-03-21T13:34:12.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:12 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-51"}]: dispatch 2024-03-21T13:34:12.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:12 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-51"}]: dispatch 2024-03-21T13:34:12.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:12 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:34:12.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:12 smithi144 ceph-mon[33633]: osdmap e366: 8 total, 8 up, 8 in 2024-03-21T13:34:12.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:12 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-51"}]: dispatch 2024-03-21T13:34:12.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:12 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-51"}]: dispatch 2024-03-21T13:34:13.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:13 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-51"}]': finished 2024-03-21T13:34:13.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:13 smithi097 ceph-mon[30040]: osdmap e367: 8 total, 8 up, 8 in 2024-03-21T13:34:13.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:13 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:34:13.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:13 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:34:13.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:13 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-51"}]: dispatch 2024-03-21T13:34:13.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:13 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-51"}]: dispatch 2024-03-21T13:34:13.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:13 smithi097 ceph-mon[30040]: pgmap v520: 324 pgs: 324 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-21T13:34:13.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:13 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-51"}]': finished 2024-03-21T13:34:13.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:13 smithi097 ceph-mon[39416]: osdmap e367: 8 total, 8 up, 8 in 2024-03-21T13:34:13.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:13 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:34:13.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:13 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:34:13.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:13 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-51"}]: dispatch 2024-03-21T13:34:13.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:13 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-51"}]: dispatch 2024-03-21T13:34:13.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:13 smithi097 ceph-mon[39416]: pgmap v520: 324 pgs: 324 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-21T13:34:13.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:13 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-51"}]': finished 2024-03-21T13:34:13.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:13 smithi144 ceph-mon[33633]: osdmap e367: 8 total, 8 up, 8 in 2024-03-21T13:34:13.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:13 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:34:13.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:13 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:34:13.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:13 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-51"}]: dispatch 2024-03-21T13:34:13.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:13 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-51"}]: dispatch 2024-03-21T13:34:13.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:13 smithi144 ceph-mon[33633]: pgmap v520: 324 pgs: 324 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-21T13:34:14.121 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:34:13 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:34:14.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:14 smithi097 ceph-mon[39416]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-03-21T13:34:14.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:14 smithi097 ceph-mon[39416]: osdmap e368: 8 total, 8 up, 8 in 2024-03-21T13:34:14.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:14 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:34:14.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:14 smithi097 ceph-mon[30040]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-03-21T13:34:14.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:14 smithi097 ceph-mon[30040]: osdmap e368: 8 total, 8 up, 8 in 2024-03-21T13:34:14.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:14 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:34:14.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:14 smithi144 ceph-mon[33633]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-03-21T13:34:14.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:14 smithi144 ceph-mon[33633]: osdmap e368: 8 total, 8 up, 8 in 2024-03-21T13:34:14.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:14 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:34:15.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:15 smithi144 ceph-mon[33633]: osdmap e369: 8 total, 8 up, 8 in 2024-03-21T13:34:15.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:15 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:34:15.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:15 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:34:15.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:15 smithi144 ceph-mon[33633]: pgmap v523: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-03-21T13:34:15.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:15 smithi097 ceph-mon[39416]: osdmap e369: 8 total, 8 up, 8 in 2024-03-21T13:34:15.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:15 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:34:15.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:15 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:34:15.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:15 smithi097 ceph-mon[39416]: pgmap v523: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-03-21T13:34:15.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:15 smithi097 ceph-mon[30040]: osdmap e369: 8 total, 8 up, 8 in 2024-03-21T13:34:15.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:15 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:34:15.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:15 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:34:15.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:15 smithi097 ceph-mon[30040]: pgmap v523: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-03-21T13:34:16.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:16 smithi144 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:34:16.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:16 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:34:16.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:16 smithi144 ceph-mon[33633]: osdmap e370: 8 total, 8 up, 8 in 2024-03-21T13:34:16.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:16 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:34:16.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:16 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:34:16.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:16 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-53"}]: dispatch 2024-03-21T13:34:16.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:16 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-53"}]: dispatch 2024-03-21T13:34:16.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:16 smithi097 ceph-mon[39416]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:34:16.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:16 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:34:16.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:16 smithi097 ceph-mon[39416]: osdmap e370: 8 total, 8 up, 8 in 2024-03-21T13:34:16.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:16 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:34:16.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:16 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:34:16.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:16 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-53"}]: dispatch 2024-03-21T13:34:16.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:16 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-53"}]: dispatch 2024-03-21T13:34:16.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:16 smithi097 ceph-mon[30040]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:34:16.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:16 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:34:16.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:16 smithi097 ceph-mon[30040]: osdmap e370: 8 total, 8 up, 8 in 2024-03-21T13:34:16.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:16 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:34:16.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:16 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:34:16.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:16 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-53"}]: dispatch 2024-03-21T13:34:16.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:16 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-53"}]: dispatch 2024-03-21T13:34:17.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:17 smithi144 ceph-mon[33633]: osdmap e371: 8 total, 8 up, 8 in 2024-03-21T13:34:17.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:17 smithi144 ceph-mon[33633]: pgmap v526: 292 pgs: 292 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-03-21T13:34:17.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:17 smithi097 ceph-mon[30040]: osdmap e371: 8 total, 8 up, 8 in 2024-03-21T13:34:17.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:17 smithi097 ceph-mon[30040]: pgmap v526: 292 pgs: 292 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-03-21T13:34:17.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:17 smithi097 ceph-mon[39416]: osdmap e371: 8 total, 8 up, 8 in 2024-03-21T13:34:17.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:17 smithi097 ceph-mon[39416]: pgmap v526: 292 pgs: 292 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-03-21T13:34:18.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:18 smithi144 ceph-mon[33633]: osdmap e372: 8 total, 8 up, 8 in 2024-03-21T13:34:18.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:18 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:34:18.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:18 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:34:18.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:18 smithi097 ceph-mon[30040]: osdmap e372: 8 total, 8 up, 8 in 2024-03-21T13:34:18.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:18 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:34:18.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:18 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:34:18.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:18 smithi097 ceph-mon[39416]: osdmap e372: 8 total, 8 up, 8 in 2024-03-21T13:34:18.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:18 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:34:18.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:18 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:34:19.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:19 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:34:19.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:19 smithi144 ceph-mon[33633]: osdmap e373: 8 total, 8 up, 8 in 2024-03-21T13:34:19.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:19 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:34:19.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:19 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:34:19.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:19 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:34:19.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:19 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-55"}]: dispatch 2024-03-21T13:34:19.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:19 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-55"}]: dispatch 2024-03-21T13:34:19.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:19 smithi144 ceph-mon[33633]: pgmap v529: 324 pgs: 1 creating+activating, 323 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-03-21T13:34:19.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:19 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:34:19.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:19 smithi097 ceph-mon[30040]: osdmap e373: 8 total, 8 up, 8 in 2024-03-21T13:34:19.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:19 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:34:19.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:19 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:34:19.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:19 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:34:19.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:19 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-55"}]: dispatch 2024-03-21T13:34:19.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:19 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-55"}]: dispatch 2024-03-21T13:34:19.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:19 smithi097 ceph-mon[30040]: pgmap v529: 324 pgs: 1 creating+activating, 323 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-03-21T13:34:19.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:19 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:34:19.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:19 smithi097 ceph-mon[39416]: osdmap e373: 8 total, 8 up, 8 in 2024-03-21T13:34:19.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:19 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:34:19.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:19 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:34:19.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:19 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:34:19.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:19 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-55"}]: dispatch 2024-03-21T13:34:19.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:19 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-55"}]: dispatch 2024-03-21T13:34:19.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:19 smithi097 ceph-mon[39416]: pgmap v529: 324 pgs: 1 creating+activating, 323 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-03-21T13:34:20.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:20 smithi144 ceph-mon[33633]: osdmap e374: 8 total, 8 up, 8 in 2024-03-21T13:34:20.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:20 smithi097 ceph-mon[30040]: osdmap e374: 8 total, 8 up, 8 in 2024-03-21T13:34:20.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:20 smithi097 ceph-mon[39416]: osdmap e374: 8 total, 8 up, 8 in 2024-03-21T13:34:21.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:21 smithi144 ceph-mon[33633]: osdmap e375: 8 total, 8 up, 8 in 2024-03-21T13:34:21.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:21 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:34:21.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:21 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:34:21.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:21 smithi144 ceph-mon[33633]: pgmap v532: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-21T13:34:21.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:21 smithi097 ceph-mon[39416]: osdmap e375: 8 total, 8 up, 8 in 2024-03-21T13:34:21.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:21 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:34:21.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:21 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:34:21.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:21 smithi097 ceph-mon[39416]: pgmap v532: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-21T13:34:21.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:21 smithi097 ceph-mon[30040]: osdmap e375: 8 total, 8 up, 8 in 2024-03-21T13:34:21.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:21 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:34:21.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:21 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:34:21.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:21 smithi097 ceph-mon[30040]: pgmap v532: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-21T13:34:22.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:34:21 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:34:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:34:22.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:22 smithi144 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:34:22.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:22 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:34:22.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:22 smithi144 ceph-mon[33633]: osdmap e376: 8 total, 8 up, 8 in 2024-03-21T13:34:22.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:22 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:34:22.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:22 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:34:22.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:22 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-57"}]: dispatch 2024-03-21T13:34:22.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:22 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-57"}]: dispatch 2024-03-21T13:34:22.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:22 smithi097 ceph-mon[39416]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:34:22.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:22 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:34:22.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:22 smithi097 ceph-mon[39416]: osdmap e376: 8 total, 8 up, 8 in 2024-03-21T13:34:22.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:22 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:34:22.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:22 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:34:22.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:22 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-57"}]: dispatch 2024-03-21T13:34:22.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:22 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-57"}]: dispatch 2024-03-21T13:34:22.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:22 smithi097 ceph-mon[30040]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:34:22.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:22 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:34:22.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:22 smithi097 ceph-mon[30040]: osdmap e376: 8 total, 8 up, 8 in 2024-03-21T13:34:22.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:22 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:34:22.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:22 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:34:22.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:22 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-57"}]: dispatch 2024-03-21T13:34:22.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:22 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-57"}]: dispatch 2024-03-21T13:34:23.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:23 smithi144 ceph-mon[33633]: osdmap e377: 8 total, 8 up, 8 in 2024-03-21T13:34:23.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:23 smithi144 ceph-mon[33633]: pgmap v535: 292 pgs: 292 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:34:23.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:23 smithi097 ceph-mon[39416]: osdmap e377: 8 total, 8 up, 8 in 2024-03-21T13:34:23.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:23 smithi097 ceph-mon[39416]: pgmap v535: 292 pgs: 292 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:34:23.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:23 smithi097 ceph-mon[30040]: osdmap e377: 8 total, 8 up, 8 in 2024-03-21T13:34:23.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:23 smithi097 ceph-mon[30040]: pgmap v535: 292 pgs: 292 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:34:24.121 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:34:23 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:34:24.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:24 smithi097 ceph-mon[39416]: osdmap e378: 8 total, 8 up, 8 in 2024-03-21T13:34:24.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:24 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:34:24.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:24 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:34:24.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:24 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:34:24.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:24 smithi097 ceph-mon[30040]: osdmap e378: 8 total, 8 up, 8 in 2024-03-21T13:34:24.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:24 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:34:24.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:24 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:34:24.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:24 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:34:24.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:24 smithi144 ceph-mon[33633]: osdmap e378: 8 total, 8 up, 8 in 2024-03-21T13:34:24.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:24 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:34:24.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:24 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:34:24.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:24 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:34:25.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:25 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:34:25.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:25 smithi097 ceph-mon[30040]: osdmap e379: 8 total, 8 up, 8 in 2024-03-21T13:34:25.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:25 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:34:25.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:25 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:34:25.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:25 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:34:25.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:25 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-59"}]: dispatch 2024-03-21T13:34:25.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:25 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-59"}]: dispatch 2024-03-21T13:34:25.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:25 smithi097 ceph-mon[30040]: pgmap v538: 324 pgs: 7 creating+peering, 25 unknown, 292 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:34:25.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:25 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:34:25.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:25 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:34:25.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:25 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:34:25.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:25 smithi097 ceph-mon[39416]: osdmap e379: 8 total, 8 up, 8 in 2024-03-21T13:34:25.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:25 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:34:25.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:25 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:34:25.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:25 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:34:25.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:25 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-59"}]: dispatch 2024-03-21T13:34:25.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:25 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-59"}]: dispatch 2024-03-21T13:34:25.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:25 smithi097 ceph-mon[39416]: pgmap v538: 324 pgs: 7 creating+peering, 25 unknown, 292 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:34:25.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:25 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:34:25.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:25 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:34:25.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:25 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:34:25.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:25 smithi144 ceph-mon[33633]: osdmap e379: 8 total, 8 up, 8 in 2024-03-21T13:34:25.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:25 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:34:25.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:25 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:34:25.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:25 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:34:25.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:25 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-59"}]: dispatch 2024-03-21T13:34:25.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:25 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-59"}]: dispatch 2024-03-21T13:34:25.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:25 smithi144 ceph-mon[33633]: pgmap v538: 324 pgs: 7 creating+peering, 25 unknown, 292 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:34:25.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:25 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:34:25.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:25 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:34:26.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:26 smithi097 ceph-mon[39416]: osdmap e380: 8 total, 8 up, 8 in 2024-03-21T13:34:26.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:26 smithi097 ceph-mon[30040]: osdmap e380: 8 total, 8 up, 8 in 2024-03-21T13:34:26.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:26 smithi144 ceph-mon[33633]: osdmap e380: 8 total, 8 up, 8 in 2024-03-21T13:34:27.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:27 smithi097 ceph-mon[39416]: osdmap e381: 8 total, 8 up, 8 in 2024-03-21T13:34:27.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:27 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:34:27.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:27 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:34:27.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:27 smithi097 ceph-mon[39416]: pgmap v541: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:34:27.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:27 smithi097 ceph-mon[39416]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:34:27.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:27 smithi097 ceph-mon[30040]: osdmap e381: 8 total, 8 up, 8 in 2024-03-21T13:34:27.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:27 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:34:27.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:27 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:34:27.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:27 smithi097 ceph-mon[30040]: pgmap v541: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:34:27.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:27 smithi097 ceph-mon[30040]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:34:27.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:27 smithi144 ceph-mon[33633]: osdmap e381: 8 total, 8 up, 8 in 2024-03-21T13:34:27.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:27 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:34:27.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:27 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:34:27.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:27 smithi144 ceph-mon[33633]: pgmap v541: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:34:27.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:27 smithi144 ceph-mon[33633]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:34:28.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:28 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:34:28.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:28 smithi097 ceph-mon[39416]: osdmap e382: 8 total, 8 up, 8 in 2024-03-21T13:34:28.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:28 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:34:28.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:28 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:34:28.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:28 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:34:28.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:28 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:34:28.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:28 smithi097 ceph-mon[30040]: osdmap e382: 8 total, 8 up, 8 in 2024-03-21T13:34:28.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:28 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:34:28.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:28 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:34:28.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:28 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:34:28.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:28 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:34:28.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:28 smithi144 ceph-mon[33633]: osdmap e382: 8 total, 8 up, 8 in 2024-03-21T13:34:28.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:28 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:34:28.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:28 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:34:28.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:28 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:34:29.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:29 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:34:29.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:29 smithi097 ceph-mon[39416]: osdmap e383: 8 total, 8 up, 8 in 2024-03-21T13:34:29.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:29 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:34:29.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:29 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:34:29.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:29 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-61"}]: dispatch 2024-03-21T13:34:29.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:29 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-61"}]: dispatch 2024-03-21T13:34:29.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:29 smithi097 ceph-mon[39416]: pgmap v544: 324 pgs: 324 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-21T13:34:29.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:29 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:34:29.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:29 smithi097 ceph-mon[30040]: osdmap e383: 8 total, 8 up, 8 in 2024-03-21T13:34:29.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:29 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:34:29.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:29 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:34:29.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:29 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-61"}]: dispatch 2024-03-21T13:34:29.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:29 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-61"}]: dispatch 2024-03-21T13:34:29.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:29 smithi097 ceph-mon[30040]: pgmap v544: 324 pgs: 324 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-21T13:34:29.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:29 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:34:29.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:29 smithi144 ceph-mon[33633]: osdmap e383: 8 total, 8 up, 8 in 2024-03-21T13:34:29.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:29 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:34:29.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:29 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:34:29.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:29 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-61"}]: dispatch 2024-03-21T13:34:29.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:29 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-61"}]: dispatch 2024-03-21T13:34:29.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:29 smithi144 ceph-mon[33633]: pgmap v544: 324 pgs: 324 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-21T13:34:30.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:30 smithi097 ceph-mon[39416]: osdmap e384: 8 total, 8 up, 8 in 2024-03-21T13:34:30.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:30 smithi097 ceph-mon[30040]: osdmap e384: 8 total, 8 up, 8 in 2024-03-21T13:34:30.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:30 smithi144 ceph-mon[33633]: osdmap e384: 8 total, 8 up, 8 in 2024-03-21T13:34:31.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:31 smithi097 ceph-mon[39416]: osdmap e385: 8 total, 8 up, 8 in 2024-03-21T13:34:31.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:31 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:34:31.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:31 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:34:31.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:31 smithi097 ceph-mon[39416]: pgmap v547: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-21T13:34:31.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:31 smithi097 ceph-mon[30040]: osdmap e385: 8 total, 8 up, 8 in 2024-03-21T13:34:31.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:31 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:34:31.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:31 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:34:31.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:31 smithi097 ceph-mon[30040]: pgmap v547: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-21T13:34:31.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:31 smithi144 ceph-mon[33633]: osdmap e385: 8 total, 8 up, 8 in 2024-03-21T13:34:31.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:31 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:34:31.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:31 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:34:31.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:31 smithi144 ceph-mon[33633]: pgmap v547: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-21T13:34:32.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:34:31 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:34:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:34:32.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:32 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:34:32.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:32 smithi097 ceph-mon[39416]: osdmap e386: 8 total, 8 up, 8 in 2024-03-21T13:34:32.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:32 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:34:32.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:32 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:34:32.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:32 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:34:32.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:32 smithi097 ceph-mon[39416]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:34:32.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:32 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:34:32.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:32 smithi097 ceph-mon[30040]: osdmap e386: 8 total, 8 up, 8 in 2024-03-21T13:34:32.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:32 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:34:32.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:32 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:34:32.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:32 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:34:32.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:32 smithi097 ceph-mon[30040]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:34:32.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:32 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:34:32.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:32 smithi144 ceph-mon[33633]: osdmap e386: 8 total, 8 up, 8 in 2024-03-21T13:34:32.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:32 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:34:32.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:32 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:34:32.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:32 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:34:32.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:32 smithi144 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:34:33.707 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:33 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:34:33.707 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:33 smithi144 ceph-mon[33633]: osdmap e387: 8 total, 8 up, 8 in 2024-03-21T13:34:33.707 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:33 smithi144 ceph-mon[33633]: pgmap v550: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:34:33.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:33 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:34:33.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:33 smithi097 ceph-mon[39416]: osdmap e387: 8 total, 8 up, 8 in 2024-03-21T13:34:33.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:33 smithi097 ceph-mon[39416]: pgmap v550: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:34:33.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:33 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:34:33.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:33 smithi097 ceph-mon[30040]: osdmap e387: 8 total, 8 up, 8 in 2024-03-21T13:34:33.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:33 smithi097 ceph-mon[30040]: pgmap v550: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:34:34.121 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:34:33 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:34:34.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:34 smithi097 ceph-mon[39416]: osdmap e388: 8 total, 8 up, 8 in 2024-03-21T13:34:34.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:34 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:34:34.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:34 smithi097 ceph-mon[30040]: osdmap e388: 8 total, 8 up, 8 in 2024-03-21T13:34:34.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:34 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:34:34.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:34 smithi144 ceph-mon[33633]: osdmap e388: 8 total, 8 up, 8 in 2024-03-21T13:34:34.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:34 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:34:35.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:35 smithi097 ceph-mon[39416]: osdmap e389: 8 total, 8 up, 8 in 2024-03-21T13:34:35.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:35 smithi097 ceph-mon[39416]: pgmap v553: 324 pgs: 324 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-03-21T13:34:35.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:35 smithi097 ceph-mon[30040]: osdmap e389: 8 total, 8 up, 8 in 2024-03-21T13:34:35.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:35 smithi097 ceph-mon[30040]: pgmap v553: 324 pgs: 324 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-03-21T13:34:35.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:35 smithi144 ceph-mon[33633]: osdmap e389: 8 total, 8 up, 8 in 2024-03-21T13:34:35.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:35 smithi144 ceph-mon[33633]: pgmap v553: 324 pgs: 324 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-03-21T13:34:36.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:36 smithi097 ceph-mon[39416]: osdmap e390: 8 total, 8 up, 8 in 2024-03-21T13:34:36.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:36 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:34:36.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:36 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:34:36.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:36 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:34:36.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:36 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:34:36.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:36 smithi097 ceph-mon[30040]: osdmap e390: 8 total, 8 up, 8 in 2024-03-21T13:34:36.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:36 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:34:36.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:36 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:34:36.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:36 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:34:36.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:36 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:34:36.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:36 smithi144 ceph-mon[33633]: osdmap e390: 8 total, 8 up, 8 in 2024-03-21T13:34:36.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:36 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:34:36.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:36 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:34:36.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:36 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:34:36.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:36 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:34:37.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:37 smithi097 ceph-mon[39416]: osdmap e391: 8 total, 8 up, 8 in 2024-03-21T13:34:37.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:37 smithi097 ceph-mon[39416]: pgmap v556: 324 pgs: 324 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-03-21T13:34:37.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:37 smithi097 ceph-mon[39416]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:34:37.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:37 smithi097 ceph-mon[30040]: osdmap e391: 8 total, 8 up, 8 in 2024-03-21T13:34:37.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:37 smithi097 ceph-mon[30040]: pgmap v556: 324 pgs: 324 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-03-21T13:34:37.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:37 smithi097 ceph-mon[30040]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:34:37.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:37 smithi144 ceph-mon[33633]: osdmap e391: 8 total, 8 up, 8 in 2024-03-21T13:34:37.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:37 smithi144 ceph-mon[33633]: pgmap v556: 324 pgs: 324 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-03-21T13:34:37.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:37 smithi144 ceph-mon[33633]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:34:39.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:39 smithi144 ceph-mon[33633]: pgmap v557: 324 pgs: 324 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-03-21T13:34:40.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:39 smithi097 ceph-mon[39416]: pgmap v557: 324 pgs: 324 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-03-21T13:34:40.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:39 smithi097 ceph-mon[30040]: pgmap v557: 324 pgs: 324 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-03-21T13:34:40.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:40 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:34:40.954 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:40 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:34:40.955 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:40 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:34:41.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:41 smithi144 ceph-mon[33633]: pgmap v558: 324 pgs: 324 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1013 B/s wr, 3 op/s 2024-03-21T13:34:41.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:41 smithi097 ceph-mon[39416]: pgmap v558: 324 pgs: 324 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1013 B/s wr, 3 op/s 2024-03-21T13:34:41.950 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:41 smithi097 ceph-mon[30040]: pgmap v558: 324 pgs: 324 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1013 B/s wr, 3 op/s 2024-03-21T13:34:42.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:34:41 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:34:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:34:43.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:43 smithi097 ceph-mon[39416]: osdmap e392: 8 total, 8 up, 8 in 2024-03-21T13:34:43.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:43 smithi097 ceph-mon[39416]: pgmap v560: 324 pgs: 324 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 870 B/s wr, 3 op/s 2024-03-21T13:34:43.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:43 smithi097 ceph-mon[30040]: osdmap e392: 8 total, 8 up, 8 in 2024-03-21T13:34:43.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:43 smithi097 ceph-mon[30040]: pgmap v560: 324 pgs: 324 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 870 B/s wr, 3 op/s 2024-03-21T13:34:43.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:43 smithi144 ceph-mon[33633]: osdmap e392: 8 total, 8 up, 8 in 2024-03-21T13:34:43.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:43 smithi144 ceph-mon[33633]: pgmap v560: 324 pgs: 324 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 870 B/s wr, 3 op/s 2024-03-21T13:34:44.121 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:34:43 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:34:44.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:44 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:34:44.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:44 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:34:44.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:44 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:34:45.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:45 smithi097 ceph-mon[39416]: pgmap v561: 324 pgs: 324 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 762 B/s wr, 3 op/s 2024-03-21T13:34:45.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:45 smithi097 ceph-mon[30040]: pgmap v561: 324 pgs: 324 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 762 B/s wr, 3 op/s 2024-03-21T13:34:45.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:45 smithi144 ceph-mon[33633]: pgmap v561: 324 pgs: 324 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 762 B/s wr, 3 op/s 2024-03-21T13:34:47.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:47 smithi097 ceph-mon[39416]: pgmap v562: 324 pgs: 324 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 614 B/s wr, 2 op/s 2024-03-21T13:34:47.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:47 smithi097 ceph-mon[39416]: osdmap e393: 8 total, 8 up, 8 in 2024-03-21T13:34:47.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:47 smithi097 ceph-mon[30040]: pgmap v562: 324 pgs: 324 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 614 B/s wr, 2 op/s 2024-03-21T13:34:47.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:47 smithi097 ceph-mon[30040]: osdmap e393: 8 total, 8 up, 8 in 2024-03-21T13:34:47.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:47 smithi144 ceph-mon[33633]: pgmap v562: 324 pgs: 324 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 614 B/s wr, 2 op/s 2024-03-21T13:34:47.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:47 smithi144 ceph-mon[33633]: osdmap e393: 8 total, 8 up, 8 in 2024-03-21T13:34:49.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:49 smithi097 ceph-mon[39416]: pgmap v564: 324 pgs: 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:34:49.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:49 smithi097 ceph-mon[30040]: pgmap v564: 324 pgs: 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:34:49.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:49 smithi144 ceph-mon[33633]: pgmap v564: 324 pgs: 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:34:51.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:51 smithi097 ceph-mon[39416]: pgmap v565: 324 pgs: 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:34:51.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:51 smithi097 ceph-mon[30040]: pgmap v565: 324 pgs: 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:34:51.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:51 smithi144 ceph-mon[33633]: pgmap v565: 324 pgs: 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:34:52.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:34:51 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:34:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:34:53.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:53 smithi097 ceph-mon[30040]: osdmap e394: 8 total, 8 up, 8 in 2024-03-21T13:34:53.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:53 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.4", "id": [0, 1]}]: dispatch 2024-03-21T13:34:53.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:53 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.8", "id": [3, 7]}]: dispatch 2024-03-21T13:34:53.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:53 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.4", "id": [0, 1]}]: dispatch 2024-03-21T13:34:53.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:53 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.a", "id": [3, 1]}]: dispatch 2024-03-21T13:34:53.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:53 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.e", "id": [6, 2]}]: dispatch 2024-03-21T13:34:53.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:53 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.8", "id": [3, 7]}]: dispatch 2024-03-21T13:34:53.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:53 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.18", "id": [3, 7]}]: dispatch 2024-03-21T13:34:53.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:53 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.1b", "id": [3, 1]}]: dispatch 2024-03-21T13:34:53.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:53 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.a", "id": [3, 1]}]: dispatch 2024-03-21T13:34:53.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:53 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "107.19", "id": [3, 7]}]: dispatch 2024-03-21T13:34:53.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:53 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.e", "id": [6, 2]}]: dispatch 2024-03-21T13:34:53.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:53 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.18", "id": [3, 7]}]: dispatch 2024-03-21T13:34:53.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:53 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.1b", "id": [3, 1]}]: dispatch 2024-03-21T13:34:53.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:53 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "107.19", "id": [3, 7]}]: dispatch 2024-03-21T13:34:53.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:53 smithi097 ceph-mon[30040]: pgmap v567: 324 pgs: 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:34:53.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:53 smithi097 ceph-mon[39416]: osdmap e394: 8 total, 8 up, 8 in 2024-03-21T13:34:53.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:53 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.4", "id": [0, 1]}]: dispatch 2024-03-21T13:34:53.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:53 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.8", "id": [3, 7]}]: dispatch 2024-03-21T13:34:53.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:53 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.4", "id": [0, 1]}]: dispatch 2024-03-21T13:34:53.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:53 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.a", "id": [3, 1]}]: dispatch 2024-03-21T13:34:53.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:53 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.e", "id": [6, 2]}]: dispatch 2024-03-21T13:34:53.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:53 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.8", "id": [3, 7]}]: dispatch 2024-03-21T13:34:53.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:53 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.18", "id": [3, 7]}]: dispatch 2024-03-21T13:34:53.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:53 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.1b", "id": [3, 1]}]: dispatch 2024-03-21T13:34:53.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:53 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.a", "id": [3, 1]}]: dispatch 2024-03-21T13:34:53.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:53 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "107.19", "id": [3, 7]}]: dispatch 2024-03-21T13:34:53.556 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:53 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.e", "id": [6, 2]}]: dispatch 2024-03-21T13:34:53.556 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:53 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.18", "id": [3, 7]}]: dispatch 2024-03-21T13:34:53.556 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:53 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.1b", "id": [3, 1]}]: dispatch 2024-03-21T13:34:53.556 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:53 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "107.19", "id": [3, 7]}]: dispatch 2024-03-21T13:34:53.556 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:53 smithi097 ceph-mon[39416]: pgmap v567: 324 pgs: 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:34:53.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:53 smithi144 ceph-mon[33633]: osdmap e394: 8 total, 8 up, 8 in 2024-03-21T13:34:53.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:53 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.4", "id": [0, 1]}]: dispatch 2024-03-21T13:34:53.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:53 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.8", "id": [3, 7]}]: dispatch 2024-03-21T13:34:53.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:53 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.4", "id": [0, 1]}]: dispatch 2024-03-21T13:34:53.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:53 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.a", "id": [3, 1]}]: dispatch 2024-03-21T13:34:53.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:53 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.e", "id": [6, 2]}]: dispatch 2024-03-21T13:34:53.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:53 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.8", "id": [3, 7]}]: dispatch 2024-03-21T13:34:53.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:53 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.18", "id": [3, 7]}]: dispatch 2024-03-21T13:34:53.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:53 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.1b", "id": [3, 1]}]: dispatch 2024-03-21T13:34:53.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:53 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.a", "id": [3, 1]}]: dispatch 2024-03-21T13:34:53.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:53 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "107.19", "id": [3, 7]}]: dispatch 2024-03-21T13:34:53.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:53 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.e", "id": [6, 2]}]: dispatch 2024-03-21T13:34:53.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:53 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.18", "id": [3, 7]}]: dispatch 2024-03-21T13:34:53.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:53 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.1b", "id": [3, 1]}]: dispatch 2024-03-21T13:34:53.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:53 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "107.19", "id": [3, 7]}]: dispatch 2024-03-21T13:34:53.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:53 smithi144 ceph-mon[33633]: pgmap v567: 324 pgs: 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:34:54.121 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:34:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:34:54.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:54 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.4", "id": [0, 1]}]': finished 2024-03-21T13:34:54.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:54 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.8", "id": [3, 7]}]': finished 2024-03-21T13:34:54.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:54 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.a", "id": [3, 1]}]': finished 2024-03-21T13:34:54.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:54 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.e", "id": [6, 2]}]': finished 2024-03-21T13:34:54.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:54 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.18", "id": [3, 7]}]': finished 2024-03-21T13:34:54.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:54 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.1b", "id": [3, 1]}]': finished 2024-03-21T13:34:54.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:54 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "107.19", "id": [3, 7]}]': finished 2024-03-21T13:34:54.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:54 smithi097 ceph-mon[39416]: osdmap e395: 8 total, 8 up, 8 in 2024-03-21T13:34:54.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:54 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:34:54.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:54 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.4", "id": [0, 1]}]': finished 2024-03-21T13:34:54.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:54 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.8", "id": [3, 7]}]': finished 2024-03-21T13:34:54.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:54 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.a", "id": [3, 1]}]': finished 2024-03-21T13:34:54.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:54 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.e", "id": [6, 2]}]': finished 2024-03-21T13:34:54.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:54 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.18", "id": [3, 7]}]': finished 2024-03-21T13:34:54.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:54 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.1b", "id": [3, 1]}]': finished 2024-03-21T13:34:54.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:54 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "107.19", "id": [3, 7]}]': finished 2024-03-21T13:34:54.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:54 smithi097 ceph-mon[30040]: osdmap e395: 8 total, 8 up, 8 in 2024-03-21T13:34:54.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:54 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:34:54.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:54 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.4", "id": [0, 1]}]': finished 2024-03-21T13:34:54.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:54 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.8", "id": [3, 7]}]': finished 2024-03-21T13:34:54.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:54 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.a", "id": [3, 1]}]': finished 2024-03-21T13:34:54.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:54 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.e", "id": [6, 2]}]': finished 2024-03-21T13:34:54.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:54 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.18", "id": [3, 7]}]': finished 2024-03-21T13:34:54.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:54 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.1b", "id": [3, 1]}]': finished 2024-03-21T13:34:54.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:54 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "107.19", "id": [3, 7]}]': finished 2024-03-21T13:34:54.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:54 smithi144 ceph-mon[33633]: osdmap e395: 8 total, 8 up, 8 in 2024-03-21T13:34:54.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:54 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:34:55.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:55 smithi144 ceph-mon[33633]: osdmap e396: 8 total, 8 up, 8 in 2024-03-21T13:34:55.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:55 smithi144 ceph-mon[33633]: pgmap v570: 324 pgs: 7 peering, 317 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:34:55.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:55 smithi097 ceph-mon[39416]: osdmap e396: 8 total, 8 up, 8 in 2024-03-21T13:34:55.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:55 smithi097 ceph-mon[39416]: pgmap v570: 324 pgs: 7 peering, 317 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:34:55.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:55 smithi097 ceph-mon[30040]: osdmap e396: 8 total, 8 up, 8 in 2024-03-21T13:34:55.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:55 smithi097 ceph-mon[30040]: pgmap v570: 324 pgs: 7 peering, 317 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:34:56.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:56 smithi144 ceph-mon[33633]: Health check failed: Reduced data availability: 3 pgs inactive, 4 pgs peering (PG_AVAILABILITY) 2024-03-21T13:34:56.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:56 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:34:56.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:56 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:34:56.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:56 smithi097 ceph-mon[39416]: Health check failed: Reduced data availability: 3 pgs inactive, 4 pgs peering (PG_AVAILABILITY) 2024-03-21T13:34:56.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:56 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:34:56.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:56 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:34:56.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:56 smithi097 ceph-mon[30040]: Health check failed: Reduced data availability: 3 pgs inactive, 4 pgs peering (PG_AVAILABILITY) 2024-03-21T13:34:56.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:56 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:34:56.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:56 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:34:57.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:57 smithi144 ceph-mon[33633]: pgmap v571: 324 pgs: 7 peering, 317 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:34:57.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:57 smithi097 ceph-mon[39416]: pgmap v571: 324 pgs: 7 peering, 317 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:34:57.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:57 smithi097 ceph-mon[30040]: pgmap v571: 324 pgs: 7 peering, 317 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:34:58.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:58 smithi144 ceph-mon[33633]: osdmap e397: 8 total, 8 up, 8 in 2024-03-21T13:34:58.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:58 smithi097 ceph-mon[39416]: osdmap e397: 8 total, 8 up, 8 in 2024-03-21T13:34:58.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:58 smithi097 ceph-mon[30040]: osdmap e397: 8 total, 8 up, 8 in 2024-03-21T13:34:59.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:34:59 smithi144 ceph-mon[33633]: pgmap v573: 324 pgs: 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1 op/s; 8 B/s, 2 objects/s recovering 2024-03-21T13:34:59.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:34:59 smithi097 ceph-mon[39416]: pgmap v573: 324 pgs: 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1 op/s; 8 B/s, 2 objects/s recovering 2024-03-21T13:34:59.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:34:59 smithi097 ceph-mon[30040]: pgmap v573: 324 pgs: 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1 op/s; 8 B/s, 2 objects/s recovering 2024-03-21T13:35:00.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:00 smithi144 ceph-mon[33633]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs inactive, 4 pgs peering) 2024-03-21T13:35:00.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:00 smithi097 ceph-mon[39416]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs inactive, 4 pgs peering) 2024-03-21T13:35:00.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:00 smithi097 ceph-mon[30040]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs inactive, 4 pgs peering) 2024-03-21T13:35:01.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:01 smithi144 ceph-mon[33633]: pgmap v574: 324 pgs: 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s; 7 B/s, 1 objects/s recovering 2024-03-21T13:35:01.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:01 smithi097 ceph-mon[39416]: pgmap v574: 324 pgs: 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s; 7 B/s, 1 objects/s recovering 2024-03-21T13:35:01.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:01 smithi097 ceph-mon[30040]: pgmap v574: 324 pgs: 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s; 7 B/s, 1 objects/s recovering 2024-03-21T13:35:02.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:35:01 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:35:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:35:03.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:03 smithi097 ceph-mon[39416]: osdmap e398: 8 total, 8 up, 8 in 2024-03-21T13:35:03.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:03 smithi097 ceph-mon[39416]: pgmap v576: 324 pgs: 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s; 6 B/s, 1 objects/s recovering 2024-03-21T13:35:03.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:03 smithi097 ceph-mon[30040]: osdmap e398: 8 total, 8 up, 8 in 2024-03-21T13:35:03.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:03 smithi097 ceph-mon[30040]: pgmap v576: 324 pgs: 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s; 6 B/s, 1 objects/s recovering 2024-03-21T13:35:03.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:03 smithi144 ceph-mon[33633]: osdmap e398: 8 total, 8 up, 8 in 2024-03-21T13:35:03.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:03 smithi144 ceph-mon[33633]: pgmap v576: 324 pgs: 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s; 6 B/s, 1 objects/s recovering 2024-03-21T13:35:04.121 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:35:03 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:35:04.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:04 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:35:04.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:04 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:35:04.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:04 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:35:05.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:05 smithi144 ceph-mon[33633]: pgmap v577: 324 pgs: 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s; 6 B/s, 1 objects/s recovering 2024-03-21T13:35:05.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:05 smithi097 ceph-mon[39416]: pgmap v577: 324 pgs: 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s; 6 B/s, 1 objects/s recovering 2024-03-21T13:35:05.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:05 smithi097 ceph-mon[30040]: pgmap v577: 324 pgs: 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s; 6 B/s, 1 objects/s recovering 2024-03-21T13:35:07.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:07 smithi144 ceph-mon[33633]: pgmap v578: 324 pgs: 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 5 B/s, 1 objects/s recovering 2024-03-21T13:35:07.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:07 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:35:07.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:07 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:35:07.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:07 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-63"}]: dispatch 2024-03-21T13:35:07.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:07 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-63"}]: dispatch 2024-03-21T13:35:08.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:07 smithi097 ceph-mon[30040]: pgmap v578: 324 pgs: 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 5 B/s, 1 objects/s recovering 2024-03-21T13:35:08.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:07 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:35:08.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:07 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:35:08.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:07 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-63"}]: dispatch 2024-03-21T13:35:08.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:07 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-63"}]: dispatch 2024-03-21T13:35:08.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:07 smithi097 ceph-mon[39416]: pgmap v578: 324 pgs: 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 5 B/s, 1 objects/s recovering 2024-03-21T13:35:08.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:07 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:35:08.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:07 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:35:08.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:07 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-63"}]: dispatch 2024-03-21T13:35:08.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:07 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-63"}]: dispatch 2024-03-21T13:35:08.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:08 smithi144 ceph-mon[33633]: osdmap e399: 8 total, 8 up, 8 in 2024-03-21T13:35:09.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:08 smithi097 ceph-mon[30040]: osdmap e399: 8 total, 8 up, 8 in 2024-03-21T13:35:09.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:08 smithi097 ceph-mon[39416]: osdmap e399: 8 total, 8 up, 8 in 2024-03-21T13:35:09.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:09 smithi144 ceph-mon[33633]: pgmap v580: 292 pgs: 292 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:35:09.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:09 smithi144 ceph-mon[33633]: osdmap e400: 8 total, 8 up, 8 in 2024-03-21T13:35:09.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:09 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:35:09.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:09 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:35:10.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:09 smithi097 ceph-mon[39416]: pgmap v580: 292 pgs: 292 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:35:10.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:09 smithi097 ceph-mon[39416]: osdmap e400: 8 total, 8 up, 8 in 2024-03-21T13:35:10.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:09 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:35:10.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:09 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:35:10.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:09 smithi097 ceph-mon[30040]: pgmap v580: 292 pgs: 292 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:35:10.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:09 smithi097 ceph-mon[30040]: osdmap e400: 8 total, 8 up, 8 in 2024-03-21T13:35:10.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:09 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:35:10.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:09 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:35:10.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:10 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:35:10.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:10 smithi144 ceph-mon[33633]: osdmap e401: 8 total, 8 up, 8 in 2024-03-21T13:35:10.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:10 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:35:10.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:10 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:35:10.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:10 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:35:10.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:10 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:35:10.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:10 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:35:10.954 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:10 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:35:10.954 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:10 smithi097 ceph-mon[39416]: osdmap e401: 8 total, 8 up, 8 in 2024-03-21T13:35:10.954 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:10 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:35:10.954 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:10 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:35:10.954 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:10 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:35:10.954 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:10 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:35:10.955 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:10 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:35:10.955 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:10 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:35:10.955 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:10 smithi097 ceph-mon[30040]: osdmap e401: 8 total, 8 up, 8 in 2024-03-21T13:35:10.955 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:10 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:35:10.955 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:10 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:35:10.955 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:10 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:35:10.955 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:10 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:35:10.955 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:10 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:35:11.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:11 smithi144 ceph-mon[33633]: pgmap v583: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:35:11.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:11 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:35:11.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:11 smithi144 ceph-mon[33633]: osdmap e402: 8 total, 8 up, 8 in 2024-03-21T13:35:11.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:11 smithi097 ceph-mon[39416]: pgmap v583: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:35:11.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:11 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:35:11.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:11 smithi097 ceph-mon[39416]: osdmap e402: 8 total, 8 up, 8 in 2024-03-21T13:35:11.950 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:11 smithi097 ceph-mon[30040]: pgmap v583: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:35:11.950 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:11 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:35:11.951 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:11 smithi097 ceph-mon[30040]: osdmap e402: 8 total, 8 up, 8 in 2024-03-21T13:35:12.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:35:11 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:35:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:35:13.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:12 smithi097 ceph-mon[39416]: osdmap e403: 8 total, 8 up, 8 in 2024-03-21T13:35:13.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:12 smithi097 ceph-mon[30040]: osdmap e403: 8 total, 8 up, 8 in 2024-03-21T13:35:13.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:12 smithi144 ceph-mon[33633]: osdmap e403: 8 total, 8 up, 8 in 2024-03-21T13:35:14.022 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:13 smithi144 ceph-mon[33633]: pgmap v586: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:35:14.023 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:13 smithi144 ceph-mon[33633]: osdmap e404: 8 total, 8 up, 8 in 2024-03-21T13:35:14.023 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:35:13 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:35:14.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:13 smithi097 ceph-mon[39416]: pgmap v586: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:35:14.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:13 smithi097 ceph-mon[39416]: osdmap e404: 8 total, 8 up, 8 in 2024-03-21T13:35:14.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:13 smithi097 ceph-mon[30040]: pgmap v586: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:35:14.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:13 smithi097 ceph-mon[30040]: osdmap e404: 8 total, 8 up, 8 in 2024-03-21T13:35:15.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:14 smithi097 ceph-mon[39416]: osdmap e405: 8 total, 8 up, 8 in 2024-03-21T13:35:15.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:14 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:35:15.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:14 smithi097 ceph-mon[30040]: osdmap e405: 8 total, 8 up, 8 in 2024-03-21T13:35:15.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:14 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:35:15.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:14 smithi144 ceph-mon[33633]: osdmap e405: 8 total, 8 up, 8 in 2024-03-21T13:35:15.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:14 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:35:16.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:15 smithi097 ceph-mon[39416]: pgmap v589: 324 pgs: 324 active+clean; 8.4 MiB data, 669 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-03-21T13:35:16.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:15 smithi097 ceph-mon[30040]: pgmap v589: 324 pgs: 324 active+clean; 8.4 MiB data, 669 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-03-21T13:35:16.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:15 smithi144 ceph-mon[33633]: pgmap v589: 324 pgs: 324 active+clean; 8.4 MiB data, 669 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-03-21T13:35:18.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:17 smithi097 ceph-mon[30040]: pgmap v590: 324 pgs: 324 active+clean; 8.4 MiB data, 669 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-03-21T13:35:18.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:17 smithi097 ceph-mon[39416]: pgmap v590: 324 pgs: 324 active+clean; 8.4 MiB data, 669 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-03-21T13:35:18.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:17 smithi144 ceph-mon[33633]: pgmap v590: 324 pgs: 324 active+clean; 8.4 MiB data, 669 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-03-21T13:35:20.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:19 smithi097 ceph-mon[30040]: pgmap v591: 324 pgs: 324 active+clean; 8.4 MiB data, 673 MiB used, 715 GiB / 715 GiB avail; 2.9 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-03-21T13:35:20.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:19 smithi097 ceph-mon[39416]: pgmap v591: 324 pgs: 324 active+clean; 8.4 MiB data, 673 MiB used, 715 GiB / 715 GiB avail; 2.9 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-03-21T13:35:20.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:19 smithi144 ceph-mon[33633]: pgmap v591: 324 pgs: 324 active+clean; 8.4 MiB data, 673 MiB used, 715 GiB / 715 GiB avail; 2.9 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-03-21T13:35:21.949 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:21 smithi097 ceph-mon[39416]: pgmap v592: 324 pgs: 324 active+clean; 8.4 MiB data, 673 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-03-21T13:35:21.950 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:21 smithi097 ceph-mon[30040]: pgmap v592: 324 pgs: 324 active+clean; 8.4 MiB data, 673 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-03-21T13:35:22.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:21 smithi144 ceph-mon[33633]: pgmap v592: 324 pgs: 324 active+clean; 8.4 MiB data, 673 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-03-21T13:35:22.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:35:21 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:35:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:35:23.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:23 smithi097 ceph-mon[30040]: osdmap e406: 8 total, 8 up, 8 in 2024-03-21T13:35:23.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:23 smithi097 ceph-mon[30040]: pgmap v594: 324 pgs: 324 active+clean; 8.4 MiB data, 673 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:35:23.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:23 smithi097 ceph-mon[39416]: osdmap e406: 8 total, 8 up, 8 in 2024-03-21T13:35:23.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:23 smithi097 ceph-mon[39416]: pgmap v594: 324 pgs: 324 active+clean; 8.4 MiB data, 673 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:35:23.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:23 smithi144 ceph-mon[33633]: osdmap e406: 8 total, 8 up, 8 in 2024-03-21T13:35:23.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:23 smithi144 ceph-mon[33633]: pgmap v594: 324 pgs: 324 active+clean; 8.4 MiB data, 673 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:35:24.121 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:35:23 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:35:24.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:24 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:35:24.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:24 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:35:24.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:24 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-65"}]: dispatch 2024-03-21T13:35:24.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:24 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-65"}]: dispatch 2024-03-21T13:35:24.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:24 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:35:24.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:24 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:35:24.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:24 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:35:24.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:24 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-65"}]: dispatch 2024-03-21T13:35:24.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:24 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-65"}]: dispatch 2024-03-21T13:35:24.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:24 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:35:24.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:24 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:35:24.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:24 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:35:24.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:24 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-65"}]: dispatch 2024-03-21T13:35:24.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:24 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-65"}]: dispatch 2024-03-21T13:35:24.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:24 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:35:25.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:25 smithi144 ceph-mon[33633]: osdmap e407: 8 total, 8 up, 8 in 2024-03-21T13:35:25.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:25 smithi144 ceph-mon[33633]: pgmap v596: 292 pgs: 292 active+clean; 8.4 MiB data, 673 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:35:25.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:25 smithi097 ceph-mon[30040]: osdmap e407: 8 total, 8 up, 8 in 2024-03-21T13:35:25.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:25 smithi097 ceph-mon[30040]: pgmap v596: 292 pgs: 292 active+clean; 8.4 MiB data, 673 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:35:25.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:25 smithi097 ceph-mon[39416]: osdmap e407: 8 total, 8 up, 8 in 2024-03-21T13:35:25.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:25 smithi097 ceph-mon[39416]: pgmap v596: 292 pgs: 292 active+clean; 8.4 MiB data, 673 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:35:26.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:26 smithi144 ceph-mon[33633]: osdmap e408: 8 total, 8 up, 8 in 2024-03-21T13:35:26.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:26 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:35:26.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:26 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:35:26.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:26 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:35:26.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:26 smithi097 ceph-mon[39416]: osdmap e408: 8 total, 8 up, 8 in 2024-03-21T13:35:26.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:26 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:35:26.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:26 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:35:26.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:26 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:35:26.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:26 smithi097 ceph-mon[30040]: osdmap e408: 8 total, 8 up, 8 in 2024-03-21T13:35:26.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:26 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:35:26.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:26 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:35:26.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:26 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:35:27.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:27 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:35:27.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:27 smithi144 ceph-mon[33633]: osdmap e409: 8 total, 8 up, 8 in 2024-03-21T13:35:27.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:27 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:35:27.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:27 smithi144 ceph-mon[33633]: pgmap v599: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 673 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:35:27.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:27 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:35:27.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:27 smithi097 ceph-mon[39416]: osdmap e409: 8 total, 8 up, 8 in 2024-03-21T13:35:27.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:27 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:35:27.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:27 smithi097 ceph-mon[39416]: pgmap v599: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 673 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:35:27.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:27 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:35:27.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:27 smithi097 ceph-mon[30040]: osdmap e409: 8 total, 8 up, 8 in 2024-03-21T13:35:27.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:27 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:35:27.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:27 smithi097 ceph-mon[30040]: pgmap v599: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 673 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:35:28.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:28 smithi144 ceph-mon[33633]: osdmap e410: 8 total, 8 up, 8 in 2024-03-21T13:35:28.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:28 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:35:28.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:28 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:35:28.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:28 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-67"}]: dispatch 2024-03-21T13:35:28.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:28 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-67"}]: dispatch 2024-03-21T13:35:28.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:28 smithi097 ceph-mon[39416]: osdmap e410: 8 total, 8 up, 8 in 2024-03-21T13:35:28.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:28 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:35:28.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:28 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:35:28.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:28 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-67"}]: dispatch 2024-03-21T13:35:28.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:28 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-67"}]: dispatch 2024-03-21T13:35:28.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:28 smithi097 ceph-mon[30040]: osdmap e410: 8 total, 8 up, 8 in 2024-03-21T13:35:28.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:28 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:35:28.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:28 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:35:28.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:28 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-67"}]: dispatch 2024-03-21T13:35:28.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:28 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-67"}]: dispatch 2024-03-21T13:35:29.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:29 smithi144 ceph-mon[33633]: osdmap e411: 8 total, 8 up, 8 in 2024-03-21T13:35:29.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:29 smithi144 ceph-mon[33633]: pgmap v602: 292 pgs: 292 active+clean; 8.4 MiB data, 678 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T13:35:29.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:29 smithi097 ceph-mon[39416]: osdmap e411: 8 total, 8 up, 8 in 2024-03-21T13:35:29.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:29 smithi097 ceph-mon[39416]: pgmap v602: 292 pgs: 292 active+clean; 8.4 MiB data, 678 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T13:35:29.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:29 smithi097 ceph-mon[30040]: osdmap e411: 8 total, 8 up, 8 in 2024-03-21T13:35:29.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:29 smithi097 ceph-mon[30040]: pgmap v602: 292 pgs: 292 active+clean; 8.4 MiB data, 678 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T13:35:29.920 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c8600 2024-03-21T13:35:29.920 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c8730 2024-03-21T13:35:29.920 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c4d20 2024-03-21T13:35:29.921 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c4e50 2024-03-21T13:35:29.921 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c4f80 2024-03-21T13:35:29.921 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c8860 2024-03-21T13:35:29.921 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c8d30 2024-03-21T13:35:29.921 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c8e80 2024-03-21T13:35:29.921 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c9460 2024-03-21T13:35:29.921 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c8990 2024-03-21T13:35:29.921 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c8ac0 2024-03-21T13:35:29.921 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c8bf0 2024-03-21T13:35:29.921 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c9570 2024-03-21T13:35:29.921 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c8fb0 2024-03-21T13:35:29.921 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c90e0 2024-03-21T13:35:29.922 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c91f0 2024-03-21T13:35:29.922 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c9320 2024-03-21T13:35:29.922 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c9b70 2024-03-21T13:35:29.922 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c96c0 2024-03-21T13:35:29.922 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c97f0 2024-03-21T13:35:29.922 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c9900 2024-03-21T13:35:29.922 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c9a50 2024-03-21T13:35:29.922 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5ca170 2024-03-21T13:35:29.922 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c9cc0 2024-03-21T13:35:29.922 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c9df0 2024-03-21T13:35:29.922 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c9f00 2024-03-21T13:35:29.922 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5ca050 2024-03-21T13:35:29.922 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5ca750 2024-03-21T13:35:29.922 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5ca2a0 2024-03-21T13:35:29.923 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5ca3d0 2024-03-21T13:35:29.923 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5ca4e0 2024-03-21T13:35:29.923 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5ca630 2024-03-21T13:35:29.923 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cad50 2024-03-21T13:35:29.923 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5ca8a0 2024-03-21T13:35:29.923 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5ca9f0 2024-03-21T13:35:29.923 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cab40 2024-03-21T13:35:29.923 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5caea0 2024-03-21T13:35:29.923 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cb3c0 2024-03-21T13:35:29.923 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cb4f0 2024-03-21T13:35:29.923 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c8240 2024-03-21T13:35:29.923 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c8370 2024-03-21T13:35:29.923 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5c84a0 2024-03-21T13:35:29.923 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cb620 2024-03-21T13:35:29.923 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cbaf0 2024-03-21T13:35:29.924 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cbc40 2024-03-21T13:35:29.924 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cafd0 2024-03-21T13:35:29.924 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cb100 2024-03-21T13:35:29.924 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cb230 2024-03-21T13:35:29.924 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cbd70 2024-03-21T13:35:29.924 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cc240 2024-03-21T13:35:29.924 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cc3c0 2024-03-21T13:35:29.924 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cc980 2024-03-21T13:35:29.924 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cbea0 2024-03-21T13:35:29.924 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cbfd0 2024-03-21T13:35:29.924 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cc100 2024-03-21T13:35:29.924 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5ccad0 2024-03-21T13:35:29.924 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cc4d0 2024-03-21T13:35:29.924 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cc620 2024-03-21T13:35:29.924 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cc750 2024-03-21T13:35:29.924 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5ccc20 2024-03-21T13:35:29.925 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cd0f0 2024-03-21T13:35:29.925 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cd220 2024-03-21T13:35:29.925 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cb750 2024-03-21T13:35:29.925 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cb880 2024-03-21T13:35:29.925 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cb9b0 2024-03-21T13:35:29.925 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cd350 2024-03-21T13:35:29.925 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cd870 2024-03-21T13:35:29.925 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cd9c0 2024-03-21T13:35:29.925 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cdfa0 2024-03-21T13:35:29.925 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cd480 2024-03-21T13:35:29.925 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cd5d0 2024-03-21T13:35:29.925 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cd700 2024-03-21T13:35:29.925 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5ce0f0 2024-03-21T13:35:29.925 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cdaf0 2024-03-21T13:35:29.925 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cdc40 2024-03-21T13:35:29.926 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cdd70 2024-03-21T13:35:29.926 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5ce220 2024-03-21T13:35:29.926 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5ce6d0 2024-03-21T13:35:29.926 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5ce800 2024-03-21T13:35:29.926 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cde80 2024-03-21T13:35:29.926 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5ce930 2024-03-21T13:35:29.926 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cead0 2024-03-21T13:35:30.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:30 smithi144 ceph-mon[33633]: osdmap e412: 8 total, 8 up, 8 in 2024-03-21T13:35:30.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:30 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:35:30.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:30 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:35:30.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:30 smithi097 ceph-mon[39416]: osdmap e412: 8 total, 8 up, 8 in 2024-03-21T13:35:30.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:30 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:35:30.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:30 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:35:30.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:30 smithi097 ceph-mon[30040]: osdmap e412: 8 total, 8 up, 8 in 2024-03-21T13:35:30.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:30 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:35:30.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:30 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:35:31.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:31 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:35:31.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:31 smithi144 ceph-mon[33633]: osdmap e413: 8 total, 8 up, 8 in 2024-03-21T13:35:31.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:31 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:35:31.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:31 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:35:31.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:31 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:35:31.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:31 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-69"}]: dispatch 2024-03-21T13:35:31.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:31 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-69"}]: dispatch 2024-03-21T13:35:31.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:31 smithi144 ceph-mon[33633]: pgmap v605: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 678 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T13:35:31.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:31 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:35:31.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:31 smithi097 ceph-mon[39416]: osdmap e413: 8 total, 8 up, 8 in 2024-03-21T13:35:31.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:31 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:35:31.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:31 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:35:31.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:31 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:35:31.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:31 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-69"}]: dispatch 2024-03-21T13:35:31.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:31 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-69"}]: dispatch 2024-03-21T13:35:31.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:31 smithi097 ceph-mon[39416]: pgmap v605: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 678 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T13:35:31.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:31 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:35:31.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:31 smithi097 ceph-mon[30040]: osdmap e413: 8 total, 8 up, 8 in 2024-03-21T13:35:31.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:31 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:35:31.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:31 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:35:31.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:31 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:35:31.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:31 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-69"}]: dispatch 2024-03-21T13:35:31.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:31 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-69"}]: dispatch 2024-03-21T13:35:31.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:31 smithi097 ceph-mon[30040]: pgmap v605: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 678 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T13:35:32.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:35:31 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:35:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:35:32.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:32 smithi144 ceph-mon[33633]: osdmap e414: 8 total, 8 up, 8 in 2024-03-21T13:35:32.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:32 smithi097 ceph-mon[30040]: osdmap e414: 8 total, 8 up, 8 in 2024-03-21T13:35:32.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:32 smithi097 ceph-mon[39416]: osdmap e414: 8 total, 8 up, 8 in 2024-03-21T13:35:33.767 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:33 smithi144 ceph-mon[33633]: osdmap e415: 8 total, 8 up, 8 in 2024-03-21T13:35:33.768 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:33 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:35:33.768 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:33 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:35:33.768 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:33 smithi144 ceph-mon[33633]: pgmap v608: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 678 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:35:33.768 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:33 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:35:33.768 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:33 smithi144 ceph-mon[33633]: osdmap e416: 8 total, 8 up, 8 in 2024-03-21T13:35:33.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:33 smithi097 ceph-mon[39416]: osdmap e415: 8 total, 8 up, 8 in 2024-03-21T13:35:33.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:33 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:35:33.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:33 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:35:33.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:33 smithi097 ceph-mon[39416]: pgmap v608: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 678 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:35:33.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:33 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:35:33.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:33 smithi097 ceph-mon[39416]: osdmap e416: 8 total, 8 up, 8 in 2024-03-21T13:35:33.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:33 smithi097 ceph-mon[30040]: osdmap e415: 8 total, 8 up, 8 in 2024-03-21T13:35:33.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:33 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:35:33.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:33 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:35:33.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:33 smithi097 ceph-mon[30040]: pgmap v608: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 678 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:35:33.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:33 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:35:33.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:33 smithi097 ceph-mon[30040]: osdmap e416: 8 total, 8 up, 8 in 2024-03-21T13:35:34.121 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:35:33 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:35:34.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:34 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:35:34.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:34 smithi097 ceph-mon[39416]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:35:34.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:34 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:35:34.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:34 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:35:34.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:34 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:35:34.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:34 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:35:34.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:34 smithi097 ceph-mon[39416]: osdmap e417: 8 total, 8 up, 8 in 2024-03-21T13:35:34.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:34 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:35:34.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:34 smithi097 ceph-mon[30040]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:35:34.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:34 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:35:34.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:34 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:35:34.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:34 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:35:34.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:34 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:35:34.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:34 smithi097 ceph-mon[30040]: osdmap e417: 8 total, 8 up, 8 in 2024-03-21T13:35:34.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:34 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:35:34.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:34 smithi144 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:35:34.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:34 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:35:34.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:34 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:35:34.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:34 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:35:34.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:34 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:35:34.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:34 smithi144 ceph-mon[33633]: osdmap e417: 8 total, 8 up, 8 in 2024-03-21T13:35:35.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:35 smithi097 ceph-mon[30040]: pgmap v611: 324 pgs: 8 creating+peering, 13 creating+activating, 303 active+clean; 8.4 MiB data, 687 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-21T13:35:35.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:35 smithi097 ceph-mon[39416]: pgmap v611: 324 pgs: 8 creating+peering, 13 creating+activating, 303 active+clean; 8.4 MiB data, 687 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-21T13:35:35.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:35 smithi144 ceph-mon[33633]: pgmap v611: 324 pgs: 8 creating+peering, 13 creating+activating, 303 active+clean; 8.4 MiB data, 687 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-21T13:35:36.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:36 smithi097 ceph-mon[30040]: osdmap e418: 8 total, 8 up, 8 in 2024-03-21T13:35:36.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:36 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:35:36.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:36 smithi097 ceph-mon[39416]: osdmap e418: 8 total, 8 up, 8 in 2024-03-21T13:35:36.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:36 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:35:36.870 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:36 smithi144 ceph-mon[33633]: osdmap e418: 8 total, 8 up, 8 in 2024-03-21T13:35:36.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:36 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:35:37.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:37 smithi097 ceph-mon[39416]: pgmap v613: 324 pgs: 8 creating+peering, 13 creating+activating, 303 active+clean; 8.4 MiB data, 687 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 737 B/s wr, 2 op/s 2024-03-21T13:35:37.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:37 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:35:37.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:37 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:35:37.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:37 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:35:37.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:37 smithi097 ceph-mon[30040]: pgmap v613: 324 pgs: 8 creating+peering, 13 creating+activating, 303 active+clean; 8.4 MiB data, 687 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 737 B/s wr, 2 op/s 2024-03-21T13:35:37.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:37 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:35:37.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:37 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:35:37.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:37 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:35:37.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:37 smithi144 ceph-mon[33633]: pgmap v613: 324 pgs: 8 creating+peering, 13 creating+activating, 303 active+clean; 8.4 MiB data, 687 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 737 B/s wr, 2 op/s 2024-03-21T13:35:37.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:37 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:35:37.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:37 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:35:37.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:37 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:35:39.873 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:39 smithi144 ceph-mon[33633]: pgmap v614: 324 pgs: 324 active+clean; 8.4 MiB data, 687 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-03-21T13:35:40.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:39 smithi097 ceph-mon[39416]: pgmap v614: 324 pgs: 324 active+clean; 8.4 MiB data, 687 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-03-21T13:35:40.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:39 smithi097 ceph-mon[30040]: pgmap v614: 324 pgs: 324 active+clean; 8.4 MiB data, 687 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-03-21T13:35:40.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:40 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:35:40.954 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:40 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:35:40.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:40 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:35:41.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:41 smithi144 ceph-mon[33633]: pgmap v615: 324 pgs: 324 active+clean; 8.4 MiB data, 687 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-03-21T13:35:41.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:41 smithi097 ceph-mon[39416]: pgmap v615: 324 pgs: 324 active+clean; 8.4 MiB data, 687 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-03-21T13:35:41.950 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:41 smithi097 ceph-mon[30040]: pgmap v615: 324 pgs: 324 active+clean; 8.4 MiB data, 687 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-03-21T13:35:42.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:35:41 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:35:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:35:42.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:42 smithi144 ceph-mon[33633]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:35:43.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:42 smithi097 ceph-mon[30040]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:35:43.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:42 smithi097 ceph-mon[39416]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:35:43.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:43 smithi144 ceph-mon[33633]: pgmap v616: 324 pgs: 324 active+clean; 8.4 MiB data, 687 MiB used, 715 GiB / 715 GiB avail; 753 B/s rd, 1.6 KiB/s wr, 3 op/s 2024-03-21T13:35:43.871 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:35:43 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:35:44.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:43 smithi097 ceph-mon[30040]: pgmap v616: 324 pgs: 324 active+clean; 8.4 MiB data, 687 MiB used, 715 GiB / 715 GiB avail; 753 B/s rd, 1.6 KiB/s wr, 3 op/s 2024-03-21T13:35:44.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:43 smithi097 ceph-mon[39416]: pgmap v616: 324 pgs: 324 active+clean; 8.4 MiB data, 687 MiB used, 715 GiB / 715 GiB avail; 753 B/s rd, 1.6 KiB/s wr, 3 op/s 2024-03-21T13:35:44.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:44 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:35:45.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:44 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:35:45.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:44 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:35:45.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:45 smithi144 ceph-mon[33633]: pgmap v617: 324 pgs: 324 active+clean; 8.4 MiB data, 687 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-03-21T13:35:46.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:45 smithi097 ceph-mon[39416]: pgmap v617: 324 pgs: 324 active+clean; 8.4 MiB data, 687 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-03-21T13:35:46.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:45 smithi097 ceph-mon[30040]: pgmap v617: 324 pgs: 324 active+clean; 8.4 MiB data, 687 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-03-21T13:35:47.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:47 smithi144 ceph-mon[33633]: pgmap v618: 324 pgs: 324 active+clean; 8.4 MiB data, 687 MiB used, 715 GiB / 715 GiB avail; 1011 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-03-21T13:35:48.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:47 smithi097 ceph-mon[39416]: pgmap v618: 324 pgs: 324 active+clean; 8.4 MiB data, 687 MiB used, 715 GiB / 715 GiB avail; 1011 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-03-21T13:35:48.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:47 smithi097 ceph-mon[30040]: pgmap v618: 324 pgs: 324 active+clean; 8.4 MiB data, 687 MiB used, 715 GiB / 715 GiB avail; 1011 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-03-21T13:35:49.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:49 smithi144 ceph-mon[33633]: pgmap v619: 324 pgs: 324 active+clean; 8.4 MiB data, 687 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-03-21T13:35:50.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:49 smithi097 ceph-mon[39416]: pgmap v619: 324 pgs: 324 active+clean; 8.4 MiB data, 687 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-03-21T13:35:50.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:49 smithi097 ceph-mon[30040]: pgmap v619: 324 pgs: 324 active+clean; 8.4 MiB data, 687 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-03-21T13:35:51.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:51 smithi144 ceph-mon[33633]: pgmap v620: 324 pgs: 324 active+clean; 8.4 MiB data, 687 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-21T13:35:51.949 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:51 smithi097 ceph-mon[30040]: pgmap v620: 324 pgs: 324 active+clean; 8.4 MiB data, 687 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-21T13:35:51.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:51 smithi097 ceph-mon[39416]: pgmap v620: 324 pgs: 324 active+clean; 8.4 MiB data, 687 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-21T13:35:52.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:35:51 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:35:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:35:54.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:53 smithi097 ceph-mon[39416]: pgmap v621: 324 pgs: 324 active+clean; 8.4 MiB data, 687 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-21T13:35:54.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:53 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "294.16", "id": [6, 4]}]: dispatch 2024-03-21T13:35:54.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:53 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "294.17", "id": [6, 4]}]: dispatch 2024-03-21T13:35:54.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:53 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "294.16", "id": [6, 4]}]: dispatch 2024-03-21T13:35:54.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:53 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "294.17", "id": [6, 4]}]: dispatch 2024-03-21T13:35:54.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:53 smithi097 ceph-mon[30040]: pgmap v621: 324 pgs: 324 active+clean; 8.4 MiB data, 687 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-21T13:35:54.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:53 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "294.16", "id": [6, 4]}]: dispatch 2024-03-21T13:35:54.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:53 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "294.17", "id": [6, 4]}]: dispatch 2024-03-21T13:35:54.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:53 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "294.16", "id": [6, 4]}]: dispatch 2024-03-21T13:35:54.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:53 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "294.17", "id": [6, 4]}]: dispatch 2024-03-21T13:35:54.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:53 smithi144 ceph-mon[33633]: pgmap v621: 324 pgs: 324 active+clean; 8.4 MiB data, 687 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-21T13:35:54.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:53 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "294.16", "id": [6, 4]}]: dispatch 2024-03-21T13:35:54.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:53 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "294.17", "id": [6, 4]}]: dispatch 2024-03-21T13:35:54.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:53 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "294.16", "id": [6, 4]}]: dispatch 2024-03-21T13:35:54.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:53 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "294.17", "id": [6, 4]}]: dispatch 2024-03-21T13:35:54.121 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:35:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:35:55.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:54 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "294.16", "id": [6, 4]}]': finished 2024-03-21T13:35:55.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:54 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "294.17", "id": [6, 4]}]': finished 2024-03-21T13:35:55.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:54 smithi097 ceph-mon[39416]: osdmap e419: 8 total, 8 up, 8 in 2024-03-21T13:35:55.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:54 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:35:55.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:54 smithi097 ceph-mon[39416]: osdmap e420: 8 total, 8 up, 8 in 2024-03-21T13:35:55.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:54 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "294.16", "id": [6, 4]}]': finished 2024-03-21T13:35:55.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:54 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "294.17", "id": [6, 4]}]': finished 2024-03-21T13:35:55.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:54 smithi097 ceph-mon[30040]: osdmap e419: 8 total, 8 up, 8 in 2024-03-21T13:35:55.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:54 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:35:55.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:54 smithi097 ceph-mon[30040]: osdmap e420: 8 total, 8 up, 8 in 2024-03-21T13:35:55.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:54 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "294.16", "id": [6, 4]}]': finished 2024-03-21T13:35:55.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:54 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "294.17", "id": [6, 4]}]': finished 2024-03-21T13:35:55.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:54 smithi144 ceph-mon[33633]: osdmap e419: 8 total, 8 up, 8 in 2024-03-21T13:35:55.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:54 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:35:55.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:54 smithi144 ceph-mon[33633]: osdmap e420: 8 total, 8 up, 8 in 2024-03-21T13:35:56.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:55 smithi097 ceph-mon[39416]: pgmap v623: 324 pgs: 324 active+clean; 8.4 MiB data, 687 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 307 B/s wr, 2 op/s 2024-03-21T13:35:56.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:55 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:35:56.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:55 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:35:56.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:55 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:35:56.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:55 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-71"}]: dispatch 2024-03-21T13:35:56.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:55 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-71"}]: dispatch 2024-03-21T13:35:56.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:55 smithi097 ceph-mon[30040]: pgmap v623: 324 pgs: 324 active+clean; 8.4 MiB data, 687 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 307 B/s wr, 2 op/s 2024-03-21T13:35:56.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:55 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:35:56.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:55 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:35:56.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:55 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:35:56.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:55 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-71"}]: dispatch 2024-03-21T13:35:56.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:55 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-71"}]: dispatch 2024-03-21T13:35:56.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:55 smithi144 ceph-mon[33633]: pgmap v623: 324 pgs: 324 active+clean; 8.4 MiB data, 687 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 307 B/s wr, 2 op/s 2024-03-21T13:35:56.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:55 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:35:56.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:55 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:35:56.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:55 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:35:56.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:55 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-71"}]: dispatch 2024-03-21T13:35:56.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:55 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-71"}]: dispatch 2024-03-21T13:35:57.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:56 smithi097 ceph-mon[39416]: osdmap e421: 8 total, 8 up, 8 in 2024-03-21T13:35:57.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:56 smithi097 ceph-mon[30040]: osdmap e421: 8 total, 8 up, 8 in 2024-03-21T13:35:57.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:56 smithi144 ceph-mon[33633]: osdmap e421: 8 total, 8 up, 8 in 2024-03-21T13:35:58.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:57 smithi097 ceph-mon[39416]: pgmap v626: 292 pgs: 292 active+clean; 8.4 MiB data, 687 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:35:58.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:57 smithi097 ceph-mon[39416]: osdmap e422: 8 total, 8 up, 8 in 2024-03-21T13:35:58.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:57 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:35:58.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:57 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:35:58.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:57 smithi097 ceph-mon[30040]: pgmap v626: 292 pgs: 292 active+clean; 8.4 MiB data, 687 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:35:58.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:57 smithi097 ceph-mon[30040]: osdmap e422: 8 total, 8 up, 8 in 2024-03-21T13:35:58.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:57 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:35:58.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:57 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:35:58.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:57 smithi144 ceph-mon[33633]: pgmap v626: 292 pgs: 292 active+clean; 8.4 MiB data, 687 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:35:58.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:57 smithi144 ceph-mon[33633]: osdmap e422: 8 total, 8 up, 8 in 2024-03-21T13:35:58.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:57 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:35:58.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:57 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:35:59.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:58 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:35:59.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:58 smithi097 ceph-mon[39416]: osdmap e423: 8 total, 8 up, 8 in 2024-03-21T13:35:59.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:58 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:35:59.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:58 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:35:59.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:58 smithi097 ceph-mon[30040]: osdmap e423: 8 total, 8 up, 8 in 2024-03-21T13:35:59.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:58 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:35:59.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:58 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:35:59.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:58 smithi144 ceph-mon[33633]: osdmap e423: 8 total, 8 up, 8 in 2024-03-21T13:35:59.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:58 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:36:00.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:59 smithi097 ceph-mon[30040]: pgmap v629: 324 pgs: 324 active+clean; 8.4 MiB data, 688 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:36:00.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:35:59 smithi097 ceph-mon[30040]: osdmap e424: 8 total, 8 up, 8 in 2024-03-21T13:36:00.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:59 smithi097 ceph-mon[39416]: pgmap v629: 324 pgs: 324 active+clean; 8.4 MiB data, 688 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:36:00.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:35:59 smithi097 ceph-mon[39416]: osdmap e424: 8 total, 8 up, 8 in 2024-03-21T13:36:00.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:59 smithi144 ceph-mon[33633]: pgmap v629: 324 pgs: 324 active+clean; 8.4 MiB data, 688 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:36:00.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:35:59 smithi144 ceph-mon[33633]: osdmap e424: 8 total, 8 up, 8 in 2024-03-21T13:36:01.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:00 smithi097 ceph-mon[30040]: osdmap e425: 8 total, 8 up, 8 in 2024-03-21T13:36:01.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:00 smithi097 ceph-mon[39416]: osdmap e425: 8 total, 8 up, 8 in 2024-03-21T13:36:01.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:00 smithi144 ceph-mon[33633]: osdmap e425: 8 total, 8 up, 8 in 2024-03-21T13:36:01.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:01 smithi097 ceph-mon[39416]: pgmap v632: 324 pgs: 324 active+clean; 8.4 MiB data, 688 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:36:01.950 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:01 smithi097 ceph-mon[30040]: pgmap v632: 324 pgs: 324 active+clean; 8.4 MiB data, 688 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:36:02.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:01 smithi144 ceph-mon[33633]: pgmap v632: 324 pgs: 324 active+clean; 8.4 MiB data, 688 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:36:02.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:36:01 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:36:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:36:04.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:03 smithi097 ceph-mon[30040]: pgmap v633: 324 pgs: 324 active+clean; 8.4 MiB data, 688 MiB used, 715 GiB / 715 GiB avail; 870 B/s rd, 870 B/s wr, 1 op/s 2024-03-21T13:36:04.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:03 smithi097 ceph-mon[39416]: pgmap v633: 324 pgs: 324 active+clean; 8.4 MiB data, 688 MiB used, 715 GiB / 715 GiB avail; 870 B/s rd, 870 B/s wr, 1 op/s 2024-03-21T13:36:04.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:03 smithi144 ceph-mon[33633]: pgmap v633: 324 pgs: 324 active+clean; 8.4 MiB data, 688 MiB used, 715 GiB / 715 GiB avail; 870 B/s rd, 870 B/s wr, 1 op/s 2024-03-21T13:36:04.121 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:36:03 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:36:05.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:04 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:36:05.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:04 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:36:05.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:04 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:36:06.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:05 smithi097 ceph-mon[39416]: pgmap v634: 324 pgs: 324 active+clean; 8.4 MiB data, 688 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.0 KiB/s wr, 4 op/s 2024-03-21T13:36:06.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:05 smithi097 ceph-mon[30040]: pgmap v634: 324 pgs: 324 active+clean; 8.4 MiB data, 688 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.0 KiB/s wr, 4 op/s 2024-03-21T13:36:06.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:05 smithi144 ceph-mon[33633]: pgmap v634: 324 pgs: 324 active+clean; 8.4 MiB data, 688 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.0 KiB/s wr, 4 op/s 2024-03-21T13:36:08.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:07 smithi097 ceph-mon[39416]: pgmap v635: 324 pgs: 324 active+clean; 8.4 MiB data, 688 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-21T13:36:08.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:07 smithi097 ceph-mon[30040]: pgmap v635: 324 pgs: 324 active+clean; 8.4 MiB data, 688 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-21T13:36:08.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:07 smithi144 ceph-mon[33633]: pgmap v635: 324 pgs: 324 active+clean; 8.4 MiB data, 688 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-21T13:36:10.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:09 smithi097 ceph-mon[39416]: pgmap v636: 324 pgs: 324 active+clean; 8.4 MiB data, 688 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 207 B/s wr, 2 op/s 2024-03-21T13:36:10.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:09 smithi097 ceph-mon[30040]: pgmap v636: 324 pgs: 324 active+clean; 8.4 MiB data, 688 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 207 B/s wr, 2 op/s 2024-03-21T13:36:10.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:09 smithi144 ceph-mon[33633]: pgmap v636: 324 pgs: 324 active+clean; 8.4 MiB data, 688 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 207 B/s wr, 2 op/s 2024-03-21T13:36:10.954 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:10 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:36:10.954 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:10 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:36:10.954 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:10 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-73"}]: dispatch 2024-03-21T13:36:10.954 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:10 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-73"}]: dispatch 2024-03-21T13:36:10.955 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:10 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:36:10.955 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:10 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:36:10.955 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:10 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:36:10.955 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:10 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-73"}]: dispatch 2024-03-21T13:36:10.955 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:10 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-73"}]: dispatch 2024-03-21T13:36:10.955 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:10 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:36:11.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:10 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:36:11.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:10 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:36:11.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:10 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-73"}]: dispatch 2024-03-21T13:36:11.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:10 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-73"}]: dispatch 2024-03-21T13:36:11.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:10 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:36:12.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:11 smithi097 ceph-mon[30040]: pgmap v637: 324 pgs: 324 active+clean; 8.4 MiB data, 688 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 188 B/s wr, 2 op/s 2024-03-21T13:36:12.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:11 smithi097 ceph-mon[30040]: osdmap e426: 8 total, 8 up, 8 in 2024-03-21T13:36:12.054 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:36:11 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:36:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:36:12.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:11 smithi097 ceph-mon[39416]: pgmap v637: 324 pgs: 324 active+clean; 8.4 MiB data, 688 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 188 B/s wr, 2 op/s 2024-03-21T13:36:12.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:11 smithi097 ceph-mon[39416]: osdmap e426: 8 total, 8 up, 8 in 2024-03-21T13:36:12.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:11 smithi144 ceph-mon[33633]: pgmap v637: 324 pgs: 324 active+clean; 8.4 MiB data, 688 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 188 B/s wr, 2 op/s 2024-03-21T13:36:12.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:11 smithi144 ceph-mon[33633]: osdmap e426: 8 total, 8 up, 8 in 2024-03-21T13:36:13.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:12 smithi097 ceph-mon[30040]: osdmap e427: 8 total, 8 up, 8 in 2024-03-21T13:36:13.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:12 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:36:13.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:12 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:36:13.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:12 smithi097 ceph-mon[39416]: osdmap e427: 8 total, 8 up, 8 in 2024-03-21T13:36:13.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:12 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:36:13.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:12 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:36:13.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:12 smithi144 ceph-mon[33633]: osdmap e427: 8 total, 8 up, 8 in 2024-03-21T13:36:13.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:12 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:36:13.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:12 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:36:14.022 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:13 smithi144 ceph-mon[33633]: pgmap v640: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 688 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:36:14.022 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:13 smithi144 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:36:14.023 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:13 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:36:14.023 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:13 smithi144 ceph-mon[33633]: osdmap e428: 8 total, 8 up, 8 in 2024-03-21T13:36:14.023 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:13 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:36:14.023 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:36:13 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:36:14.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:13 smithi097 ceph-mon[39416]: pgmap v640: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 688 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:36:14.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:13 smithi097 ceph-mon[39416]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:36:14.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:13 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:36:14.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:13 smithi097 ceph-mon[39416]: osdmap e428: 8 total, 8 up, 8 in 2024-03-21T13:36:14.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:13 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:36:14.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:13 smithi097 ceph-mon[30040]: pgmap v640: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 688 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:36:14.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:13 smithi097 ceph-mon[30040]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:36:14.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:13 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:36:14.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:13 smithi097 ceph-mon[30040]: osdmap e428: 8 total, 8 up, 8 in 2024-03-21T13:36:14.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:13 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:36:15.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:14 smithi097 ceph-mon[39416]: osdmap e429: 8 total, 8 up, 8 in 2024-03-21T13:36:15.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:14 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:36:15.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:14 smithi097 ceph-mon[30040]: osdmap e429: 8 total, 8 up, 8 in 2024-03-21T13:36:15.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:14 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:36:15.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:14 smithi144 ceph-mon[33633]: osdmap e429: 8 total, 8 up, 8 in 2024-03-21T13:36:15.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:14 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:36:16.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:15 smithi097 ceph-mon[39416]: pgmap v643: 324 pgs: 324 active+clean; 8.4 MiB data, 693 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:36:16.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:15 smithi097 ceph-mon[39416]: osdmap e430: 8 total, 8 up, 8 in 2024-03-21T13:36:16.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:15 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:36:16.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:15 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:36:16.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:15 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-75"}]: dispatch 2024-03-21T13:36:16.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:15 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-75"}]: dispatch 2024-03-21T13:36:16.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:15 smithi097 ceph-mon[30040]: pgmap v643: 324 pgs: 324 active+clean; 8.4 MiB data, 693 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:36:16.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:15 smithi097 ceph-mon[30040]: osdmap e430: 8 total, 8 up, 8 in 2024-03-21T13:36:16.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:15 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:36:16.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:15 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:36:16.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:15 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-75"}]: dispatch 2024-03-21T13:36:16.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:15 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-75"}]: dispatch 2024-03-21T13:36:16.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:15 smithi144 ceph-mon[33633]: pgmap v643: 324 pgs: 324 active+clean; 8.4 MiB data, 693 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:36:16.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:15 smithi144 ceph-mon[33633]: osdmap e430: 8 total, 8 up, 8 in 2024-03-21T13:36:16.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:15 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:36:16.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:15 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:36:16.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:15 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-75"}]: dispatch 2024-03-21T13:36:16.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:15 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-75"}]: dispatch 2024-03-21T13:36:17.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:16 smithi097 ceph-mon[39416]: osdmap e431: 8 total, 8 up, 8 in 2024-03-21T13:36:17.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:16 smithi097 ceph-mon[30040]: osdmap e431: 8 total, 8 up, 8 in 2024-03-21T13:36:17.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:16 smithi144 ceph-mon[33633]: osdmap e431: 8 total, 8 up, 8 in 2024-03-21T13:36:18.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:17 smithi097 ceph-mon[39416]: pgmap v646: 292 pgs: 292 active+clean; 8.4 MiB data, 693 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:36:18.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:17 smithi097 ceph-mon[39416]: osdmap e432: 8 total, 8 up, 8 in 2024-03-21T13:36:18.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:17 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:36:18.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:17 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:36:18.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:17 smithi097 ceph-mon[30040]: pgmap v646: 292 pgs: 292 active+clean; 8.4 MiB data, 693 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:36:18.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:17 smithi097 ceph-mon[30040]: osdmap e432: 8 total, 8 up, 8 in 2024-03-21T13:36:18.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:17 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:36:18.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:17 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:36:18.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:17 smithi144 ceph-mon[33633]: pgmap v646: 292 pgs: 292 active+clean; 8.4 MiB data, 693 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:36:18.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:17 smithi144 ceph-mon[33633]: osdmap e432: 8 total, 8 up, 8 in 2024-03-21T13:36:18.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:17 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:36:18.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:17 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:36:19.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:18 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:36:19.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:18 smithi097 ceph-mon[39416]: osdmap e433: 8 total, 8 up, 8 in 2024-03-21T13:36:19.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:18 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:36:19.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:18 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:36:19.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:18 smithi097 ceph-mon[30040]: osdmap e433: 8 total, 8 up, 8 in 2024-03-21T13:36:19.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:18 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:36:19.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:18 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:36:19.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:18 smithi144 ceph-mon[33633]: osdmap e433: 8 total, 8 up, 8 in 2024-03-21T13:36:19.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:18 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:36:20.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:19 smithi097 ceph-mon[39416]: pgmap v649: 324 pgs: 324 active+clean; 8.4 MiB data, 693 MiB used, 715 GiB / 715 GiB avail; 13 KiB/s rd, 511 B/s wr, 22 op/s 2024-03-21T13:36:20.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:19 smithi097 ceph-mon[39416]: osdmap e434: 8 total, 8 up, 8 in 2024-03-21T13:36:20.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:19 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:36:20.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:19 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:36:20.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:19 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-77"}]: dispatch 2024-03-21T13:36:20.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:19 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-77"}]: dispatch 2024-03-21T13:36:20.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:19 smithi097 ceph-mon[30040]: pgmap v649: 324 pgs: 324 active+clean; 8.4 MiB data, 693 MiB used, 715 GiB / 715 GiB avail; 13 KiB/s rd, 511 B/s wr, 22 op/s 2024-03-21T13:36:20.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:19 smithi097 ceph-mon[30040]: osdmap e434: 8 total, 8 up, 8 in 2024-03-21T13:36:20.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:19 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:36:20.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:19 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:36:20.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:19 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-77"}]: dispatch 2024-03-21T13:36:20.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:19 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-77"}]: dispatch 2024-03-21T13:36:20.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:19 smithi144 ceph-mon[33633]: pgmap v649: 324 pgs: 324 active+clean; 8.4 MiB data, 693 MiB used, 715 GiB / 715 GiB avail; 13 KiB/s rd, 511 B/s wr, 22 op/s 2024-03-21T13:36:20.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:19 smithi144 ceph-mon[33633]: osdmap e434: 8 total, 8 up, 8 in 2024-03-21T13:36:20.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:19 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:36:20.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:19 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:36:20.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:19 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-77"}]: dispatch 2024-03-21T13:36:20.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:19 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-77"}]: dispatch 2024-03-21T13:36:21.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:20 smithi097 ceph-mon[39416]: osdmap e435: 8 total, 8 up, 8 in 2024-03-21T13:36:21.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:20 smithi097 ceph-mon[30040]: osdmap e435: 8 total, 8 up, 8 in 2024-03-21T13:36:21.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:20 smithi144 ceph-mon[33633]: osdmap e435: 8 total, 8 up, 8 in 2024-03-21T13:36:22.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:21 smithi144 ceph-mon[33633]: pgmap v652: 292 pgs: 292 active+clean; 8.4 MiB data, 693 MiB used, 715 GiB / 715 GiB avail; 13 KiB/s rd, 767 B/s wr, 22 op/s 2024-03-21T13:36:22.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:21 smithi144 ceph-mon[33633]: osdmap e436: 8 total, 8 up, 8 in 2024-03-21T13:36:22.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:21 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:36:22.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:21 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:36:22.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:36:21 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:36:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:36:22.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:21 smithi097 ceph-mon[39416]: pgmap v652: 292 pgs: 292 active+clean; 8.4 MiB data, 693 MiB used, 715 GiB / 715 GiB avail; 13 KiB/s rd, 767 B/s wr, 22 op/s 2024-03-21T13:36:22.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:21 smithi097 ceph-mon[39416]: osdmap e436: 8 total, 8 up, 8 in 2024-03-21T13:36:22.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:21 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:36:22.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:21 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:36:22.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:21 smithi097 ceph-mon[30040]: pgmap v652: 292 pgs: 292 active+clean; 8.4 MiB data, 693 MiB used, 715 GiB / 715 GiB avail; 13 KiB/s rd, 767 B/s wr, 22 op/s 2024-03-21T13:36:22.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:21 smithi097 ceph-mon[30040]: osdmap e436: 8 total, 8 up, 8 in 2024-03-21T13:36:22.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:21 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:36:22.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:21 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:36:23.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:22 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:36:23.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:22 smithi144 ceph-mon[33633]: osdmap e437: 8 total, 8 up, 8 in 2024-03-21T13:36:23.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:22 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:36:23.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:22 smithi144 ceph-mon[33633]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:36:23.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:22 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:36:23.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:22 smithi097 ceph-mon[39416]: osdmap e437: 8 total, 8 up, 8 in 2024-03-21T13:36:23.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:22 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:36:23.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:22 smithi097 ceph-mon[39416]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:36:23.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:22 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:36:23.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:22 smithi097 ceph-mon[30040]: osdmap e437: 8 total, 8 up, 8 in 2024-03-21T13:36:23.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:22 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:36:23.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:22 smithi097 ceph-mon[30040]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:36:24.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:23 smithi144 ceph-mon[33633]: pgmap v655: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 693 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:36:24.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:23 smithi144 ceph-mon[33633]: osdmap e438: 8 total, 8 up, 8 in 2024-03-21T13:36:24.121 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:36:23 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:36:24.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:23 smithi097 ceph-mon[39416]: pgmap v655: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 693 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:36:24.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:23 smithi097 ceph-mon[39416]: osdmap e438: 8 total, 8 up, 8 in 2024-03-21T13:36:24.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:23 smithi097 ceph-mon[30040]: pgmap v655: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 693 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:36:24.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:23 smithi097 ceph-mon[30040]: osdmap e438: 8 total, 8 up, 8 in 2024-03-21T13:36:25.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:24 smithi144 ceph-mon[33633]: osdmap e439: 8 total, 8 up, 8 in 2024-03-21T13:36:25.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:24 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:36:25.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:24 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "298.d"}]: dispatch 2024-03-21T13:36:25.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:24 smithi144 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "298.d"}]: dispatch 2024-03-21T13:36:25.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:24 smithi097 ceph-mon[39416]: osdmap e439: 8 total, 8 up, 8 in 2024-03-21T13:36:25.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:24 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:36:25.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:24 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "298.d"}]: dispatch 2024-03-21T13:36:25.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:24 smithi097 ceph-mon[39416]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "298.d"}]: dispatch 2024-03-21T13:36:25.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:24 smithi097 ceph-mon[30040]: osdmap e439: 8 total, 8 up, 8 in 2024-03-21T13:36:25.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:24 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:36:25.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:24 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "298.d"}]: dispatch 2024-03-21T13:36:25.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:24 smithi097 ceph-mon[30040]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "298.d"}]: dispatch 2024-03-21T13:36:26.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:25 smithi144 ceph-mon[33633]: pgmap v658: 324 pgs: 324 active+clean; 8.4 MiB data, 682 MiB used, 715 GiB / 715 GiB avail; 97 KiB/s rd, 1.2 KiB/s wr, 161 op/s 2024-03-21T13:36:26.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:25 smithi144 ceph-mon[33633]: 298.d deep-scrub starts 2024-03-21T13:36:26.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:25 smithi144 ceph-mon[33633]: 298.d deep-scrub ok 2024-03-21T13:36:26.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:25 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:36:26.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:25 smithi097 ceph-mon[39416]: pgmap v658: 324 pgs: 324 active+clean; 8.4 MiB data, 682 MiB used, 715 GiB / 715 GiB avail; 97 KiB/s rd, 1.2 KiB/s wr, 161 op/s 2024-03-21T13:36:26.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:25 smithi097 ceph-mon[39416]: 298.d deep-scrub starts 2024-03-21T13:36:26.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:25 smithi097 ceph-mon[39416]: 298.d deep-scrub ok 2024-03-21T13:36:26.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:25 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:36:26.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:25 smithi097 ceph-mon[30040]: pgmap v658: 324 pgs: 324 active+clean; 8.4 MiB data, 682 MiB used, 715 GiB / 715 GiB avail; 97 KiB/s rd, 1.2 KiB/s wr, 161 op/s 2024-03-21T13:36:26.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:25 smithi097 ceph-mon[30040]: 298.d deep-scrub starts 2024-03-21T13:36:26.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:25 smithi097 ceph-mon[30040]: 298.d deep-scrub ok 2024-03-21T13:36:26.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:25 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:36:28.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:27 smithi144 ceph-mon[33633]: pgmap v659: 324 pgs: 324 active+clean; 8.4 MiB data, 682 MiB used, 715 GiB / 715 GiB avail; 67 KiB/s rd, 889 B/s wr, 112 op/s 2024-03-21T13:36:28.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:27 smithi097 ceph-mon[39416]: pgmap v659: 324 pgs: 324 active+clean; 8.4 MiB data, 682 MiB used, 715 GiB / 715 GiB avail; 67 KiB/s rd, 889 B/s wr, 112 op/s 2024-03-21T13:36:28.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:27 smithi097 ceph-mon[30040]: pgmap v659: 324 pgs: 324 active+clean; 8.4 MiB data, 682 MiB used, 715 GiB / 715 GiB avail; 67 KiB/s rd, 889 B/s wr, 112 op/s 2024-03-21T13:36:30.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:29 smithi144 ceph-mon[33633]: pgmap v660: 324 pgs: 324 active+clean; 8.4 MiB data, 682 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s rd, 910 B/s wr, 97 op/s 2024-03-21T13:36:30.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:29 smithi097 ceph-mon[39416]: pgmap v660: 324 pgs: 324 active+clean; 8.4 MiB data, 682 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s rd, 910 B/s wr, 97 op/s 2024-03-21T13:36:30.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:29 smithi097 ceph-mon[30040]: pgmap v660: 324 pgs: 324 active+clean; 8.4 MiB data, 682 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s rd, 910 B/s wr, 97 op/s 2024-03-21T13:36:32.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:31 smithi144 ceph-mon[33633]: pgmap v661: 324 pgs: 324 active+clean; 8.4 MiB data, 682 MiB used, 715 GiB / 715 GiB avail; 49 KiB/s rd, 767 B/s wr, 82 op/s 2024-03-21T13:36:32.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:31 smithi097 ceph-mon[39416]: pgmap v661: 324 pgs: 324 active+clean; 8.4 MiB data, 682 MiB used, 715 GiB / 715 GiB avail; 49 KiB/s rd, 767 B/s wr, 82 op/s 2024-03-21T13:36:32.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:31 smithi097 ceph-mon[30040]: pgmap v661: 324 pgs: 324 active+clean; 8.4 MiB data, 682 MiB used, 715 GiB / 715 GiB avail; 49 KiB/s rd, 767 B/s wr, 82 op/s 2024-03-21T13:36:32.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:36:31 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:36:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:36:34.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:33 smithi144 ceph-mon[33633]: pgmap v662: 324 pgs: 324 active+clean; 8.4 MiB data, 682 MiB used, 715 GiB / 715 GiB avail; 40 KiB/s rd, 631 B/s wr, 67 op/s 2024-03-21T13:36:34.121 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:36:33 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:36:34.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:33 smithi097 ceph-mon[39416]: pgmap v662: 324 pgs: 324 active+clean; 8.4 MiB data, 682 MiB used, 715 GiB / 715 GiB avail; 40 KiB/s rd, 631 B/s wr, 67 op/s 2024-03-21T13:36:34.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:33 smithi097 ceph-mon[30040]: pgmap v662: 324 pgs: 324 active+clean; 8.4 MiB data, 682 MiB used, 715 GiB / 715 GiB avail; 40 KiB/s rd, 631 B/s wr, 67 op/s 2024-03-21T13:36:35.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:34 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:36:35.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:34 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:36:35.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:34 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:36:36.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:35 smithi144 ceph-mon[33633]: pgmap v663: 324 pgs: 324 active+clean; 8.4 MiB data, 682 MiB used, 715 GiB / 715 GiB avail; 954 B/s rd, 95 B/s wr, 1 op/s 2024-03-21T13:36:36.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:35 smithi097 ceph-mon[30040]: pgmap v663: 324 pgs: 324 active+clean; 8.4 MiB data, 682 MiB used, 715 GiB / 715 GiB avail; 954 B/s rd, 95 B/s wr, 1 op/s 2024-03-21T13:36:36.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:35 smithi097 ceph-mon[39416]: pgmap v663: 324 pgs: 324 active+clean; 8.4 MiB data, 682 MiB used, 715 GiB / 715 GiB avail; 954 B/s rd, 95 B/s wr, 1 op/s 2024-03-21T13:36:38.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:37 smithi144 ceph-mon[33633]: pgmap v664: 324 pgs: 324 active+clean; 8.4 MiB data, 682 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 85 B/s wr, 1 op/s 2024-03-21T13:36:38.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:37 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:36:38.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:37 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:36:38.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:37 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:36:38.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:37 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:36:38.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:37 smithi097 ceph-mon[39416]: pgmap v664: 324 pgs: 324 active+clean; 8.4 MiB data, 682 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 85 B/s wr, 1 op/s 2024-03-21T13:36:38.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:37 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:36:38.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:37 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:36:38.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:37 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:36:38.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:37 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:36:38.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:37 smithi097 ceph-mon[30040]: pgmap v664: 324 pgs: 324 active+clean; 8.4 MiB data, 682 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 85 B/s wr, 1 op/s 2024-03-21T13:36:38.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:37 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:36:38.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:37 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:36:38.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:37 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:36:38.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:37 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:36:40.104 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:39 smithi097 ceph-mon[39416]: pgmap v665: 324 pgs: 324 active+clean; 8.4 MiB data, 682 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 1 op/s 2024-03-21T13:36:40.105 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:39 smithi097 ceph-mon[30040]: pgmap v665: 324 pgs: 324 active+clean; 8.4 MiB data, 682 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 1 op/s 2024-03-21T13:36:40.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:39 smithi144 ceph-mon[33633]: pgmap v665: 324 pgs: 324 active+clean; 8.4 MiB data, 682 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 1 op/s 2024-03-21T13:36:41.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:40 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:36:41.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:40 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:36:41.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:40 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:36:42.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:41 smithi144 ceph-mon[33633]: pgmap v666: 324 pgs: 324 active+clean; 8.4 MiB data, 682 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:36:42.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:41 smithi097 ceph-mon[39416]: pgmap v666: 324 pgs: 324 active+clean; 8.4 MiB data, 682 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:36:42.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:41 smithi097 ceph-mon[30040]: pgmap v666: 324 pgs: 324 active+clean; 8.4 MiB data, 682 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:36:42.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:36:41 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:36:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:36:43.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:43 smithi144 ceph-mon[33633]: pgmap v667: 324 pgs: 324 active+clean; 8.4 MiB data, 682 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:36:43.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:43 smithi097 ceph-mon[30040]: pgmap v667: 324 pgs: 324 active+clean; 8.4 MiB data, 682 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:36:43.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:43 smithi097 ceph-mon[39416]: pgmap v667: 324 pgs: 324 active+clean; 8.4 MiB data, 682 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:36:44.121 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 13:36:43 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T13:36:43.855+0000 7fc759e70700 -1 snap_mapper.add_oid found existing snaps mapped on 298:b032b8ff:test-rados-api-smithi097-79906-80::foo:2, removing 2024-03-21T13:36:44.122 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:36:43 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:36:44.303 INFO:journalctl@ceph.osd.1.smithi097.stdout:Mar 21 13:36:43 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-1[49765]: 2024-03-21T13:36:43.855+0000 7fc95dd32700 -1 snap_mapper.add_oid found existing snaps mapped on 298:b032b8ff:test-rados-api-smithi097-79906-80::foo:2, removing 2024-03-21T13:36:44.304 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 21 13:36:43 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-3[62397]: 2024-03-21T13:36:43.855+0000 7f7dcfd55700 -1 snap_mapper.add_oid found existing snaps mapped on 298:b032b8ff:test-rados-api-smithi097-79906-80::foo:2, removing 2024-03-21T13:36:44.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:44 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:36:44.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:44 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:36:44.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:44 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:36:44.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:44 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-79"}]: dispatch 2024-03-21T13:36:44.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:44 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-79"}]: dispatch 2024-03-21T13:36:44.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:44 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:36:44.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:44 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:36:44.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:44 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:36:44.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:44 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-79"}]: dispatch 2024-03-21T13:36:44.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:44 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-79"}]: dispatch 2024-03-21T13:36:44.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:44 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:36:44.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:44 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:36:44.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:44 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:36:44.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:44 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-79"}]: dispatch 2024-03-21T13:36:44.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:44 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-79"}]: dispatch 2024-03-21T13:36:45.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:45 smithi144 ceph-mon[33633]: osdmap e440: 8 total, 8 up, 8 in 2024-03-21T13:36:45.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:45 smithi144 ceph-mon[33633]: pgmap v669: 292 pgs: 292 active+clean; 8.4 MiB data, 682 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:36:45.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:45 smithi097 ceph-mon[30040]: osdmap e440: 8 total, 8 up, 8 in 2024-03-21T13:36:45.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:45 smithi097 ceph-mon[30040]: pgmap v669: 292 pgs: 292 active+clean; 8.4 MiB data, 682 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:36:45.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:45 smithi097 ceph-mon[39416]: osdmap e440: 8 total, 8 up, 8 in 2024-03-21T13:36:45.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:45 smithi097 ceph-mon[39416]: pgmap v669: 292 pgs: 292 active+clean; 8.4 MiB data, 682 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:36:46.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:46 smithi144 ceph-mon[33633]: osdmap e441: 8 total, 8 up, 8 in 2024-03-21T13:36:46.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:46 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:36:46.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:46 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:36:46.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:46 smithi097 ceph-mon[30040]: osdmap e441: 8 total, 8 up, 8 in 2024-03-21T13:36:46.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:46 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:36:46.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:46 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:36:46.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:46 smithi097 ceph-mon[39416]: osdmap e441: 8 total, 8 up, 8 in 2024-03-21T13:36:46.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:46 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:36:46.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:46 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:36:47.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:47 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:36:47.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:47 smithi144 ceph-mon[33633]: osdmap e442: 8 total, 8 up, 8 in 2024-03-21T13:36:47.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:47 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:36:47.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:47 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:36:47.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:47 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:36:47.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:47 smithi144 ceph-mon[33633]: pgmap v672: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 682 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:36:47.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:47 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:36:47.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:47 smithi097 ceph-mon[30040]: osdmap e442: 8 total, 8 up, 8 in 2024-03-21T13:36:47.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:47 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:36:47.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:47 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:36:47.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:47 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:36:47.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:47 smithi097 ceph-mon[30040]: pgmap v672: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 682 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:36:47.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:47 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:36:47.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:47 smithi097 ceph-mon[39416]: osdmap e442: 8 total, 8 up, 8 in 2024-03-21T13:36:47.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:47 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:36:47.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:47 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:36:47.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:47 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:36:47.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:47 smithi097 ceph-mon[39416]: pgmap v672: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 682 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:36:48.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:48 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:36:48.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:48 smithi144 ceph-mon[33633]: osdmap e443: 8 total, 8 up, 8 in 2024-03-21T13:36:48.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:48 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_tier","val": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:36:48.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:48 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_tier","val": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:36:48.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:48 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:36:48.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:48 smithi097 ceph-mon[30040]: osdmap e443: 8 total, 8 up, 8 in 2024-03-21T13:36:48.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:48 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_tier","val": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:36:48.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:48 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_tier","val": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:36:48.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:48 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:36:48.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:48 smithi097 ceph-mon[39416]: osdmap e443: 8 total, 8 up, 8 in 2024-03-21T13:36:48.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:48 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_tier","val": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:36:48.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:48 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_tier","val": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:36:49.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:49 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_tier","val": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:36:49.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:49 smithi097 ceph-mon[39416]: osdmap e444: 8 total, 8 up, 8 in 2024-03-21T13:36:49.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:49 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:36:49.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:49 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:36:49.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:49 smithi097 ceph-mon[39416]: pgmap v675: 324 pgs: 324 active+clean; 8.4 MiB data, 682 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-21T13:36:49.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:49 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_tier","val": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:36:49.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:49 smithi097 ceph-mon[30040]: osdmap e444: 8 total, 8 up, 8 in 2024-03-21T13:36:49.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:49 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:36:49.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:49 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:36:49.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:49 smithi097 ceph-mon[30040]: pgmap v675: 324 pgs: 324 active+clean; 8.4 MiB data, 682 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-21T13:36:49.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:49 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_tier","val": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:36:49.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:49 smithi144 ceph-mon[33633]: osdmap e444: 8 total, 8 up, 8 in 2024-03-21T13:36:49.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:49 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:36:49.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:49 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:36:49.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:49 smithi144 ceph-mon[33633]: pgmap v675: 324 pgs: 324 active+clean; 8.4 MiB data, 682 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-21T13:36:50.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:50 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-21T13:36:50.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:50 smithi097 ceph-mon[39416]: osdmap e445: 8 total, 8 up, 8 in 2024-03-21T13:36:50.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:50 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:36:50.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:50 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:36:50.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:50 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-21T13:36:50.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:50 smithi097 ceph-mon[30040]: osdmap e445: 8 total, 8 up, 8 in 2024-03-21T13:36:50.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:50 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:36:50.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:50 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:36:50.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:50 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-21T13:36:50.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:50 smithi144 ceph-mon[33633]: osdmap e445: 8 total, 8 up, 8 in 2024-03-21T13:36:50.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:50 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:36:50.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:50 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:36:51.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:51 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-21T13:36:51.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:51 smithi097 ceph-mon[39416]: osdmap e446: 8 total, 8 up, 8 in 2024-03-21T13:36:51.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:51 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-03-21T13:36:51.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:51 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-03-21T13:36:51.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:51 smithi097 ceph-mon[39416]: pgmap v678: 324 pgs: 324 active+clean; 8.4 MiB data, 682 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-21T13:36:51.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:51 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-21T13:36:51.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:51 smithi097 ceph-mon[30040]: osdmap e446: 8 total, 8 up, 8 in 2024-03-21T13:36:51.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:51 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-03-21T13:36:51.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:51 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-03-21T13:36:51.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:51 smithi097 ceph-mon[30040]: pgmap v678: 324 pgs: 324 active+clean; 8.4 MiB data, 682 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-21T13:36:51.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:51 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-21T13:36:51.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:51 smithi144 ceph-mon[33633]: osdmap e446: 8 total, 8 up, 8 in 2024-03-21T13:36:51.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:51 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-03-21T13:36:51.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:51 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-03-21T13:36:51.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:51 smithi144 ceph-mon[33633]: pgmap v678: 324 pgs: 324 active+clean; 8.4 MiB data, 682 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-21T13:36:51.936 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cf0d0 2024-03-21T13:36:51.936 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5ccd50 2024-03-21T13:36:51.936 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cce80 2024-03-21T13:36:51.936 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5ccfb0 2024-03-21T13:36:51.936 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cf200 2024-03-21T13:36:51.936 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cec20 2024-03-21T13:36:51.936 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5ced50 2024-03-21T13:36:51.936 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cee60 2024-03-21T13:36:51.936 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cef90 2024-03-21T13:36:51.936 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cf7e0 2024-03-21T13:36:51.936 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cf330 2024-03-21T13:36:51.937 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cf480 2024-03-21T13:36:51.937 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cf5d0 2024-03-21T13:36:51.937 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cf930 2024-03-21T13:36:51.937 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cfe00 2024-03-21T13:36:51.937 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cff30 2024-03-21T13:36:51.937 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5ce330 2024-03-21T13:36:51.937 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5ce460 2024-03-21T13:36:51.937 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5ce590 2024-03-21T13:36:51.937 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d0060 2024-03-21T13:36:51.937 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d0530 2024-03-21T13:36:51.937 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d0680 2024-03-21T13:36:51.937 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cfa60 2024-03-21T13:36:51.937 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cfb90 2024-03-21T13:36:51.938 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5cfcc0 2024-03-21T13:36:51.938 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d07b0 2024-03-21T13:36:51.938 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d0ca0 2024-03-21T13:36:51.938 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d0df0 2024-03-21T13:36:51.938 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d0190 2024-03-21T13:36:51.938 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d02c0 2024-03-21T13:36:51.938 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d03f0 2024-03-21T13:36:51.938 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d0f20 2024-03-21T13:36:51.938 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d1410 2024-03-21T13:36:51.938 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d1540 2024-03-21T13:36:51.938 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d08e0 2024-03-21T13:36:51.938 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d0a30 2024-03-21T13:36:51.938 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d0b40 2024-03-21T13:36:51.939 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d1650 2024-03-21T13:36:51.939 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d1b60 2024-03-21T13:36:51.939 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d1cb0 2024-03-21T13:36:51.939 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d1070 2024-03-21T13:36:51.939 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d11a0 2024-03-21T13:36:51.939 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d12d0 2024-03-21T13:36:51.939 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d1de0 2024-03-21T13:36:51.939 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d22d0 2024-03-21T13:36:51.939 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d2420 2024-03-21T13:36:51.939 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d17a0 2024-03-21T13:36:51.939 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d18f0 2024-03-21T13:36:51.939 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d1a20 2024-03-21T13:36:51.939 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d2550 2024-03-21T13:36:51.939 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d2a20 2024-03-21T13:36:51.939 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d2b50 2024-03-21T13:36:51.940 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d2660 2024-03-21T13:36:51.940 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d27b0 2024-03-21T13:36:51.940 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d2900 2024-03-21T13:36:51.940 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d2c60 2024-03-21T13:36:51.940 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d3130 2024-03-21T13:36:51.940 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d3260 2024-03-21T13:36:51.940 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d1ef0 2024-03-21T13:36:51.940 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d2020 2024-03-21T13:36:51.940 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d2170 2024-03-21T13:36:51.940 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d3370 2024-03-21T13:36:51.940 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d34e0 2024-03-21T13:36:51.940 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d3ae0 2024-03-21T13:36:51.940 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d3630 2024-03-21T13:36:51.940 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d3760 2024-03-21T13:36:51.941 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d3890 2024-03-21T13:36:51.941 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d39c0 2024-03-21T13:36:51.941 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d40c0 2024-03-21T13:36:51.941 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d3c10 2024-03-21T13:36:51.941 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d3d60 2024-03-21T13:36:51.941 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d3e90 2024-03-21T13:36:51.941 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d41f0 2024-03-21T13:36:51.941 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d46c0 2024-03-21T13:36:51.941 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d47f0 2024-03-21T13:36:51.941 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d2d70 2024-03-21T13:36:51.941 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d2ea0 2024-03-21T13:36:51.941 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d2fd0 2024-03-21T13:36:51.941 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d4920 2024-03-21T13:36:51.941 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d4df0 2024-03-21T13:36:51.941 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d4f40 2024-03-21T13:36:51.942 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d4300 2024-03-21T13:36:52.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:36:51 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:36:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:36:52.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:52 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-03-21T13:36:52.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:52 smithi097 ceph-mon[30040]: osdmap e447: 8 total, 8 up, 8 in 2024-03-21T13:36:52.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:52 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-03-21T13:36:52.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:52 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-03-21T13:36:52.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:52 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-03-21T13:36:52.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:52 smithi097 ceph-mon[39416]: osdmap e447: 8 total, 8 up, 8 in 2024-03-21T13:36:52.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:52 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-03-21T13:36:52.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:52 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-03-21T13:36:52.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:52 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-03-21T13:36:52.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:52 smithi144 ceph-mon[33633]: osdmap e447: 8 total, 8 up, 8 in 2024-03-21T13:36:52.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:52 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-03-21T13:36:52.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:52 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-03-21T13:36:53.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:53 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-03-21T13:36:53.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:53 smithi097 ceph-mon[39416]: osdmap e448: 8 total, 8 up, 8 in 2024-03-21T13:36:53.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:53 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:36:53.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:53 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:36:53.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:53 smithi097 ceph-mon[39416]: pgmap v681: 324 pgs: 324 active+clean; 8.4 MiB data, 682 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:36:53.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:53 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-03-21T13:36:53.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:53 smithi097 ceph-mon[30040]: osdmap e448: 8 total, 8 up, 8 in 2024-03-21T13:36:53.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:53 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:36:53.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:53 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:36:53.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:53 smithi097 ceph-mon[30040]: pgmap v681: 324 pgs: 324 active+clean; 8.4 MiB data, 682 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:36:53.854 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:53 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-03-21T13:36:53.855 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:53 smithi144 ceph-mon[33633]: osdmap e448: 8 total, 8 up, 8 in 2024-03-21T13:36:53.855 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:53 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:36:53.855 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:53 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:36:53.855 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:53 smithi144 ceph-mon[33633]: pgmap v681: 324 pgs: 324 active+clean; 8.4 MiB data, 682 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:36:54.121 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:36:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:36:54.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:54 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-21T13:36:54.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:54 smithi097 ceph-mon[30040]: osdmap e449: 8 total, 8 up, 8 in 2024-03-21T13:36:54.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:54 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:36:54.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:54 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:36:54.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:54 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-81"}]: dispatch 2024-03-21T13:36:54.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:54 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-81"}]: dispatch 2024-03-21T13:36:54.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:54 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:36:54.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:54 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-21T13:36:54.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:54 smithi097 ceph-mon[39416]: osdmap e449: 8 total, 8 up, 8 in 2024-03-21T13:36:54.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:54 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:36:54.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:54 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:36:54.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:54 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-81"}]: dispatch 2024-03-21T13:36:54.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:54 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-81"}]: dispatch 2024-03-21T13:36:54.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:54 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:36:54.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:54 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-21T13:36:54.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:54 smithi144 ceph-mon[33633]: osdmap e449: 8 total, 8 up, 8 in 2024-03-21T13:36:54.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:54 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:36:54.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:54 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:36:54.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:54 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-81"}]: dispatch 2024-03-21T13:36:54.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:54 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-81"}]: dispatch 2024-03-21T13:36:54.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:54 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:36:55.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:55 smithi097 ceph-mon[39416]: osdmap e450: 8 total, 8 up, 8 in 2024-03-21T13:36:55.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:55 smithi097 ceph-mon[39416]: pgmap v684: 292 pgs: 292 active+clean; 8.4 MiB data, 683 MiB used, 715 GiB / 715 GiB avail; 7.5 KiB/s rd, 9.0 KiB/s wr, 28 op/s 2024-03-21T13:36:55.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:55 smithi097 ceph-mon[30040]: osdmap e450: 8 total, 8 up, 8 in 2024-03-21T13:36:55.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:55 smithi097 ceph-mon[30040]: pgmap v684: 292 pgs: 292 active+clean; 8.4 MiB data, 683 MiB used, 715 GiB / 715 GiB avail; 7.5 KiB/s rd, 9.0 KiB/s wr, 28 op/s 2024-03-21T13:36:55.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:55 smithi144 ceph-mon[33633]: osdmap e450: 8 total, 8 up, 8 in 2024-03-21T13:36:55.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:55 smithi144 ceph-mon[33633]: pgmap v684: 292 pgs: 292 active+clean; 8.4 MiB data, 683 MiB used, 715 GiB / 715 GiB avail; 7.5 KiB/s rd, 9.0 KiB/s wr, 28 op/s 2024-03-21T13:36:56.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:56 smithi097 ceph-mon[39416]: osdmap e451: 8 total, 8 up, 8 in 2024-03-21T13:36:56.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:56 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:36:56.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:56 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:36:56.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:56 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:36:56.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:56 smithi097 ceph-mon[30040]: osdmap e451: 8 total, 8 up, 8 in 2024-03-21T13:36:56.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:56 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:36:56.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:56 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:36:56.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:56 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:36:56.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:56 smithi144 ceph-mon[33633]: osdmap e451: 8 total, 8 up, 8 in 2024-03-21T13:36:56.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:56 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:36:56.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:56 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:36:56.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:56 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:36:57.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:57 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:36:57.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:57 smithi097 ceph-mon[39416]: osdmap e452: 8 total, 8 up, 8 in 2024-03-21T13:36:57.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:57 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:36:57.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:57 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:36:57.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:57 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:36:57.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:57 smithi097 ceph-mon[39416]: pgmap v687: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 683 MiB used, 715 GiB / 715 GiB avail; 7.5 KiB/s rd, 9.0 KiB/s wr, 28 op/s 2024-03-21T13:36:57.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:57 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:36:57.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:57 smithi097 ceph-mon[30040]: osdmap e452: 8 total, 8 up, 8 in 2024-03-21T13:36:57.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:57 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:36:57.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:57 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:36:57.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:57 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:36:57.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:57 smithi097 ceph-mon[30040]: pgmap v687: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 683 MiB used, 715 GiB / 715 GiB avail; 7.5 KiB/s rd, 9.0 KiB/s wr, 28 op/s 2024-03-21T13:36:57.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:57 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:36:57.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:57 smithi144 ceph-mon[33633]: osdmap e452: 8 total, 8 up, 8 in 2024-03-21T13:36:57.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:57 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:36:57.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:57 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:36:57.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:57 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:36:57.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:57 smithi144 ceph-mon[33633]: pgmap v687: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 683 MiB used, 715 GiB / 715 GiB avail; 7.5 KiB/s rd, 9.0 KiB/s wr, 28 op/s 2024-03-21T13:36:58.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:58 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:36:58.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:58 smithi097 ceph-mon[39416]: osdmap e453: 8 total, 8 up, 8 in 2024-03-21T13:36:58.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:58 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-83","var": "dedup_tier","val": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:36:58.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:58 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-83","var": "dedup_tier","val": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:36:58.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:58 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:36:58.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:58 smithi097 ceph-mon[30040]: osdmap e453: 8 total, 8 up, 8 in 2024-03-21T13:36:58.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:58 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-83","var": "dedup_tier","val": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:36:58.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:58 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-83","var": "dedup_tier","val": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:36:58.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:58 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:36:58.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:58 smithi144 ceph-mon[33633]: osdmap e453: 8 total, 8 up, 8 in 2024-03-21T13:36:58.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:58 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-83","var": "dedup_tier","val": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:36:58.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:58 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-83","var": "dedup_tier","val": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:36:59.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:59 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-83","var": "dedup_tier","val": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:36:59.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:59 smithi097 ceph-mon[39416]: osdmap e454: 8 total, 8 up, 8 in 2024-03-21T13:36:59.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:59 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:36:59.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:59 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:36:59.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:36:59 smithi097 ceph-mon[39416]: pgmap v690: 324 pgs: 324 active+clean; 8.4 MiB data, 684 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-03-21T13:36:59.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:59 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-83","var": "dedup_tier","val": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:36:59.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:59 smithi097 ceph-mon[30040]: osdmap e454: 8 total, 8 up, 8 in 2024-03-21T13:36:59.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:59 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:36:59.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:59 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:36:59.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:36:59 smithi097 ceph-mon[30040]: pgmap v690: 324 pgs: 324 active+clean; 8.4 MiB data, 684 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-03-21T13:36:59.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:59 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-83","var": "dedup_tier","val": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:36:59.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:59 smithi144 ceph-mon[33633]: osdmap e454: 8 total, 8 up, 8 in 2024-03-21T13:36:59.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:59 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:36:59.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:59 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:36:59.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:36:59 smithi144 ceph-mon[33633]: pgmap v690: 324 pgs: 324 active+clean; 8.4 MiB data, 684 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-03-21T13:37:00.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:00 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-21T13:37:00.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:00 smithi097 ceph-mon[39416]: osdmap e455: 8 total, 8 up, 8 in 2024-03-21T13:37:00.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:00 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:37:00.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:00 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:37:00.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:00 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-21T13:37:00.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:00 smithi097 ceph-mon[30040]: osdmap e455: 8 total, 8 up, 8 in 2024-03-21T13:37:00.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:00 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:37:00.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:00 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:37:00.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:00 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-21T13:37:00.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:00 smithi144 ceph-mon[33633]: osdmap e455: 8 total, 8 up, 8 in 2024-03-21T13:37:00.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:00 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:37:00.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:00 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:37:01.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:01 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-21T13:37:01.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:01 smithi097 ceph-mon[39416]: osdmap e456: 8 total, 8 up, 8 in 2024-03-21T13:37:01.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:01 smithi097 ceph-mon[39416]: pgmap v693: 324 pgs: 324 active+clean; 8.4 MiB data, 684 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-03-21T13:37:01.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:01 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-21T13:37:01.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:01 smithi097 ceph-mon[30040]: osdmap e456: 8 total, 8 up, 8 in 2024-03-21T13:37:01.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:01 smithi097 ceph-mon[30040]: pgmap v693: 324 pgs: 324 active+clean; 8.4 MiB data, 684 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-03-21T13:37:01.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:01 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-21T13:37:01.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:01 smithi144 ceph-mon[33633]: osdmap e456: 8 total, 8 up, 8 in 2024-03-21T13:37:01.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:01 smithi144 ceph-mon[33633]: pgmap v693: 324 pgs: 324 active+clean; 8.4 MiB data, 684 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-03-21T13:37:02.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:37:01 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:37:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:37:02.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:02 smithi097 ceph-mon[39416]: osdmap e457: 8 total, 8 up, 8 in 2024-03-21T13:37:02.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:02 smithi097 ceph-mon[30040]: osdmap e457: 8 total, 8 up, 8 in 2024-03-21T13:37:02.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:02 smithi144 ceph-mon[33633]: osdmap e457: 8 total, 8 up, 8 in 2024-03-21T13:37:03.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:03 smithi097 ceph-mon[39416]: osdmap e458: 8 total, 8 up, 8 in 2024-03-21T13:37:03.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:03 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:37:03.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:03 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:37:03.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:03 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-83"}]: dispatch 2024-03-21T13:37:03.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:03 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-83"}]: dispatch 2024-03-21T13:37:03.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:03 smithi097 ceph-mon[39416]: pgmap v696: 324 pgs: 324 active+clean; 8.4 MiB data, 684 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:37:03.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:03 smithi097 ceph-mon[30040]: osdmap e458: 8 total, 8 up, 8 in 2024-03-21T13:37:03.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:03 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:37:03.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:03 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:37:03.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:03 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-83"}]: dispatch 2024-03-21T13:37:03.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:03 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-83"}]: dispatch 2024-03-21T13:37:03.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:03 smithi097 ceph-mon[30040]: pgmap v696: 324 pgs: 324 active+clean; 8.4 MiB data, 684 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:37:03.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:03 smithi144 ceph-mon[33633]: osdmap e458: 8 total, 8 up, 8 in 2024-03-21T13:37:03.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:03 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:37:03.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:03 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:37:03.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:03 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-83"}]: dispatch 2024-03-21T13:37:03.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:03 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-83"}]: dispatch 2024-03-21T13:37:03.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:03 smithi144 ceph-mon[33633]: pgmap v696: 324 pgs: 324 active+clean; 8.4 MiB data, 684 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:37:04.121 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:37:03 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:37:04.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:04 smithi097 ceph-mon[39416]: osdmap e459: 8 total, 8 up, 8 in 2024-03-21T13:37:04.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:04 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:37:04.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:04 smithi097 ceph-mon[30040]: osdmap e459: 8 total, 8 up, 8 in 2024-03-21T13:37:04.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:04 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:37:04.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:04 smithi144 ceph-mon[33633]: osdmap e459: 8 total, 8 up, 8 in 2024-03-21T13:37:04.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:04 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:37:05.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:05 smithi097 ceph-mon[39416]: osdmap e460: 8 total, 8 up, 8 in 2024-03-21T13:37:05.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:05 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:37:05.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:05 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:37:05.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:05 smithi097 ceph-mon[39416]: pgmap v699: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 684 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-03-21T13:37:05.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:05 smithi097 ceph-mon[30040]: osdmap e460: 8 total, 8 up, 8 in 2024-03-21T13:37:05.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:05 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:37:05.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:05 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:37:05.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:05 smithi097 ceph-mon[30040]: pgmap v699: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 684 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-03-21T13:37:05.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:05 smithi144 ceph-mon[33633]: osdmap e460: 8 total, 8 up, 8 in 2024-03-21T13:37:05.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:05 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:37:05.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:05 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:37:05.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:05 smithi144 ceph-mon[33633]: pgmap v699: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 684 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-03-21T13:37:06.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:06 smithi097 ceph-mon[39416]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:37:06.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:06 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:37:06.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:06 smithi097 ceph-mon[39416]: osdmap e461: 8 total, 8 up, 8 in 2024-03-21T13:37:06.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:06 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:37:06.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:06 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:37:06.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:06 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:37:06.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:06 smithi097 ceph-mon[30040]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:37:06.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:06 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:37:06.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:06 smithi097 ceph-mon[30040]: osdmap e461: 8 total, 8 up, 8 in 2024-03-21T13:37:06.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:06 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:37:06.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:06 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:37:06.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:06 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:37:06.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:06 smithi144 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:37:06.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:06 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:37:06.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:06 smithi144 ceph-mon[33633]: osdmap e461: 8 total, 8 up, 8 in 2024-03-21T13:37:06.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:06 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:37:06.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:06 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:37:06.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:06 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:37:07.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:07 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:37:07.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:07 smithi097 ceph-mon[39416]: osdmap e462: 8 total, 8 up, 8 in 2024-03-21T13:37:07.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:07 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-85","var": "dedup_tier","val": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:37:07.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:07 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-85","var": "dedup_tier","val": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:37:07.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:07 smithi097 ceph-mon[39416]: pgmap v702: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 684 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-03-21T13:37:07.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:07 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:37:07.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:07 smithi097 ceph-mon[30040]: osdmap e462: 8 total, 8 up, 8 in 2024-03-21T13:37:07.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:07 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-85","var": "dedup_tier","val": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:37:07.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:07 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-85","var": "dedup_tier","val": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:37:07.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:07 smithi097 ceph-mon[30040]: pgmap v702: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 684 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-03-21T13:37:07.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:07 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:37:07.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:07 smithi144 ceph-mon[33633]: osdmap e462: 8 total, 8 up, 8 in 2024-03-21T13:37:07.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:07 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-85","var": "dedup_tier","val": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:37:07.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:07 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-85","var": "dedup_tier","val": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:37:07.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:07 smithi144 ceph-mon[33633]: pgmap v702: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 684 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-03-21T13:37:08.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:08 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-85","var": "dedup_tier","val": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:37:08.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:08 smithi097 ceph-mon[39416]: osdmap e463: 8 total, 8 up, 8 in 2024-03-21T13:37:08.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:08 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:37:08.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:08 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:37:08.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:08 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-85","var": "dedup_tier","val": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:37:08.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:08 smithi097 ceph-mon[30040]: osdmap e463: 8 total, 8 up, 8 in 2024-03-21T13:37:08.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:08 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:37:08.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:08 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:37:08.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:08 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-85","var": "dedup_tier","val": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:37:08.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:08 smithi144 ceph-mon[33633]: osdmap e463: 8 total, 8 up, 8 in 2024-03-21T13:37:08.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:08 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:37:08.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:08 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:37:09.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:09 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-21T13:37:09.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:09 smithi097 ceph-mon[39416]: osdmap e464: 8 total, 8 up, 8 in 2024-03-21T13:37:09.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:09 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:37:09.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:09 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:37:09.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:09 smithi097 ceph-mon[39416]: pgmap v705: 324 pgs: 324 active+clean; 8.4 MiB data, 685 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:37:09.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:09 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-21T13:37:09.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:09 smithi097 ceph-mon[30040]: osdmap e464: 8 total, 8 up, 8 in 2024-03-21T13:37:09.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:09 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:37:09.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:09 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:37:09.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:09 smithi097 ceph-mon[30040]: pgmap v705: 324 pgs: 324 active+clean; 8.4 MiB data, 685 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:37:09.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:09 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-21T13:37:09.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:09 smithi144 ceph-mon[33633]: osdmap e464: 8 total, 8 up, 8 in 2024-03-21T13:37:09.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:09 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:37:09.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:09 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:37:09.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:09 smithi144 ceph-mon[33633]: pgmap v705: 324 pgs: 324 active+clean; 8.4 MiB data, 685 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:37:10.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:10 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-21T13:37:10.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:10 smithi097 ceph-mon[30040]: osdmap e465: 8 total, 8 up, 8 in 2024-03-21T13:37:10.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:10 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:37:10.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:10 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-21T13:37:10.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:10 smithi097 ceph-mon[39416]: osdmap e465: 8 total, 8 up, 8 in 2024-03-21T13:37:10.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:10 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:37:10.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:10 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-21T13:37:10.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:10 smithi144 ceph-mon[33633]: osdmap e465: 8 total, 8 up, 8 in 2024-03-21T13:37:10.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:10 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:37:11.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:11 smithi097 ceph-mon[39416]: osdmap e466: 8 total, 8 up, 8 in 2024-03-21T13:37:11.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:11 smithi097 ceph-mon[39416]: pgmap v708: 324 pgs: 324 active+clean; 8.4 MiB data, 685 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:37:11.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:11 smithi097 ceph-mon[30040]: osdmap e466: 8 total, 8 up, 8 in 2024-03-21T13:37:11.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:11 smithi097 ceph-mon[30040]: pgmap v708: 324 pgs: 324 active+clean; 8.4 MiB data, 685 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:37:11.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:11 smithi144 ceph-mon[33633]: osdmap e466: 8 total, 8 up, 8 in 2024-03-21T13:37:11.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:11 smithi144 ceph-mon[33633]: pgmap v708: 324 pgs: 324 active+clean; 8.4 MiB data, 685 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:37:12.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:37:11 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:37:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:37:12.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:12 smithi097 ceph-mon[39416]: osdmap e467: 8 total, 8 up, 8 in 2024-03-21T13:37:12.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:12 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:37:12.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:12 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:37:12.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:12 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-85"}]: dispatch 2024-03-21T13:37:12.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:12 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-85"}]: dispatch 2024-03-21T13:37:12.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:12 smithi097 ceph-mon[39416]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:37:12.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:12 smithi097 ceph-mon[30040]: osdmap e467: 8 total, 8 up, 8 in 2024-03-21T13:37:12.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:12 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:37:12.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:12 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:37:12.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:12 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-85"}]: dispatch 2024-03-21T13:37:12.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:12 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-85"}]: dispatch 2024-03-21T13:37:12.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:12 smithi097 ceph-mon[30040]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:37:12.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:12 smithi144 ceph-mon[33633]: osdmap e467: 8 total, 8 up, 8 in 2024-03-21T13:37:12.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:12 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:37:12.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:12 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:37:12.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:12 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-85"}]: dispatch 2024-03-21T13:37:12.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:12 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-85"}]: dispatch 2024-03-21T13:37:12.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:12 smithi144 ceph-mon[33633]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:37:13.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:13 smithi144 ceph-mon[33633]: osdmap e468: 8 total, 8 up, 8 in 2024-03-21T13:37:13.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:13 smithi144 ceph-mon[33633]: pgmap v711: 292 pgs: 292 active+clean; 8.4 MiB data, 685 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:37:14.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:13 smithi097 ceph-mon[39416]: osdmap e468: 8 total, 8 up, 8 in 2024-03-21T13:37:14.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:13 smithi097 ceph-mon[39416]: pgmap v711: 292 pgs: 292 active+clean; 8.4 MiB data, 685 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:37:14.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:13 smithi097 ceph-mon[30040]: osdmap e468: 8 total, 8 up, 8 in 2024-03-21T13:37:14.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:13 smithi097 ceph-mon[30040]: pgmap v711: 292 pgs: 292 active+clean; 8.4 MiB data, 685 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:37:14.371 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:37:13 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:37:14.580 INFO:tasks.workunit.client.0.smithi097.stdout: OK ] LibRadosTwoPoolsPP.ProxyRead (18102 ms) 2024-03-21T13:37:14.580 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.CachePin 2024-03-21T13:37:14.580 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.CachePin (22559 ms) 2024-03-21T13:37:14.580 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.SetRedirectRead 2024-03-21T13:37:14.580 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.SetRedirectRead (3017 ms) 2024-03-21T13:37:14.580 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestPromoteRead 2024-03-21T13:37:14.580 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: require_osd_release = squid 2024-03-21T13:37:14.580 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestPromoteRead (3028 ms) 2024-03-21T13:37:14.581 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRefRead 2024-03-21T13:37:14.581 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRefRead (3016 ms) 2024-03-21T13:37:14.581 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestUnset 2024-03-21T13:37:14.581 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: require_osd_release = squid 2024-03-21T13:37:14.581 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestUnset (3024 ms) 2024-03-21T13:37:14.581 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestDedupRefRead 2024-03-21T13:37:14.581 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: require_osd_release = squid 2024-03-21T13:37:14.581 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestDedupRefRead (4029 ms) 2024-03-21T13:37:14.581 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount 2024-03-21T13:37:14.581 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: require_osd_release = squid 2024-03-21T13:37:14.581 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount (38162 ms) 2024-03-21T13:37:14.581 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 2024-03-21T13:37:14.581 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: require_osd_release = squid 2024-03-21T13:37:14.581 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 (16731 ms) 2024-03-21T13:37:14.581 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestTestSnapCreate 2024-03-21T13:37:14.581 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: require_osd_release = squid 2024-03-21T13:37:14.582 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestTestSnapCreate (4033 ms) 2024-03-21T13:37:14.582 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote 2024-03-21T13:37:14.582 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: require_osd_release = squid 2024-03-21T13:37:14.582 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote (3022 ms) 2024-03-21T13:37:14.582 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification 2024-03-21T13:37:14.582 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: require_osd_release = squid 2024-03-21T13:37:14.582 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification (24285 ms) 2024-03-21T13:37:14.582 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapIncCount 2024-03-21T13:37:14.582 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: require_osd_release = squid 2024-03-21T13:37:14.582 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapIncCount (15063 ms) 2024-03-21T13:37:14.582 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvict 2024-03-21T13:37:14.582 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: require_osd_release = squid 2024-03-21T13:37:14.582 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvict (5041 ms) 2024-03-21T13:37:14.582 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictPromote 2024-03-21T13:37:14.582 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: require_osd_release = squid 2024-03-21T13:37:14.583 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictPromote (4036 ms) 2024-03-21T13:37:14.583 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch 2024-03-21T13:37:14.583 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: require_osd_release = squid 2024-03-21T13:37:14.583 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: waiting for scrubs... 2024-03-21T13:37:14.583 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: done waiting 2024-03-21T13:37:14.583 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch (24538 ms) 2024-03-21T13:37:14.583 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.DedupFlushRead 2024-03-21T13:37:14.583 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: require_osd_release = squid 2024-03-21T13:37:14.583 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.DedupFlushRead (10089 ms) 2024-03-21T13:37:14.583 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushSnap 2024-03-21T13:37:14.583 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: require_osd_release = squid 2024-03-21T13:37:14.583 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushSnap (9078 ms) 2024-03-21T13:37:14.583 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushDupCount 2024-03-21T13:37:14.583 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: require_osd_release = squid 2024-03-21T13:37:14.583 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushDupCount (9066 ms) 2024-03-21T13:37:14.583 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringFlush 2024-03-21T13:37:14.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:14 smithi144 ceph-mon[33633]: osdmap e469: 8 total, 8 up, 8 in 2024-03-21T13:37:14.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:14 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:37:14.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:14 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:37:14.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:14 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:37:15.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:14 smithi097 ceph-mon[39416]: osdmap e469: 8 total, 8 up, 8 in 2024-03-21T13:37:15.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:14 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:37:15.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:14 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:37:15.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:14 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:37:15.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:14 smithi097 ceph-mon[30040]: osdmap e469: 8 total, 8 up, 8 in 2024-03-21T13:37:15.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:14 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:37:15.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:14 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:37:15.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:14 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:37:15.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:15 smithi144 ceph-mon[33633]: pgmap v713: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 686 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-03-21T13:37:15.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:15 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:37:15.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:15 smithi144 ceph-mon[33633]: osdmap e470: 8 total, 8 up, 8 in 2024-03-21T13:37:15.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:15 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:37:16.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:15 smithi097 ceph-mon[30040]: pgmap v713: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 686 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-03-21T13:37:16.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:15 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:37:16.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:15 smithi097 ceph-mon[30040]: osdmap e470: 8 total, 8 up, 8 in 2024-03-21T13:37:16.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:15 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:37:16.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:15 smithi097 ceph-mon[39416]: pgmap v713: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 686 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-03-21T13:37:16.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:15 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:37:16.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:15 smithi097 ceph-mon[39416]: osdmap e470: 8 total, 8 up, 8 in 2024-03-21T13:37:16.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:15 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:37:16.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:16 smithi144 ceph-mon[33633]: osdmap e471: 8 total, 8 up, 8 in 2024-03-21T13:37:16.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:16 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:37:16.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:16 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:37:17.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:16 smithi097 ceph-mon[30040]: osdmap e471: 8 total, 8 up, 8 in 2024-03-21T13:37:17.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:16 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:37:17.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:16 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:37:17.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:16 smithi097 ceph-mon[39416]: osdmap e471: 8 total, 8 up, 8 in 2024-03-21T13:37:17.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:16 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:37:17.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:16 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:37:17.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:17 smithi144 ceph-mon[33633]: pgmap v716: 356 pgs: 64 unknown, 292 active+clean; 8.4 MiB data, 686 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-03-21T13:37:17.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:17 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:37:17.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:17 smithi144 ceph-mon[33633]: osdmap e472: 8 total, 8 up, 8 in 2024-03-21T13:37:17.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:17 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-87","var": "dedup_tier","val": "test-rados-api-smithi097-79906-89-test-flush"}]: dispatch 2024-03-21T13:37:17.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:17 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-87","var": "dedup_tier","val": "test-rados-api-smithi097-79906-89-test-flush"}]: dispatch 2024-03-21T13:37:17.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:17 smithi144 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:37:18.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:17 smithi097 ceph-mon[39416]: pgmap v716: 356 pgs: 64 unknown, 292 active+clean; 8.4 MiB data, 686 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-03-21T13:37:18.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:17 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:37:18.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:17 smithi097 ceph-mon[39416]: osdmap e472: 8 total, 8 up, 8 in 2024-03-21T13:37:18.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:17 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-87","var": "dedup_tier","val": "test-rados-api-smithi097-79906-89-test-flush"}]: dispatch 2024-03-21T13:37:18.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:17 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-87","var": "dedup_tier","val": "test-rados-api-smithi097-79906-89-test-flush"}]: dispatch 2024-03-21T13:37:18.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:17 smithi097 ceph-mon[39416]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:37:18.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:17 smithi097 ceph-mon[30040]: pgmap v716: 356 pgs: 64 unknown, 292 active+clean; 8.4 MiB data, 686 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-03-21T13:37:18.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:17 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:37:18.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:17 smithi097 ceph-mon[30040]: osdmap e472: 8 total, 8 up, 8 in 2024-03-21T13:37:18.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:17 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-87","var": "dedup_tier","val": "test-rados-api-smithi097-79906-89-test-flush"}]: dispatch 2024-03-21T13:37:18.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:17 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-87","var": "dedup_tier","val": "test-rados-api-smithi097-79906-89-test-flush"}]: dispatch 2024-03-21T13:37:18.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:17 smithi097 ceph-mon[30040]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:37:18.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:18 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-87","var": "dedup_tier","val": "test-rados-api-smithi097-79906-89-test-flush"}]': finished 2024-03-21T13:37:18.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:18 smithi144 ceph-mon[33633]: osdmap e473: 8 total, 8 up, 8 in 2024-03-21T13:37:18.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:18 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:37:18.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:18 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:37:19.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:18 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-87","var": "dedup_tier","val": "test-rados-api-smithi097-79906-89-test-flush"}]': finished 2024-03-21T13:37:19.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:18 smithi097 ceph-mon[39416]: osdmap e473: 8 total, 8 up, 8 in 2024-03-21T13:37:19.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:18 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:37:19.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:18 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:37:19.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:18 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-87","var": "dedup_tier","val": "test-rados-api-smithi097-79906-89-test-flush"}]': finished 2024-03-21T13:37:19.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:18 smithi097 ceph-mon[30040]: osdmap e473: 8 total, 8 up, 8 in 2024-03-21T13:37:19.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:18 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:37:19.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:18 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:37:19.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:19 smithi144 ceph-mon[33633]: pgmap v719: 356 pgs: 356 active+clean; 8.4 MiB data, 686 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:37:19.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:19 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-21T13:37:19.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:19 smithi144 ceph-mon[33633]: osdmap e474: 8 total, 8 up, 8 in 2024-03-21T13:37:19.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:19 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:37:19.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:19 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:37:20.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:19 smithi097 ceph-mon[39416]: pgmap v719: 356 pgs: 356 active+clean; 8.4 MiB data, 686 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:37:20.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:19 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-21T13:37:20.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:19 smithi097 ceph-mon[39416]: osdmap e474: 8 total, 8 up, 8 in 2024-03-21T13:37:20.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:19 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:37:20.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:19 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:37:20.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:19 smithi097 ceph-mon[30040]: pgmap v719: 356 pgs: 356 active+clean; 8.4 MiB data, 686 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:37:20.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:19 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-21T13:37:20.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:19 smithi097 ceph-mon[30040]: osdmap e474: 8 total, 8 up, 8 in 2024-03-21T13:37:20.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:19 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:37:20.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:19 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:37:20.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:20 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-21T13:37:20.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:20 smithi144 ceph-mon[33633]: osdmap e475: 8 total, 8 up, 8 in 2024-03-21T13:37:21.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:20 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-21T13:37:21.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:20 smithi097 ceph-mon[39416]: osdmap e475: 8 total, 8 up, 8 in 2024-03-21T13:37:21.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:20 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-21T13:37:21.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:20 smithi097 ceph-mon[30040]: osdmap e475: 8 total, 8 up, 8 in 2024-03-21T13:37:21.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:21 smithi144 ceph-mon[33633]: pgmap v722: 356 pgs: 356 active+clean; 8.4 MiB data, 686 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:37:21.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:21 smithi144 ceph-mon[33633]: osdmap e476: 8 total, 8 up, 8 in 2024-03-21T13:37:21.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:21 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:37:21.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:21 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:37:21.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:21 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-87"}]: dispatch 2024-03-21T13:37:21.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:21 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-87"}]: dispatch 2024-03-21T13:37:21.949 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:21 smithi097 ceph-mon[39416]: pgmap v722: 356 pgs: 356 active+clean; 8.4 MiB data, 686 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:37:21.949 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:21 smithi097 ceph-mon[39416]: osdmap e476: 8 total, 8 up, 8 in 2024-03-21T13:37:21.949 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:21 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:37:21.949 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:21 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:37:21.949 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:21 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-87"}]: dispatch 2024-03-21T13:37:21.949 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:21 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-87"}]: dispatch 2024-03-21T13:37:21.949 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:21 smithi097 ceph-mon[30040]: pgmap v722: 356 pgs: 356 active+clean; 8.4 MiB data, 686 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:37:21.950 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:21 smithi097 ceph-mon[30040]: osdmap e476: 8 total, 8 up, 8 in 2024-03-21T13:37:21.950 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:21 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:37:21.950 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:21 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:37:21.950 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:21 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-87"}]: dispatch 2024-03-21T13:37:21.950 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:21 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-87"}]: dispatch 2024-03-21T13:37:22.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:37:21 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:37:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:37:23.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:22 smithi097 ceph-mon[39416]: osdmap e477: 8 total, 8 up, 8 in 2024-03-21T13:37:23.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:22 smithi097 ceph-mon[30040]: osdmap e477: 8 total, 8 up, 8 in 2024-03-21T13:37:23.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:22 smithi144 ceph-mon[33633]: osdmap e477: 8 total, 8 up, 8 in 2024-03-21T13:37:23.884 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:23 smithi144 ceph-mon[33633]: pgmap v725: 292 pgs: 292 active+clean; 8.4 MiB data, 686 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:37:23.884 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:23 smithi144 ceph-mon[33633]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:37:23.884 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:23 smithi144 ceph-mon[33633]: osdmap e478: 8 total, 8 up, 8 in 2024-03-21T13:37:23.884 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:23 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:37:23.884 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:23 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:37:24.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:23 smithi097 ceph-mon[39416]: pgmap v725: 292 pgs: 292 active+clean; 8.4 MiB data, 686 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:37:24.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:23 smithi097 ceph-mon[39416]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:37:24.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:23 smithi097 ceph-mon[39416]: osdmap e478: 8 total, 8 up, 8 in 2024-03-21T13:37:24.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:23 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:37:24.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:23 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:37:24.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:23 smithi097 ceph-mon[30040]: pgmap v725: 292 pgs: 292 active+clean; 8.4 MiB data, 686 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:37:24.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:23 smithi097 ceph-mon[30040]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:37:24.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:23 smithi097 ceph-mon[30040]: osdmap e478: 8 total, 8 up, 8 in 2024-03-21T13:37:24.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:23 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:37:24.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:23 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:37:24.371 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:37:23 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:37:25.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:24 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:37:25.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:24 smithi097 ceph-mon[39416]: osdmap e479: 8 total, 8 up, 8 in 2024-03-21T13:37:25.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:24 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:37:25.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:24 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:37:25.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:24 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:37:25.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:24 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:37:25.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:24 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:37:25.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:24 smithi097 ceph-mon[30040]: osdmap e479: 8 total, 8 up, 8 in 2024-03-21T13:37:25.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:24 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:37:25.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:24 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:37:25.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:24 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:37:25.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:24 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:37:25.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:24 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:37:25.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:24 smithi144 ceph-mon[33633]: osdmap e479: 8 total, 8 up, 8 in 2024-03-21T13:37:25.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:24 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:37:25.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:24 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:37:25.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:24 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:37:25.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:24 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:37:26.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:25 smithi097 ceph-mon[39416]: pgmap v728: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 695 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-21T13:37:26.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:25 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:37:26.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:25 smithi097 ceph-mon[39416]: osdmap e480: 8 total, 8 up, 8 in 2024-03-21T13:37:26.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:25 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:37:26.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:25 smithi097 ceph-mon[30040]: pgmap v728: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 695 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-21T13:37:26.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:25 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:37:26.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:25 smithi097 ceph-mon[30040]: osdmap e480: 8 total, 8 up, 8 in 2024-03-21T13:37:26.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:25 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:37:26.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:25 smithi144 ceph-mon[33633]: pgmap v728: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 695 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-21T13:37:26.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:25 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:37:26.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:25 smithi144 ceph-mon[33633]: osdmap e480: 8 total, 8 up, 8 in 2024-03-21T13:37:26.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:25 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:37:27.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:26 smithi097 ceph-mon[30040]: osdmap e481: 8 total, 8 up, 8 in 2024-03-21T13:37:27.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:26 smithi097 ceph-mon[39416]: osdmap e481: 8 total, 8 up, 8 in 2024-03-21T13:37:27.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:26 smithi144 ceph-mon[33633]: osdmap e481: 8 total, 8 up, 8 in 2024-03-21T13:37:28.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:27 smithi097 ceph-mon[39416]: pgmap v731: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 695 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-21T13:37:28.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:27 smithi097 ceph-mon[39416]: osdmap e482: 8 total, 8 up, 8 in 2024-03-21T13:37:28.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:27 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:37:28.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:27 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:37:28.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:27 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-90"}]: dispatch 2024-03-21T13:37:28.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:27 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-90"}]: dispatch 2024-03-21T13:37:28.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:27 smithi097 ceph-mon[30040]: pgmap v731: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 695 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-21T13:37:28.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:27 smithi097 ceph-mon[30040]: osdmap e482: 8 total, 8 up, 8 in 2024-03-21T13:37:28.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:27 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:37:28.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:27 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:37:28.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:27 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-90"}]: dispatch 2024-03-21T13:37:28.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:27 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-90"}]: dispatch 2024-03-21T13:37:28.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:27 smithi144 ceph-mon[33633]: pgmap v731: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 695 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-21T13:37:28.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:27 smithi144 ceph-mon[33633]: osdmap e482: 8 total, 8 up, 8 in 2024-03-21T13:37:28.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:27 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:37:28.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:27 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:37:28.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:27 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-90"}]: dispatch 2024-03-21T13:37:28.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:27 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-90"}]: dispatch 2024-03-21T13:37:29.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:28 smithi097 ceph-mon[39416]: osdmap e483: 8 total, 8 up, 8 in 2024-03-21T13:37:29.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:28 smithi097 ceph-mon[30040]: osdmap e483: 8 total, 8 up, 8 in 2024-03-21T13:37:29.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:28 smithi144 ceph-mon[33633]: osdmap e483: 8 total, 8 up, 8 in 2024-03-21T13:37:30.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:29 smithi097 ceph-mon[39416]: pgmap v734: 292 pgs: 292 active+clean; 8.4 MiB data, 700 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-21T13:37:30.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:29 smithi097 ceph-mon[39416]: osdmap e484: 8 total, 8 up, 8 in 2024-03-21T13:37:30.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:29 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:37:30.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:29 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:37:30.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:29 smithi097 ceph-mon[30040]: pgmap v734: 292 pgs: 292 active+clean; 8.4 MiB data, 700 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-21T13:37:30.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:29 smithi097 ceph-mon[30040]: osdmap e484: 8 total, 8 up, 8 in 2024-03-21T13:37:30.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:29 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:37:30.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:29 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:37:30.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:29 smithi144 ceph-mon[33633]: pgmap v734: 292 pgs: 292 active+clean; 8.4 MiB data, 700 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-21T13:37:30.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:29 smithi144 ceph-mon[33633]: osdmap e484: 8 total, 8 up, 8 in 2024-03-21T13:37:30.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:29 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:37:30.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:29 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:37:31.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:30 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:37:31.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:30 smithi097 ceph-mon[39416]: osdmap e485: 8 total, 8 up, 8 in 2024-03-21T13:37:31.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:30 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:37:31.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:30 smithi097 ceph-mon[39416]: osdmap e486: 8 total, 8 up, 8 in 2024-03-21T13:37:31.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:30 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:37:31.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:30 smithi097 ceph-mon[30040]: osdmap e485: 8 total, 8 up, 8 in 2024-03-21T13:37:31.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:30 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:37:31.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:30 smithi097 ceph-mon[30040]: osdmap e486: 8 total, 8 up, 8 in 2024-03-21T13:37:31.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:30 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:37:31.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:30 smithi144 ceph-mon[33633]: osdmap e485: 8 total, 8 up, 8 in 2024-03-21T13:37:31.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:30 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:37:31.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:30 smithi144 ceph-mon[33633]: osdmap e486: 8 total, 8 up, 8 in 2024-03-21T13:37:32.053 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:37:31 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:37:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:37:32.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:31 smithi097 ceph-mon[39416]: pgmap v737: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 700 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-21T13:37:32.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:31 smithi097 ceph-mon[39416]: osdmap e487: 8 total, 8 up, 8 in 2024-03-21T13:37:32.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:31 smithi097 ceph-mon[30040]: pgmap v737: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 700 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-21T13:37:32.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:31 smithi097 ceph-mon[30040]: osdmap e487: 8 total, 8 up, 8 in 2024-03-21T13:37:32.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:31 smithi144 ceph-mon[33633]: pgmap v737: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 700 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-21T13:37:32.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:31 smithi144 ceph-mon[33633]: osdmap e487: 8 total, 8 up, 8 in 2024-03-21T13:37:33.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:32 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:37:33.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:32 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:37:33.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:32 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-92"}]: dispatch 2024-03-21T13:37:33.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:32 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-92"}]: dispatch 2024-03-21T13:37:33.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:32 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:37:33.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:32 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:37:33.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:32 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-92"}]: dispatch 2024-03-21T13:37:33.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:32 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-92"}]: dispatch 2024-03-21T13:37:33.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:32 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:37:33.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:32 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:37:33.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:32 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-92"}]: dispatch 2024-03-21T13:37:33.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:32 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-92"}]: dispatch 2024-03-21T13:37:34.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:33 smithi097 ceph-mon[39416]: pgmap v740: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 700 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:37:34.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:33 smithi097 ceph-mon[39416]: osdmap e488: 8 total, 8 up, 8 in 2024-03-21T13:37:34.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:33 smithi097 ceph-mon[30040]: pgmap v740: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 700 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:37:34.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:33 smithi097 ceph-mon[30040]: osdmap e488: 8 total, 8 up, 8 in 2024-03-21T13:37:34.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:33 smithi144 ceph-mon[33633]: pgmap v740: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 700 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:37:34.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:33 smithi144 ceph-mon[33633]: osdmap e488: 8 total, 8 up, 8 in 2024-03-21T13:37:34.121 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:37:33 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:37:35.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:34 smithi097 ceph-mon[30040]: osdmap e489: 8 total, 8 up, 8 in 2024-03-21T13:37:35.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:34 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:37:35.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:34 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:37:35.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:34 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:37:35.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:34 smithi097 ceph-mon[39416]: osdmap e489: 8 total, 8 up, 8 in 2024-03-21T13:37:35.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:34 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:37:35.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:34 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:37:35.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:34 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:37:35.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:34 smithi144 ceph-mon[33633]: osdmap e489: 8 total, 8 up, 8 in 2024-03-21T13:37:35.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:34 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:37:35.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:34 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:37:35.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:34 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:37:36.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:35 smithi097 ceph-mon[39416]: pgmap v743: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 705 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-21T13:37:36.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:35 smithi097 ceph-mon[39416]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:37:36.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:35 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:37:36.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:35 smithi097 ceph-mon[39416]: osdmap e490: 8 total, 8 up, 8 in 2024-03-21T13:37:36.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:35 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:37:36.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:35 smithi097 ceph-mon[30040]: pgmap v743: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 705 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-21T13:37:36.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:35 smithi097 ceph-mon[30040]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:37:36.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:35 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:37:36.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:35 smithi097 ceph-mon[30040]: osdmap e490: 8 total, 8 up, 8 in 2024-03-21T13:37:36.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:35 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:37:36.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:35 smithi144 ceph-mon[33633]: pgmap v743: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 705 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-21T13:37:36.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:35 smithi144 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:37:36.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:35 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:37:36.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:35 smithi144 ceph-mon[33633]: osdmap e490: 8 total, 8 up, 8 in 2024-03-21T13:37:36.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:35 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:37:37.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:36 smithi097 ceph-mon[39416]: osdmap e491: 8 total, 8 up, 8 in 2024-03-21T13:37:37.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:36 smithi097 ceph-mon[30040]: osdmap e491: 8 total, 8 up, 8 in 2024-03-21T13:37:37.120 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:36 smithi144 ceph-mon[33633]: osdmap e491: 8 total, 8 up, 8 in 2024-03-21T13:37:38.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:37 smithi097 ceph-mon[39416]: pgmap v746: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 705 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-21T13:37:38.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:37 smithi097 ceph-mon[39416]: osdmap e492: 8 total, 8 up, 8 in 2024-03-21T13:37:38.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:37 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:37:38.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:37 smithi097 ceph-mon[30040]: pgmap v746: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 705 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-21T13:37:38.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:37 smithi097 ceph-mon[30040]: osdmap e492: 8 total, 8 up, 8 in 2024-03-21T13:37:38.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:37 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:37:38.120 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:37 smithi144 ceph-mon[33633]: pgmap v746: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 705 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-21T13:37:38.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:37 smithi144 ceph-mon[33633]: osdmap e492: 8 total, 8 up, 8 in 2024-03-21T13:37:38.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:37 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:37:39.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:38 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:37:39.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:38 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:37:39.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:38 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:37:39.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:38 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:37:39.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:38 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:37:39.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:38 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:37:39.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:38 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:37:39.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:38 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:37:39.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:38 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:37:39.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:38 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:37:39.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:38 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:37:39.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:38 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:37:39.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:38 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:37:39.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:38 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:37:39.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:38 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:37:39.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:38 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:37:39.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:38 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:37:39.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:38 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:37:39.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:38 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:37:39.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:38 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:37:39.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:38 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:37:39.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:38 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:37:39.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:38 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:37:39.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:38 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:37:39.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:38 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:37:39.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:38 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:37:39.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:38 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:37:39.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:38 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:37:39.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:38 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:37:39.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:38 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:37:39.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:38 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:37:39.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:38 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:37:39.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:38 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:37:39.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:38 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:37:39.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:38 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:37:39.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:38 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:37:39.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:38 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:37:39.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:38 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:37:39.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:38 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:37:39.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:38 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:37:39.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:38 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:37:39.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:38 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:37:39.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:38 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:37:39.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:38 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:37:39.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:38 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:37:39.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:38 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:37:39.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:38 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:37:39.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:38 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:37:39.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:38 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:37:39.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:38 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:37:39.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:38 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:37:39.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:38 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:37:39.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:38 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:37:39.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:38 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:37:39.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:38 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:37:39.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:38 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:37:39.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:38 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:37:39.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:38 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:37:39.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:38 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:37:39.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:38 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:37:39.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:38 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:37:39.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:38 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:37:39.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:38 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:37:40.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:39 smithi097 ceph-mon[30040]: Adjusting osd_memory_target on smithi144 to 3248M 2024-03-21T13:37:40.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:39 smithi097 ceph-mon[30040]: Adjusting osd_memory_target on smithi097 to 3248M 2024-03-21T13:37:40.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:39 smithi097 ceph-mon[30040]: pgmap v748: 324 pgs: 324 active+clean; 8.4 MiB data, 681 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-03-21T13:37:40.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:39 smithi097 ceph-mon[39416]: Adjusting osd_memory_target on smithi144 to 3248M 2024-03-21T13:37:40.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:39 smithi097 ceph-mon[39416]: Adjusting osd_memory_target on smithi097 to 3248M 2024-03-21T13:37:40.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:39 smithi097 ceph-mon[39416]: pgmap v748: 324 pgs: 324 active+clean; 8.4 MiB data, 681 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-03-21T13:37:40.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:39 smithi144 ceph-mon[33633]: Adjusting osd_memory_target on smithi144 to 3248M 2024-03-21T13:37:40.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:39 smithi144 ceph-mon[33633]: Adjusting osd_memory_target on smithi097 to 3248M 2024-03-21T13:37:40.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:39 smithi144 ceph-mon[33633]: pgmap v748: 324 pgs: 324 active+clean; 8.4 MiB data, 681 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-03-21T13:37:41.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:40 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:37:41.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:40 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:37:41.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:40 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:37:42.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:41 smithi097 ceph-mon[30040]: pgmap v749: 324 pgs: 324 active+clean; 8.4 MiB data, 681 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-03-21T13:37:42.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:41 smithi097 ceph-mon[39416]: pgmap v749: 324 pgs: 324 active+clean; 8.4 MiB data, 681 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-03-21T13:37:42.054 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:37:41 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:37:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:37:42.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:41 smithi144 ceph-mon[33633]: pgmap v749: 324 pgs: 324 active+clean; 8.4 MiB data, 681 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-03-21T13:37:43.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:42 smithi097 ceph-mon[30040]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:37:43.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:42 smithi097 ceph-mon[39416]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:37:43.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:42 smithi144 ceph-mon[33633]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:37:44.022 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:43 smithi144 ceph-mon[33633]: pgmap v750: 324 pgs: 324 active+clean; 8.4 MiB data, 681 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-03-21T13:37:44.023 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:37:43 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:37:44.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:43 smithi097 ceph-mon[39416]: pgmap v750: 324 pgs: 324 active+clean; 8.4 MiB data, 681 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-03-21T13:37:44.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:43 smithi097 ceph-mon[30040]: pgmap v750: 324 pgs: 324 active+clean; 8.4 MiB data, 681 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-03-21T13:37:45.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:44 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:37:45.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:44 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:37:45.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:44 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:37:46.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:45 smithi097 ceph-mon[30040]: pgmap v751: 324 pgs: 324 active+clean; 8.4 MiB data, 681 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 927 B/s wr, 4 op/s 2024-03-21T13:37:46.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:45 smithi097 ceph-mon[39416]: pgmap v751: 324 pgs: 324 active+clean; 8.4 MiB data, 681 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 927 B/s wr, 4 op/s 2024-03-21T13:37:46.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:45 smithi144 ceph-mon[33633]: pgmap v751: 324 pgs: 324 active+clean; 8.4 MiB data, 681 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 927 B/s wr, 4 op/s 2024-03-21T13:37:48.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:47 smithi097 ceph-mon[39416]: pgmap v752: 324 pgs: 324 active+clean; 8.4 MiB data, 681 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 818 B/s wr, 3 op/s 2024-03-21T13:37:48.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:47 smithi097 ceph-mon[30040]: pgmap v752: 324 pgs: 324 active+clean; 8.4 MiB data, 681 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 818 B/s wr, 3 op/s 2024-03-21T13:37:48.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:47 smithi144 ceph-mon[33633]: pgmap v752: 324 pgs: 324 active+clean; 8.4 MiB data, 681 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 818 B/s wr, 3 op/s 2024-03-21T13:37:49.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:48 smithi097 ceph-mon[30040]: osdmap e493: 8 total, 8 up, 8 in 2024-03-21T13:37:49.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:48 smithi097 ceph-mon[39416]: osdmap e493: 8 total, 8 up, 8 in 2024-03-21T13:37:49.120 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:48 smithi144 ceph-mon[33633]: osdmap e493: 8 total, 8 up, 8 in 2024-03-21T13:37:50.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:49 smithi097 ceph-mon[30040]: pgmap v754: 324 pgs: 324 active+clean; 8.4 MiB data, 669 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:37:50.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:49 smithi097 ceph-mon[39416]: pgmap v754: 324 pgs: 324 active+clean; 8.4 MiB data, 669 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:37:50.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:49 smithi144 ceph-mon[33633]: pgmap v754: 324 pgs: 324 active+clean; 8.4 MiB data, 669 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:37:52.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:51 smithi097 ceph-mon[39416]: pgmap v755: 324 pgs: 324 active+clean; 8.4 MiB data, 669 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:37:52.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:51 smithi097 ceph-mon[30040]: pgmap v755: 324 pgs: 324 active+clean; 8.4 MiB data, 669 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:37:52.053 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:37:51 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:37:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:37:52.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:51 smithi144 ceph-mon[33633]: pgmap v755: 324 pgs: 324 active+clean; 8.4 MiB data, 669 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:37:53.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:53 smithi144 ceph-mon[33633]: osdmap e494: 8 total, 8 up, 8 in 2024-03-21T13:37:53.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:53 smithi144 ceph-mon[33633]: pgmap v757: 324 pgs: 324 active+clean; 8.4 MiB data, 669 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:37:53.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:53 smithi097 ceph-mon[39416]: osdmap e494: 8 total, 8 up, 8 in 2024-03-21T13:37:53.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:53 smithi097 ceph-mon[39416]: pgmap v757: 324 pgs: 324 active+clean; 8.4 MiB data, 669 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:37:53.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:53 smithi097 ceph-mon[30040]: osdmap e494: 8 total, 8 up, 8 in 2024-03-21T13:37:53.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:53 smithi097 ceph-mon[30040]: pgmap v757: 324 pgs: 324 active+clean; 8.4 MiB data, 669 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:37:54.342 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:37:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:37:54.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:54 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "306.9", "id": [7, 2]}]: dispatch 2024-03-21T13:37:54.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:54 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "306.9", "id": [7, 2]}]: dispatch 2024-03-21T13:37:54.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:54 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:37:54.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:54 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "306.9", "id": [7, 2]}]: dispatch 2024-03-21T13:37:54.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:54 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "306.9", "id": [7, 2]}]: dispatch 2024-03-21T13:37:54.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:54 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:37:54.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:54 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "306.9", "id": [7, 2]}]: dispatch 2024-03-21T13:37:54.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:54 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "306.9", "id": [7, 2]}]: dispatch 2024-03-21T13:37:54.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:54 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:37:55.121 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:37:54 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=cleanup t=2024-03-21T13:37:54.658479892Z level=info msg="Completed cleanup jobs" duration=2.31721ms 2024-03-21T13:37:55.121 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:37:54 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=sqlstore.transactions t=2024-03-21T13:37:54.666631589Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-03-21T13:37:55.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:55 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "306.9", "id": [7, 2]}]': finished 2024-03-21T13:37:55.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:55 smithi144 ceph-mon[33633]: osdmap e495: 8 total, 8 up, 8 in 2024-03-21T13:37:55.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:55 smithi144 ceph-mon[33633]: pgmap v759: 324 pgs: 324 active+clean; 8.4 MiB data, 669 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-21T13:37:55.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:55 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "306.9", "id": [7, 2]}]': finished 2024-03-21T13:37:55.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:55 smithi097 ceph-mon[39416]: osdmap e495: 8 total, 8 up, 8 in 2024-03-21T13:37:55.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:55 smithi097 ceph-mon[39416]: pgmap v759: 324 pgs: 324 active+clean; 8.4 MiB data, 669 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-21T13:37:55.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:55 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "306.9", "id": [7, 2]}]': finished 2024-03-21T13:37:55.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:55 smithi097 ceph-mon[30040]: osdmap e495: 8 total, 8 up, 8 in 2024-03-21T13:37:55.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:55 smithi097 ceph-mon[30040]: pgmap v759: 324 pgs: 324 active+clean; 8.4 MiB data, 669 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-21T13:37:56.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:56 smithi144 ceph-mon[33633]: osdmap e496: 8 total, 8 up, 8 in 2024-03-21T13:37:56.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:56 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:37:56.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:56 smithi097 ceph-mon[39416]: osdmap e496: 8 total, 8 up, 8 in 2024-03-21T13:37:56.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:56 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:37:56.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:56 smithi097 ceph-mon[30040]: osdmap e496: 8 total, 8 up, 8 in 2024-03-21T13:37:56.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:56 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:37:57.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:57 smithi144 ceph-mon[33633]: pgmap v761: 324 pgs: 324 active+clean; 8.4 MiB data, 669 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:37:57.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:57 smithi097 ceph-mon[39416]: pgmap v761: 324 pgs: 324 active+clean; 8.4 MiB data, 669 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:37:57.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:57 smithi097 ceph-mon[30040]: pgmap v761: 324 pgs: 324 active+clean; 8.4 MiB data, 669 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:37:58.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:58 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:37:58.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:58 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:37:58.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:58 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-94"}]: dispatch 2024-03-21T13:37:58.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:58 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-94"}]: dispatch 2024-03-21T13:37:58.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:58 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:37:58.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:58 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:37:58.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:58 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-94"}]: dispatch 2024-03-21T13:37:58.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:58 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-94"}]: dispatch 2024-03-21T13:37:58.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:58 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:37:58.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:58 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:37:58.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:58 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-94"}]: dispatch 2024-03-21T13:37:58.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:58 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-94"}]: dispatch 2024-03-21T13:37:59.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:59 smithi144 ceph-mon[33633]: osdmap e497: 8 total, 8 up, 8 in 2024-03-21T13:37:59.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:37:59 smithi144 ceph-mon[33633]: pgmap v763: 292 pgs: 292 active+clean; 8.4 MiB data, 669 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:37:59.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:59 smithi097 ceph-mon[39416]: osdmap e497: 8 total, 8 up, 8 in 2024-03-21T13:37:59.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:37:59 smithi097 ceph-mon[39416]: pgmap v763: 292 pgs: 292 active+clean; 8.4 MiB data, 669 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:37:59.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:59 smithi097 ceph-mon[30040]: osdmap e497: 8 total, 8 up, 8 in 2024-03-21T13:37:59.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:37:59 smithi097 ceph-mon[30040]: pgmap v763: 292 pgs: 292 active+clean; 8.4 MiB data, 669 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:38:00.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:00 smithi097 ceph-mon[30040]: osdmap e498: 8 total, 8 up, 8 in 2024-03-21T13:38:00.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:00 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:38:00.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:00 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:38:00.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:00 smithi097 ceph-mon[39416]: osdmap e498: 8 total, 8 up, 8 in 2024-03-21T13:38:00.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:00 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:38:00.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:00 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:38:00.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:00 smithi144 ceph-mon[33633]: osdmap e498: 8 total, 8 up, 8 in 2024-03-21T13:38:00.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:00 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:38:00.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:00 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:38:01.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:01 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:38:01.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:01 smithi097 ceph-mon[30040]: osdmap e499: 8 total, 8 up, 8 in 2024-03-21T13:38:01.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:01 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:38:01.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:01 smithi097 ceph-mon[30040]: pgmap v766: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 669 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:38:01.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:01 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:38:01.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:01 smithi097 ceph-mon[39416]: osdmap e499: 8 total, 8 up, 8 in 2024-03-21T13:38:01.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:01 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:38:01.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:01 smithi097 ceph-mon[39416]: pgmap v766: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 669 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:38:01.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:01 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:38:01.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:01 smithi144 ceph-mon[33633]: osdmap e499: 8 total, 8 up, 8 in 2024-03-21T13:38:01.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:01 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:38:01.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:01 smithi144 ceph-mon[33633]: pgmap v766: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 669 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:38:02.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:38:01 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:38:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:38:02.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:02 smithi097 ceph-mon[39416]: osdmap e500: 8 total, 8 up, 8 in 2024-03-21T13:38:02.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:02 smithi097 ceph-mon[30040]: osdmap e500: 8 total, 8 up, 8 in 2024-03-21T13:38:02.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:02 smithi144 ceph-mon[33633]: osdmap e500: 8 total, 8 up, 8 in 2024-03-21T13:38:03.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:03 smithi097 ceph-mon[39416]: pgmap v768: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 669 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 242 B/s wr, 1 op/s 2024-03-21T13:38:03.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:03 smithi097 ceph-mon[30040]: pgmap v768: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 669 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 242 B/s wr, 1 op/s 2024-03-21T13:38:03.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:03 smithi144 ceph-mon[33633]: pgmap v768: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 669 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 242 B/s wr, 1 op/s 2024-03-21T13:38:04.371 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:38:03 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:38:04.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:04 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:38:04.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:04 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:38:04.873 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:04 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:38:05.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:05 smithi097 ceph-mon[39416]: pgmap v769: 324 pgs: 324 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 682 B/s rd, 853 B/s wr, 2 op/s 2024-03-21T13:38:05.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:05 smithi097 ceph-mon[30040]: pgmap v769: 324 pgs: 324 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 682 B/s rd, 853 B/s wr, 2 op/s 2024-03-21T13:38:05.870 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:05 smithi144 ceph-mon[33633]: pgmap v769: 324 pgs: 324 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 682 B/s rd, 853 B/s wr, 2 op/s 2024-03-21T13:38:08.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:07 smithi097 ceph-mon[39416]: pgmap v770: 324 pgs: 324 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 568 B/s rd, 710 B/s wr, 2 op/s 2024-03-21T13:38:08.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:07 smithi097 ceph-mon[30040]: pgmap v770: 324 pgs: 324 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 568 B/s rd, 710 B/s wr, 2 op/s 2024-03-21T13:38:08.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:07 smithi144 ceph-mon[33633]: pgmap v770: 324 pgs: 324 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 568 B/s rd, 710 B/s wr, 2 op/s 2024-03-21T13:38:10.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:09 smithi097 ceph-mon[39416]: pgmap v771: 324 pgs: 324 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 623 B/s wr, 2 op/s 2024-03-21T13:38:10.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:09 smithi097 ceph-mon[30040]: pgmap v771: 324 pgs: 324 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 623 B/s wr, 2 op/s 2024-03-21T13:38:10.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:09 smithi144 ceph-mon[33633]: pgmap v771: 324 pgs: 324 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 623 B/s wr, 2 op/s 2024-03-21T13:38:10.954 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:10 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:38:10.955 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:10 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:38:11.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:10 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:38:11.803 INFO:journalctl@ceph.osd.1.smithi097.stdout:Mar 21 13:38:11 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-1[49765]: 2024-03-21T13:38:11.420+0000 7fc962d3c700 -1 snap_mapper.add_oid found existing snaps mapped on 107:63dc08e0:test-rados-api-smithi097-79906-97::foo:21, removing 2024-03-21T13:38:11.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:11 smithi097 ceph-mon[39416]: pgmap v772: 324 pgs: 324 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 921 B/s rd, 511 B/s wr, 1 op/s 2024-03-21T13:38:11.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:11 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:38:11.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:11 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:38:11.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:11 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-96"}]: dispatch 2024-03-21T13:38:11.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:11 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-96"}]: dispatch 2024-03-21T13:38:11.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:11 smithi097 ceph-mon[30040]: pgmap v772: 324 pgs: 324 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 921 B/s rd, 511 B/s wr, 1 op/s 2024-03-21T13:38:11.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:11 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:38:11.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:11 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:38:11.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:11 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-96"}]: dispatch 2024-03-21T13:38:11.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:11 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-96"}]: dispatch 2024-03-21T13:38:11.871 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 13:38:11 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T13:38:11.421+0000 7f88e15aa700 -1 snap_mapper.add_oid found existing snaps mapped on 107:63dc08e0:test-rados-api-smithi097-79906-97::foo:21, removing 2024-03-21T13:38:11.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:11 smithi144 ceph-mon[33633]: pgmap v772: 324 pgs: 324 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 921 B/s rd, 511 B/s wr, 1 op/s 2024-03-21T13:38:11.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:11 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:38:11.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:11 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:38:11.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:11 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-96"}]: dispatch 2024-03-21T13:38:11.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:11 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-96"}]: dispatch 2024-03-21T13:38:11.872 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 21 13:38:11 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-5[45082]: 2024-03-21T13:38:11.421+0000 7f973c3de700 -1 snap_mapper.add_oid found existing snaps mapped on 107:63dc08e0:test-rados-api-smithi097-79906-97::foo:21, removing 2024-03-21T13:38:12.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:38:11 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:38:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:38:13.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:12 smithi097 ceph-mon[30040]: osdmap e501: 8 total, 8 up, 8 in 2024-03-21T13:38:13.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:12 smithi097 ceph-mon[39416]: osdmap e501: 8 total, 8 up, 8 in 2024-03-21T13:38:13.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:12 smithi144 ceph-mon[33633]: osdmap e501: 8 total, 8 up, 8 in 2024-03-21T13:38:13.928 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:13 smithi144 ceph-mon[33633]: pgmap v774: 292 pgs: 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 921 B/s rd, 204 B/s wr, 1 op/s 2024-03-21T13:38:13.928 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:13 smithi144 ceph-mon[33633]: osdmap e502: 8 total, 8 up, 8 in 2024-03-21T13:38:13.928 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:13 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:38:13.928 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:13 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:38:14.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:13 smithi097 ceph-mon[39416]: pgmap v774: 292 pgs: 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 921 B/s rd, 204 B/s wr, 1 op/s 2024-03-21T13:38:14.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:13 smithi097 ceph-mon[39416]: osdmap e502: 8 total, 8 up, 8 in 2024-03-21T13:38:14.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:13 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:38:14.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:13 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:38:14.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:13 smithi097 ceph-mon[30040]: pgmap v774: 292 pgs: 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 921 B/s rd, 204 B/s wr, 1 op/s 2024-03-21T13:38:14.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:13 smithi097 ceph-mon[30040]: osdmap e502: 8 total, 8 up, 8 in 2024-03-21T13:38:14.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:13 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:38:14.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:13 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:38:14.371 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:38:13 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:38:15.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:14 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:38:15.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:14 smithi097 ceph-mon[39416]: osdmap e503: 8 total, 8 up, 8 in 2024-03-21T13:38:15.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:14 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:38:15.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:14 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:38:15.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:14 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:38:15.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:14 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:38:15.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:14 smithi097 ceph-mon[30040]: osdmap e503: 8 total, 8 up, 8 in 2024-03-21T13:38:15.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:14 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:38:15.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:14 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:38:15.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:14 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:38:15.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:14 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:38:15.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:14 smithi144 ceph-mon[33633]: osdmap e503: 8 total, 8 up, 8 in 2024-03-21T13:38:15.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:14 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:38:15.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:14 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:38:15.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:14 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:38:16.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:15 smithi097 ceph-mon[39416]: pgmap v777: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:38:16.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:15 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-98", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:38:16.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:15 smithi097 ceph-mon[39416]: osdmap e504: 8 total, 8 up, 8 in 2024-03-21T13:38:16.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:15 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-98", "mode": "writeback"}]: dispatch 2024-03-21T13:38:16.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:15 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-98", "mode": "writeback"}]: dispatch 2024-03-21T13:38:16.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:15 smithi097 ceph-mon[30040]: pgmap v777: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:38:16.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:15 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-98", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:38:16.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:15 smithi097 ceph-mon[30040]: osdmap e504: 8 total, 8 up, 8 in 2024-03-21T13:38:16.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:15 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-98", "mode": "writeback"}]: dispatch 2024-03-21T13:38:16.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:15 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-98", "mode": "writeback"}]: dispatch 2024-03-21T13:38:16.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:15 smithi144 ceph-mon[33633]: pgmap v777: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:38:16.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:15 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-98", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:38:16.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:15 smithi144 ceph-mon[33633]: osdmap e504: 8 total, 8 up, 8 in 2024-03-21T13:38:16.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:15 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-98", "mode": "writeback"}]: dispatch 2024-03-21T13:38:16.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:15 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-98", "mode": "writeback"}]: dispatch 2024-03-21T13:38:17.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:16 smithi097 ceph-mon[39416]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:38:17.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:16 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-98", "mode": "writeback"}]': finished 2024-03-21T13:38:17.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:16 smithi097 ceph-mon[39416]: osdmap e505: 8 total, 8 up, 8 in 2024-03-21T13:38:17.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:16 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-98"}]: dispatch 2024-03-21T13:38:17.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:16 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-98"}]: dispatch 2024-03-21T13:38:17.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:16 smithi097 ceph-mon[30040]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:38:17.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:16 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-98", "mode": "writeback"}]': finished 2024-03-21T13:38:17.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:16 smithi097 ceph-mon[30040]: osdmap e505: 8 total, 8 up, 8 in 2024-03-21T13:38:17.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:16 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-98"}]: dispatch 2024-03-21T13:38:17.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:16 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-98"}]: dispatch 2024-03-21T13:38:17.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:16 smithi144 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:38:17.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:16 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-98", "mode": "writeback"}]': finished 2024-03-21T13:38:17.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:16 smithi144 ceph-mon[33633]: osdmap e505: 8 total, 8 up, 8 in 2024-03-21T13:38:17.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:16 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-98"}]: dispatch 2024-03-21T13:38:17.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:16 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-98"}]: dispatch 2024-03-21T13:38:18.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:17 smithi097 ceph-mon[30040]: pgmap v780: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:38:18.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:17 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-98"}]': finished 2024-03-21T13:38:18.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:17 smithi097 ceph-mon[30040]: osdmap e506: 8 total, 8 up, 8 in 2024-03-21T13:38:18.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:17 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:38:18.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:17 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:38:18.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:17 smithi097 ceph-mon[39416]: pgmap v780: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:38:18.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:17 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-98"}]': finished 2024-03-21T13:38:18.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:17 smithi097 ceph-mon[39416]: osdmap e506: 8 total, 8 up, 8 in 2024-03-21T13:38:18.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:17 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:38:18.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:17 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:38:18.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:17 smithi144 ceph-mon[33633]: pgmap v780: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:38:18.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:17 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-98"}]': finished 2024-03-21T13:38:18.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:17 smithi144 ceph-mon[33633]: osdmap e506: 8 total, 8 up, 8 in 2024-03-21T13:38:18.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:17 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:38:18.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:17 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:38:19.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:18 smithi097 ceph-mon[39416]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:38:19.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:18 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-98","var": "hit_set_type","val": "bloom"}]': finished 2024-03-21T13:38:19.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:18 smithi097 ceph-mon[39416]: osdmap e507: 8 total, 8 up, 8 in 2024-03-21T13:38:19.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:18 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-98","var": "hit_set_count","val": "1"}]: dispatch 2024-03-21T13:38:19.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:18 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-98","var": "hit_set_count","val": "1"}]: dispatch 2024-03-21T13:38:19.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:18 smithi097 ceph-mon[30040]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:38:19.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:18 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-98","var": "hit_set_type","val": "bloom"}]': finished 2024-03-21T13:38:19.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:18 smithi097 ceph-mon[30040]: osdmap e507: 8 total, 8 up, 8 in 2024-03-21T13:38:19.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:18 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-98","var": "hit_set_count","val": "1"}]: dispatch 2024-03-21T13:38:19.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:18 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-98","var": "hit_set_count","val": "1"}]: dispatch 2024-03-21T13:38:19.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:18 smithi144 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:38:19.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:18 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-98","var": "hit_set_type","val": "bloom"}]': finished 2024-03-21T13:38:19.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:18 smithi144 ceph-mon[33633]: osdmap e507: 8 total, 8 up, 8 in 2024-03-21T13:38:19.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:18 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-98","var": "hit_set_count","val": "1"}]: dispatch 2024-03-21T13:38:19.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:18 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-98","var": "hit_set_count","val": "1"}]: dispatch 2024-03-21T13:38:19.950 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d4450 2024-03-21T13:38:19.950 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d4580 2024-03-21T13:38:19.950 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d5070 2024-03-21T13:38:19.950 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d5540 2024-03-21T13:38:19.950 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d5670 2024-03-21T13:38:19.950 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d5180 2024-03-21T13:38:19.951 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d52d0 2024-03-21T13:38:19.951 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d5400 2024-03-21T13:38:19.951 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d57c0 2024-03-21T13:38:19.951 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d5c90 2024-03-21T13:38:19.951 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d5dc0 2024-03-21T13:38:19.951 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d6380 2024-03-21T13:38:19.951 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d58f0 2024-03-21T13:38:19.951 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d5a00 2024-03-21T13:38:19.951 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d5b50 2024-03-21T13:38:19.951 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d64d0 2024-03-21T13:38:19.951 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d5ed0 2024-03-21T13:38:19.951 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d6020 2024-03-21T13:38:19.951 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d6150 2024-03-21T13:38:19.951 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d65e0 2024-03-21T13:38:19.951 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d6ab0 2024-03-21T13:38:19.951 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d6c00 2024-03-21T13:38:19.952 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d4a30 2024-03-21T13:38:19.952 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d4b60 2024-03-21T13:38:19.952 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d4c90 2024-03-21T13:38:19.952 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d6d30 2024-03-21T13:38:19.952 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d7200 2024-03-21T13:38:19.952 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d7330 2024-03-21T13:38:19.952 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d78f0 2024-03-21T13:38:19.952 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d6e60 2024-03-21T13:38:19.952 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d6f90 2024-03-21T13:38:19.952 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d70e0 2024-03-21T13:38:19.952 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d7a40 2024-03-21T13:38:19.952 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d7440 2024-03-21T13:38:19.952 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d7570 2024-03-21T13:38:19.952 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d7680 2024-03-21T13:38:19.952 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d77d0 2024-03-21T13:38:19.952 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d8020 2024-03-21T13:38:19.952 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d7b70 2024-03-21T13:38:19.953 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d7cc0 2024-03-21T13:38:19.953 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d7dd0 2024-03-21T13:38:19.953 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d7f00 2024-03-21T13:38:19.953 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d8620 2024-03-21T13:38:19.953 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d8170 2024-03-21T13:38:19.953 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d82a0 2024-03-21T13:38:19.953 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d83b0 2024-03-21T13:38:19.953 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d8500 2024-03-21T13:38:19.953 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d8c20 2024-03-21T13:38:19.953 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d8770 2024-03-21T13:38:19.953 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d88c0 2024-03-21T13:38:19.953 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 0x55ca2a5d89f0 2024-03-21T13:38:19.953 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:38:19.953 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:38:19.953 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:38:19.953 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:38:19.954 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:38:19.954 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:38:19.954 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:38:19.954 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:38:19.954 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:38:19.954 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:38:19.954 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:38:19.954 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:38:19.954 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:38:19.954 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:38:19.954 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:38:19.954 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:38:19.954 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:38:19.954 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:38:19.955 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:38:19.955 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:38:19.955 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:38:19.955 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:38:19.955 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:38:19.955 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:38:19.955 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:38:19.955 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:38:19.955 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:38:19.955 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:38:19.955 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:38:19.955 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:38:19.955 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:38:19.955 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:38:19.955 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:38:19.955 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:38:19.956 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:38:19.956 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:38:19.956 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:38:19.956 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:38:20.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:19 smithi097 ceph-mon[30040]: pgmap v783: 324 pgs: 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:38:20.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:19 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-98","var": "hit_set_count","val": "1"}]': finished 2024-03-21T13:38:20.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:19 smithi097 ceph-mon[30040]: osdmap e508: 8 total, 8 up, 8 in 2024-03-21T13:38:20.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:19 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-98","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:38:20.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:19 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-98","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:38:20.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:19 smithi097 ceph-mon[39416]: pgmap v783: 324 pgs: 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:38:20.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:19 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-98","var": "hit_set_count","val": "1"}]': finished 2024-03-21T13:38:20.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:19 smithi097 ceph-mon[39416]: osdmap e508: 8 total, 8 up, 8 in 2024-03-21T13:38:20.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:19 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-98","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:38:20.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:19 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-98","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:38:20.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:19 smithi144 ceph-mon[33633]: pgmap v783: 324 pgs: 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:38:20.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:19 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-98","var": "hit_set_count","val": "1"}]': finished 2024-03-21T13:38:20.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:19 smithi144 ceph-mon[33633]: osdmap e508: 8 total, 8 up, 8 in 2024-03-21T13:38:20.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:19 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-98","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:38:20.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:19 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-98","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:38:21.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:20 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-98","var": "hit_set_period","val": "600"}]': finished 2024-03-21T13:38:21.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:20 smithi097 ceph-mon[30040]: osdmap e509: 8 total, 8 up, 8 in 2024-03-21T13:38:21.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:20 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-98","var": "target_max_objects","val": "250"}]: dispatch 2024-03-21T13:38:21.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:20 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-98","var": "target_max_objects","val": "250"}]: dispatch 2024-03-21T13:38:21.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:20 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-98","var": "target_max_objects","val": "250"}]': finished 2024-03-21T13:38:21.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:20 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-98","var": "hit_set_period","val": "600"}]': finished 2024-03-21T13:38:21.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:20 smithi097 ceph-mon[39416]: osdmap e509: 8 total, 8 up, 8 in 2024-03-21T13:38:21.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:20 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-98","var": "target_max_objects","val": "250"}]: dispatch 2024-03-21T13:38:21.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:20 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-98","var": "target_max_objects","val": "250"}]: dispatch 2024-03-21T13:38:21.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:20 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-98","var": "target_max_objects","val": "250"}]': finished 2024-03-21T13:38:21.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:20 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-98","var": "hit_set_period","val": "600"}]': finished 2024-03-21T13:38:21.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:20 smithi144 ceph-mon[33633]: osdmap e509: 8 total, 8 up, 8 in 2024-03-21T13:38:21.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:20 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-98","var": "target_max_objects","val": "250"}]: dispatch 2024-03-21T13:38:21.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:20 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-98","var": "target_max_objects","val": "250"}]: dispatch 2024-03-21T13:38:21.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:20 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-98","var": "target_max_objects","val": "250"}]': finished 2024-03-21T13:38:22.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:21 smithi097 ceph-mon[30040]: pgmap v786: 324 pgs: 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:38:22.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:21 smithi097 ceph-mon[30040]: osdmap e510: 8 total, 8 up, 8 in 2024-03-21T13:38:22.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:21 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:38:22.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:21 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:38:22.054 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:38:21 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:38:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:38:22.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:21 smithi097 ceph-mon[39416]: pgmap v786: 324 pgs: 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:38:22.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:21 smithi097 ceph-mon[39416]: osdmap e510: 8 total, 8 up, 8 in 2024-03-21T13:38:22.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:21 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:38:22.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:21 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:38:22.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:21 smithi144 ceph-mon[33633]: pgmap v786: 324 pgs: 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:38:22.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:21 smithi144 ceph-mon[33633]: osdmap e510: 8 total, 8 up, 8 in 2024-03-21T13:38:22.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:21 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:38:22.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:21 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:38:23.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:22 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:38:23.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:22 smithi097 ceph-mon[30040]: osdmap e511: 8 total, 8 up, 8 in 2024-03-21T13:38:23.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:22 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-98"}]: dispatch 2024-03-21T13:38:23.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:22 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-98"}]: dispatch 2024-03-21T13:38:23.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:22 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-98"}]': finished 2024-03-21T13:38:23.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:22 smithi097 ceph-mon[30040]: osdmap e512: 8 total, 8 up, 8 in 2024-03-21T13:38:23.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:22 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:38:23.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:22 smithi097 ceph-mon[39416]: osdmap e511: 8 total, 8 up, 8 in 2024-03-21T13:38:23.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:22 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-98"}]: dispatch 2024-03-21T13:38:23.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:22 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-98"}]: dispatch 2024-03-21T13:38:23.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:22 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-98"}]': finished 2024-03-21T13:38:23.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:22 smithi097 ceph-mon[39416]: osdmap e512: 8 total, 8 up, 8 in 2024-03-21T13:38:23.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:22 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:38:23.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:22 smithi144 ceph-mon[33633]: osdmap e511: 8 total, 8 up, 8 in 2024-03-21T13:38:23.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:22 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-98"}]: dispatch 2024-03-21T13:38:23.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:22 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-98"}]: dispatch 2024-03-21T13:38:23.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:22 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-98"}]': finished 2024-03-21T13:38:23.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:22 smithi144 ceph-mon[33633]: osdmap e512: 8 total, 8 up, 8 in 2024-03-21T13:38:24.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:23 smithi097 ceph-mon[39416]: pgmap v789: 324 pgs: 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:38:24.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:23 smithi097 ceph-mon[30040]: pgmap v789: 324 pgs: 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:38:24.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:23 smithi144 ceph-mon[33633]: pgmap v789: 324 pgs: 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:38:24.121 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:38:23 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:38:25.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:24 smithi097 ceph-mon[39416]: osdmap e513: 8 total, 8 up, 8 in 2024-03-21T13:38:25.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:24 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:38:25.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:24 smithi097 ceph-mon[30040]: osdmap e513: 8 total, 8 up, 8 in 2024-03-21T13:38:25.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:24 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:38:25.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:24 smithi144 ceph-mon[33633]: osdmap e513: 8 total, 8 up, 8 in 2024-03-21T13:38:25.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:24 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:38:25.796 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:25 smithi097 ceph-mon[39416]: pgmap v792: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:38:25.796 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:25 smithi097 ceph-mon[39416]: osdmap e514: 8 total, 8 up, 8 in 2024-03-21T13:38:25.796 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:25 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:38:25.797 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:25 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:38:25.797 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:25 smithi097 ceph-mon[30040]: pgmap v792: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:38:25.797 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:25 smithi097 ceph-mon[30040]: osdmap e514: 8 total, 8 up, 8 in 2024-03-21T13:38:25.797 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:25 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:38:25.797 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:25 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:38:25.797 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:25 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:38:26.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:25 smithi144 ceph-mon[33633]: pgmap v792: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:38:26.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:25 smithi144 ceph-mon[33633]: osdmap e514: 8 total, 8 up, 8 in 2024-03-21T13:38:26.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:25 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:38:26.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:25 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:38:26.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:25 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:38:27.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:25 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:38:27.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:26 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:38:27.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:26 smithi097 ceph-mon[39416]: osdmap e515: 8 total, 8 up, 8 in 2024-03-21T13:38:27.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:26 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:38:27.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:26 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:38:27.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:26 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:38:27.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:26 smithi097 ceph-mon[30040]: osdmap e515: 8 total, 8 up, 8 in 2024-03-21T13:38:27.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:26 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:38:27.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:26 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:38:27.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:26 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:38:27.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:26 smithi144 ceph-mon[33633]: osdmap e515: 8 total, 8 up, 8 in 2024-03-21T13:38:27.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:26 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:38:27.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:26 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:38:28.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:27 smithi144 ceph-mon[33633]: pgmap v795: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:38:28.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:27 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-100", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:38:28.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:27 smithi144 ceph-mon[33633]: osdmap e516: 8 total, 8 up, 8 in 2024-03-21T13:38:28.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:27 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-100"}]: dispatch 2024-03-21T13:38:28.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:27 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-100"}]: dispatch 2024-03-21T13:38:28.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:27 smithi097 ceph-mon[39416]: pgmap v795: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:38:28.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:27 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-100", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:38:28.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:27 smithi097 ceph-mon[39416]: osdmap e516: 8 total, 8 up, 8 in 2024-03-21T13:38:28.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:27 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-100"}]: dispatch 2024-03-21T13:38:28.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:27 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-100"}]: dispatch 2024-03-21T13:38:28.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:27 smithi097 ceph-mon[30040]: pgmap v795: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:38:28.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:27 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-100", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:38:28.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:27 smithi097 ceph-mon[30040]: osdmap e516: 8 total, 8 up, 8 in 2024-03-21T13:38:28.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:27 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-100"}]: dispatch 2024-03-21T13:38:28.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:27 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-100"}]: dispatch 2024-03-21T13:38:29.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:28 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-100"}]': finished 2024-03-21T13:38:29.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:28 smithi144 ceph-mon[33633]: osdmap e517: 8 total, 8 up, 8 in 2024-03-21T13:38:29.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:28 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-100", "mode": "writeback"}]: dispatch 2024-03-21T13:38:29.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:28 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-100", "mode": "writeback"}]: dispatch 2024-03-21T13:38:29.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:28 smithi144 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:38:29.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:28 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-100", "mode": "writeback"}]': finished 2024-03-21T13:38:29.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:28 smithi144 ceph-mon[33633]: osdmap e518: 8 total, 8 up, 8 in 2024-03-21T13:38:29.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:28 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-100"}]': finished 2024-03-21T13:38:29.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:28 smithi097 ceph-mon[39416]: osdmap e517: 8 total, 8 up, 8 in 2024-03-21T13:38:29.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:28 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-100", "mode": "writeback"}]: dispatch 2024-03-21T13:38:29.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:28 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-100", "mode": "writeback"}]: dispatch 2024-03-21T13:38:29.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:28 smithi097 ceph-mon[39416]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:38:29.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:28 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-100", "mode": "writeback"}]': finished 2024-03-21T13:38:29.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:28 smithi097 ceph-mon[39416]: osdmap e518: 8 total, 8 up, 8 in 2024-03-21T13:38:29.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:28 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-6", "overlaypool": "test-rados-api-smithi097-79906-100"}]': finished 2024-03-21T13:38:29.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:28 smithi097 ceph-mon[30040]: osdmap e517: 8 total, 8 up, 8 in 2024-03-21T13:38:29.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:28 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-100", "mode": "writeback"}]: dispatch 2024-03-21T13:38:29.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:28 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-100", "mode": "writeback"}]: dispatch 2024-03-21T13:38:29.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:28 smithi097 ceph-mon[30040]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:38:29.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:28 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-100", "mode": "writeback"}]': finished 2024-03-21T13:38:29.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:28 smithi097 ceph-mon[30040]: osdmap e518: 8 total, 8 up, 8 in 2024-03-21T13:38:30.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:29 smithi144 ceph-mon[33633]: pgmap v798: 324 pgs: 324 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:38:30.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:29 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-100","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:38:30.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:29 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-100","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:38:30.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:29 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-100","var": "hit_set_count","val": "2"}]': finished 2024-03-21T13:38:30.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:29 smithi144 ceph-mon[33633]: osdmap e519: 8 total, 8 up, 8 in 2024-03-21T13:38:30.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:29 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-100","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:38:30.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:29 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-100","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:38:30.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:29 smithi097 ceph-mon[39416]: pgmap v798: 324 pgs: 324 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:38:30.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:29 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-100","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:38:30.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:29 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-100","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:38:30.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:29 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-100","var": "hit_set_count","val": "2"}]': finished 2024-03-21T13:38:30.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:29 smithi097 ceph-mon[39416]: osdmap e519: 8 total, 8 up, 8 in 2024-03-21T13:38:30.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:29 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-100","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:38:30.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:29 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-100","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:38:30.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:29 smithi097 ceph-mon[30040]: pgmap v798: 324 pgs: 324 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:38:30.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:29 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-100","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:38:30.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:29 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-100","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:38:30.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:29 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-100","var": "hit_set_count","val": "2"}]': finished 2024-03-21T13:38:30.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:29 smithi097 ceph-mon[30040]: osdmap e519: 8 total, 8 up, 8 in 2024-03-21T13:38:30.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:29 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-100","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:38:30.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:29 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-100","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:38:32.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:31 smithi144 ceph-mon[33633]: pgmap v801: 324 pgs: 324 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:38:32.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:31 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-100","var": "hit_set_period","val": "600"}]': finished 2024-03-21T13:38:32.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:31 smithi144 ceph-mon[33633]: osdmap e520: 8 total, 8 up, 8 in 2024-03-21T13:38:32.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:31 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-21T13:38:32.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:31 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-21T13:38:32.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:38:31 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:38:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:38:32.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:31 smithi097 ceph-mon[39416]: pgmap v801: 324 pgs: 324 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:38:32.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:31 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-100","var": "hit_set_period","val": "600"}]': finished 2024-03-21T13:38:32.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:31 smithi097 ceph-mon[39416]: osdmap e520: 8 total, 8 up, 8 in 2024-03-21T13:38:32.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:31 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-21T13:38:32.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:31 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-21T13:38:32.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:31 smithi097 ceph-mon[30040]: pgmap v801: 324 pgs: 324 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:38:32.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:31 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-100","var": "hit_set_period","val": "600"}]': finished 2024-03-21T13:38:32.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:31 smithi097 ceph-mon[30040]: osdmap e520: 8 total, 8 up, 8 in 2024-03-21T13:38:32.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:31 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-21T13:38:32.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:31 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-21T13:38:33.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:32 smithi144 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:38:33.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:32 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-21T13:38:33.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:32 smithi144 ceph-mon[33633]: osdmap e521: 8 total, 8 up, 8 in 2024-03-21T13:38:33.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:32 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-03-21T13:38:33.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:32 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-03-21T13:38:33.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:32 smithi097 ceph-mon[39416]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:38:33.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:32 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-21T13:38:33.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:32 smithi097 ceph-mon[39416]: osdmap e521: 8 total, 8 up, 8 in 2024-03-21T13:38:33.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:32 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-03-21T13:38:33.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:32 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-03-21T13:38:33.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:32 smithi097 ceph-mon[30040]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:38:33.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:32 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-21T13:38:33.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:32 smithi097 ceph-mon[30040]: osdmap e521: 8 total, 8 up, 8 in 2024-03-21T13:38:33.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:32 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-03-21T13:38:33.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:32 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-03-21T13:38:34.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:33 smithi144 ceph-mon[33633]: pgmap v804: 324 pgs: 324 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:38:34.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:33 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-03-21T13:38:34.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:33 smithi144 ceph-mon[33633]: osdmap e522: 8 total, 8 up, 8 in 2024-03-21T13:38:34.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:33 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:38:34.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:33 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:38:34.122 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:38:33 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:38:34.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:33 smithi097 ceph-mon[39416]: pgmap v804: 324 pgs: 324 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:38:34.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:33 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-03-21T13:38:34.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:33 smithi097 ceph-mon[39416]: osdmap e522: 8 total, 8 up, 8 in 2024-03-21T13:38:34.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:33 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:38:34.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:33 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:38:34.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:33 smithi097 ceph-mon[30040]: pgmap v804: 324 pgs: 324 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:38:34.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:33 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-03-21T13:38:34.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:33 smithi097 ceph-mon[30040]: osdmap e522: 8 total, 8 up, 8 in 2024-03-21T13:38:34.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:33 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:38:34.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:33 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:38:35.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:34 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:38:35.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:34 smithi144 ceph-mon[33633]: osdmap e523: 8 total, 8 up, 8 in 2024-03-21T13:38:35.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:34 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-100"}]: dispatch 2024-03-21T13:38:35.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:34 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-100"}]: dispatch 2024-03-21T13:38:35.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:34 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:38:35.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:34 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-100"}]': finished 2024-03-21T13:38:35.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:34 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:38:35.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:34 smithi097 ceph-mon[39416]: osdmap e523: 8 total, 8 up, 8 in 2024-03-21T13:38:35.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:34 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-100"}]: dispatch 2024-03-21T13:38:35.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:34 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-100"}]: dispatch 2024-03-21T13:38:35.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:34 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:38:35.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:34 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-100"}]': finished 2024-03-21T13:38:35.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:34 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]': finished 2024-03-21T13:38:35.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:34 smithi097 ceph-mon[30040]: osdmap e523: 8 total, 8 up, 8 in 2024-03-21T13:38:35.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:34 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-100"}]: dispatch 2024-03-21T13:38:35.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:34 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-100"}]: dispatch 2024-03-21T13:38:35.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:34 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:38:35.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:34 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-100"}]': finished 2024-03-21T13:38:36.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:35 smithi144 ceph-mon[33633]: pgmap v807: 324 pgs: 324 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-21T13:38:36.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:35 smithi144 ceph-mon[33633]: osdmap e524: 8 total, 8 up, 8 in 2024-03-21T13:38:36.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:35 smithi097 ceph-mon[39416]: pgmap v807: 324 pgs: 324 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-21T13:38:36.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:35 smithi097 ceph-mon[39416]: osdmap e524: 8 total, 8 up, 8 in 2024-03-21T13:38:36.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:35 smithi097 ceph-mon[30040]: pgmap v807: 324 pgs: 324 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-21T13:38:36.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:35 smithi097 ceph-mon[30040]: osdmap e524: 8 total, 8 up, 8 in 2024-03-21T13:38:37.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:36 smithi144 ceph-mon[33633]: osdmap e525: 8 total, 8 up, 8 in 2024-03-21T13:38:37.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:36 smithi097 ceph-mon[39416]: osdmap e525: 8 total, 8 up, 8 in 2024-03-21T13:38:37.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:36 smithi097 ceph-mon[30040]: osdmap e525: 8 total, 8 up, 8 in 2024-03-21T13:38:38.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:37 smithi144 ceph-mon[33633]: pgmap v810: 292 pgs: 292 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:38:38.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:37 smithi144 ceph-mon[33633]: osdmap e526: 8 total, 8 up, 8 in 2024-03-21T13:38:38.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:37 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:38:38.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:37 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:38:38.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:37 smithi097 ceph-mon[39416]: pgmap v810: 292 pgs: 292 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:38:38.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:37 smithi097 ceph-mon[39416]: osdmap e526: 8 total, 8 up, 8 in 2024-03-21T13:38:38.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:37 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:38:38.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:37 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:38:38.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:37 smithi097 ceph-mon[30040]: pgmap v810: 292 pgs: 292 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:38:38.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:37 smithi097 ceph-mon[30040]: osdmap e526: 8 total, 8 up, 8 in 2024-03-21T13:38:38.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:37 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:38:38.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:37 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:38:39.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:38 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:38:39.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:38 smithi097 ceph-mon[39416]: osdmap e527: 8 total, 8 up, 8 in 2024-03-21T13:38:39.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:38 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:38:39.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:38 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:38:39.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:38 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:38:39.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:38 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:38:39.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:38 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:38:39.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:38 smithi097 ceph-mon[30040]: osdmap e527: 8 total, 8 up, 8 in 2024-03-21T13:38:39.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:38 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:38:39.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:38 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:38:39.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:38 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:38:39.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:38 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:38:39.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:38 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:38:39.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:38 smithi144 ceph-mon[33633]: osdmap e527: 8 total, 8 up, 8 in 2024-03-21T13:38:39.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:38 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:38:39.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:38 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:38:39.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:38 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:38:39.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:38 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:38:40.204 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:39 smithi097 ceph-mon[39416]: pgmap v813: 324 pgs: 324 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:38:40.204 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:39 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:38:40.204 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:39 smithi097 ceph-mon[39416]: osdmap e528: 8 total, 8 up, 8 in 2024-03-21T13:38:40.204 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:39 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:38:40.204 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:39 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:38:40.204 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:39 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:38:40.205 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:39 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:38:40.205 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:39 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:38:40.205 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:39 smithi097 ceph-mon[30040]: pgmap v813: 324 pgs: 324 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:38:40.205 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:39 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:38:40.205 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:39 smithi097 ceph-mon[30040]: osdmap e528: 8 total, 8 up, 8 in 2024-03-21T13:38:40.205 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:39 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:38:40.205 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:39 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:38:40.205 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:39 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:38:40.206 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:39 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:38:40.206 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:39 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:38:40.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:39 smithi144 ceph-mon[33633]: pgmap v813: 324 pgs: 324 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:38:40.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:39 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:38:40.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:39 smithi144 ceph-mon[33633]: osdmap e528: 8 total, 8 up, 8 in 2024-03-21T13:38:40.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:39 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:38:40.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:39 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:38:40.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:39 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:38:40.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:39 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:38:40.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:39 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:38:41.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:40 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-21T13:38:41.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:40 smithi097 ceph-mon[30040]: osdmap e529: 8 total, 8 up, 8 in 2024-03-21T13:38:41.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:40 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:38:41.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:40 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:38:41.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:40 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:38:41.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:40 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-21T13:38:41.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:40 smithi097 ceph-mon[39416]: osdmap e529: 8 total, 8 up, 8 in 2024-03-21T13:38:41.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:40 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:38:41.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:40 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:38:41.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:40 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:38:41.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:40 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-21T13:38:41.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:40 smithi144 ceph-mon[33633]: osdmap e529: 8 total, 8 up, 8 in 2024-03-21T13:38:41.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:40 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:38:41.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:40 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:38:41.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:40 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:38:42.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:38:41 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:38:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:38:42.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:41 smithi097 ceph-mon[30040]: pgmap v816: 324 pgs: 324 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:38:42.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:41 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-21T13:38:42.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:41 smithi097 ceph-mon[30040]: osdmap e530: 8 total, 8 up, 8 in 2024-03-21T13:38:42.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:41 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:38:42.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:41 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:38:42.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:41 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-102"}]: dispatch 2024-03-21T13:38:42.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:41 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-102"}]: dispatch 2024-03-21T13:38:42.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:41 smithi097 ceph-mon[39416]: pgmap v816: 324 pgs: 324 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:38:42.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:41 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-21T13:38:42.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:41 smithi097 ceph-mon[39416]: osdmap e530: 8 total, 8 up, 8 in 2024-03-21T13:38:42.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:41 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:38:42.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:41 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:38:42.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:41 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-102"}]: dispatch 2024-03-21T13:38:42.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:41 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-102"}]: dispatch 2024-03-21T13:38:42.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:41 smithi144 ceph-mon[33633]: pgmap v816: 324 pgs: 324 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:38:42.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:41 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-21T13:38:42.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:41 smithi144 ceph-mon[33633]: osdmap e530: 8 total, 8 up, 8 in 2024-03-21T13:38:42.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:41 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:38:42.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:41 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-6"}]: dispatch 2024-03-21T13:38:42.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:41 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1757309821' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-102"}]: dispatch 2024-03-21T13:38:42.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:41 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-6", "tierpool": "test-rados-api-smithi097-79906-102"}]: dispatch 2024-03-21T13:38:43.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:42 smithi097 ceph-mon[39416]: osdmap e531: 8 total, 8 up, 8 in 2024-03-21T13:38:43.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:42 smithi097 ceph-mon[30040]: osdmap e531: 8 total, 8 up, 8 in 2024-03-21T13:38:43.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:42 smithi144 ceph-mon[33633]: osdmap e531: 8 total, 8 up, 8 in 2024-03-21T13:38:44.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:43 smithi097 ceph-mon[39416]: pgmap v819: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:38:44.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:43 smithi097 ceph-mon[39416]: osdmap e532: 8 total, 8 up, 8 in 2024-03-21T13:38:44.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:43 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/553707059' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi097-79906-104"}]: dispatch 2024-03-21T13:38:44.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:43 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/553707059' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi097-79906-104"}]: dispatch 2024-03-21T13:38:44.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:43 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/553707059' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi097-79906-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:38:44.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:43 smithi097 ceph-mon[30040]: pgmap v819: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:38:44.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:43 smithi097 ceph-mon[30040]: osdmap e532: 8 total, 8 up, 8 in 2024-03-21T13:38:44.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:43 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/553707059' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi097-79906-104"}]: dispatch 2024-03-21T13:38:44.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:43 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/553707059' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi097-79906-104"}]: dispatch 2024-03-21T13:38:44.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:43 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/553707059' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi097-79906-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:38:44.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:43 smithi144 ceph-mon[33633]: pgmap v819: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:38:44.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:43 smithi144 ceph-mon[33633]: osdmap e532: 8 total, 8 up, 8 in 2024-03-21T13:38:44.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:43 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/553707059' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi097-79906-104"}]: dispatch 2024-03-21T13:38:44.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:43 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/553707059' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi097-79906-104"}]: dispatch 2024-03-21T13:38:44.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:43 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/553707059' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi097-79906-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:38:44.372 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:38:43 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:38:45.272 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:44 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/553707059' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi097-79906-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:38:45.273 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:44 smithi144 ceph-mon[33633]: osdmap e533: 8 total, 8 up, 8 in 2024-03-21T13:38:45.273 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:44 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/553707059' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi097-79906-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi097-79906-104"}]: dispatch 2024-03-21T13:38:45.273 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:44 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:38:45.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:44 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/553707059' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi097-79906-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:38:45.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:44 smithi097 ceph-mon[39416]: osdmap e533: 8 total, 8 up, 8 in 2024-03-21T13:38:45.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:44 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/553707059' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi097-79906-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi097-79906-104"}]: dispatch 2024-03-21T13:38:45.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:44 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:38:45.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:44 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/553707059' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi097-79906-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:38:45.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:44 smithi097 ceph-mon[30040]: osdmap e533: 8 total, 8 up, 8 in 2024-03-21T13:38:45.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:44 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/553707059' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi097-79906-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi097-79906-104"}]: dispatch 2024-03-21T13:38:45.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:44 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:38:46.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:45 smithi097 ceph-mon[39416]: pgmap v822: 260 pgs: 260 active+clean; 582 KiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:38:46.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:45 smithi097 ceph-mon[39416]: osdmap e534: 8 total, 8 up, 8 in 2024-03-21T13:38:46.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:45 smithi097 ceph-mon[30040]: pgmap v822: 260 pgs: 260 active+clean; 582 KiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:38:46.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:45 smithi097 ceph-mon[30040]: osdmap e534: 8 total, 8 up, 8 in 2024-03-21T13:38:46.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:45 smithi144 ceph-mon[33633]: pgmap v822: 260 pgs: 260 active+clean; 582 KiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:38:46.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:45 smithi144 ceph-mon[33633]: osdmap e534: 8 total, 8 up, 8 in 2024-03-21T13:38:47.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:46 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/553707059' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi097-79906-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi097-79906-104"}]': finished 2024-03-21T13:38:47.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:46 smithi097 ceph-mon[39416]: osdmap e535: 8 total, 8 up, 8 in 2024-03-21T13:38:47.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:46 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/553707059' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi097-79906-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi097-79906-104"}]': finished 2024-03-21T13:38:47.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:46 smithi097 ceph-mon[30040]: osdmap e535: 8 total, 8 up, 8 in 2024-03-21T13:38:47.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:46 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/553707059' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi097-79906-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi097-79906-104"}]': finished 2024-03-21T13:38:47.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:46 smithi144 ceph-mon[33633]: osdmap e535: 8 total, 8 up, 8 in 2024-03-21T13:38:48.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:47 smithi097 ceph-mon[39416]: pgmap v825: 268 pgs: 8 unknown, 260 active+clean; 582 KiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:38:48.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:47 smithi097 ceph-mon[39416]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:38:48.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:47 smithi097 ceph-mon[39416]: osdmap e536: 8 total, 8 up, 8 in 2024-03-21T13:38:48.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:47 smithi097 ceph-mon[30040]: pgmap v825: 268 pgs: 8 unknown, 260 active+clean; 582 KiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:38:48.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:47 smithi097 ceph-mon[30040]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:38:48.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:47 smithi097 ceph-mon[30040]: osdmap e536: 8 total, 8 up, 8 in 2024-03-21T13:38:48.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:47 smithi144 ceph-mon[33633]: pgmap v825: 268 pgs: 8 unknown, 260 active+clean; 582 KiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:38:48.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:47 smithi144 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:38:48.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:47 smithi144 ceph-mon[33633]: osdmap e536: 8 total, 8 up, 8 in 2024-03-21T13:38:49.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:49 smithi097 ceph-mon[39416]: osdmap e537: 8 total, 8 up, 8 in 2024-03-21T13:38:49.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:49 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1961013474' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:38:49.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:49 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1961013474' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:38:49.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:49 smithi097 ceph-mon[39416]: osdmap e538: 8 total, 8 up, 8 in 2024-03-21T13:38:49.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:49 smithi097 ceph-mon[30040]: osdmap e537: 8 total, 8 up, 8 in 2024-03-21T13:38:49.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:49 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1961013474' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:38:49.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:49 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1961013474' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:38:49.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:49 smithi097 ceph-mon[30040]: osdmap e538: 8 total, 8 up, 8 in 2024-03-21T13:38:49.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:49 smithi144 ceph-mon[33633]: osdmap e537: 8 total, 8 up, 8 in 2024-03-21T13:38:49.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:49 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1961013474' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:38:49.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:49 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1961013474' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:38:49.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:49 smithi144 ceph-mon[33633]: osdmap e538: 8 total, 8 up, 8 in 2024-03-21T13:38:50.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:50 smithi097 ceph-mon[30040]: pgmap v828: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:38:50.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:50 smithi097 ceph-mon[30040]: osdmap e539: 8 total, 8 up, 8 in 2024-03-21T13:38:50.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:50 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1961013474' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:38:50.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:50 smithi097 ceph-mon[39416]: pgmap v828: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:38:50.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:50 smithi097 ceph-mon[39416]: osdmap e539: 8 total, 8 up, 8 in 2024-03-21T13:38:50.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:50 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1961013474' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:38:50.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:50 smithi144 ceph-mon[33633]: pgmap v828: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:38:50.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:50 smithi144 ceph-mon[33633]: osdmap e539: 8 total, 8 up, 8 in 2024-03-21T13:38:50.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:50 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1961013474' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:38:52.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:38:51 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:38:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:38:52.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:51 smithi097 ceph-mon[39416]: pgmap v831: 332 pgs: 32 unknown, 32 creating+peering, 268 active+clean; 582 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:38:52.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:51 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1961013474' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:38:52.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:51 smithi097 ceph-mon[39416]: osdmap e540: 8 total, 8 up, 8 in 2024-03-21T13:38:52.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:51 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1961013474' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-107", "tierpool": "test-rados-api-smithi097-79906-107-cache"}]: dispatch 2024-03-21T13:38:52.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:51 smithi097 ceph-mon[30040]: pgmap v831: 332 pgs: 32 unknown, 32 creating+peering, 268 active+clean; 582 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:38:52.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:51 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1961013474' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:38:52.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:51 smithi097 ceph-mon[30040]: osdmap e540: 8 total, 8 up, 8 in 2024-03-21T13:38:52.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:51 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1961013474' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-107", "tierpool": "test-rados-api-smithi097-79906-107-cache"}]: dispatch 2024-03-21T13:38:52.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:51 smithi144 ceph-mon[33633]: pgmap v831: 332 pgs: 32 unknown, 32 creating+peering, 268 active+clean; 582 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:38:52.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:51 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1961013474' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:38:52.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:51 smithi144 ceph-mon[33633]: osdmap e540: 8 total, 8 up, 8 in 2024-03-21T13:38:52.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:51 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1961013474' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-107", "tierpool": "test-rados-api-smithi097-79906-107-cache"}]: dispatch 2024-03-21T13:38:53.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:52 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1961013474' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-107", "tierpool": "test-rados-api-smithi097-79906-107-cache"}]': finished 2024-03-21T13:38:53.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:52 smithi097 ceph-mon[30040]: osdmap e541: 8 total, 8 up, 8 in 2024-03-21T13:38:53.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:52 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1961013474' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-107", "overlaypool": "test-rados-api-smithi097-79906-107-cache"}]: dispatch 2024-03-21T13:38:53.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:52 smithi097 ceph-mon[30040]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:38:53.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:52 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1961013474' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-107", "overlaypool": "test-rados-api-smithi097-79906-107-cache"}]': finished 2024-03-21T13:38:53.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:52 smithi097 ceph-mon[30040]: osdmap e542: 8 total, 8 up, 8 in 2024-03-21T13:38:53.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:52 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1961013474' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-107-cache", "mode": "writeback"}]: dispatch 2024-03-21T13:38:53.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:52 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1961013474' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-107", "tierpool": "test-rados-api-smithi097-79906-107-cache"}]': finished 2024-03-21T13:38:53.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:52 smithi097 ceph-mon[39416]: osdmap e541: 8 total, 8 up, 8 in 2024-03-21T13:38:53.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:52 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1961013474' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-107", "overlaypool": "test-rados-api-smithi097-79906-107-cache"}]: dispatch 2024-03-21T13:38:53.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:52 smithi097 ceph-mon[39416]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:38:53.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:52 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1961013474' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-107", "overlaypool": "test-rados-api-smithi097-79906-107-cache"}]': finished 2024-03-21T13:38:53.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:52 smithi097 ceph-mon[39416]: osdmap e542: 8 total, 8 up, 8 in 2024-03-21T13:38:53.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:52 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1961013474' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-107-cache", "mode": "writeback"}]: dispatch 2024-03-21T13:38:53.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:52 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1961013474' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-107", "tierpool": "test-rados-api-smithi097-79906-107-cache"}]': finished 2024-03-21T13:38:53.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:52 smithi144 ceph-mon[33633]: osdmap e541: 8 total, 8 up, 8 in 2024-03-21T13:38:53.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:52 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1961013474' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-107", "overlaypool": "test-rados-api-smithi097-79906-107-cache"}]: dispatch 2024-03-21T13:38:53.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:52 smithi144 ceph-mon[33633]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:38:53.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:52 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1961013474' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-107", "overlaypool": "test-rados-api-smithi097-79906-107-cache"}]': finished 2024-03-21T13:38:53.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:52 smithi144 ceph-mon[33633]: osdmap e542: 8 total, 8 up, 8 in 2024-03-21T13:38:53.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:52 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1961013474' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-107-cache", "mode": "writeback"}]: dispatch 2024-03-21T13:38:54.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:53 smithi097 ceph-mon[39416]: pgmap v834: 332 pgs: 32 unknown, 32 creating+peering, 268 active+clean; 582 KiB data, 604 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:38:54.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:53 smithi097 ceph-mon[39416]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:38:54.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:53 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1961013474' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-107-cache", "mode": "writeback"}]': finished 2024-03-21T13:38:54.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:53 smithi097 ceph-mon[39416]: osdmap e543: 8 total, 8 up, 8 in 2024-03-21T13:38:54.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:53 smithi097 ceph-mon[30040]: pgmap v834: 332 pgs: 32 unknown, 32 creating+peering, 268 active+clean; 582 KiB data, 604 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:38:54.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:53 smithi097 ceph-mon[30040]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:38:54.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:53 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1961013474' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-107-cache", "mode": "writeback"}]': finished 2024-03-21T13:38:54.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:53 smithi097 ceph-mon[30040]: osdmap e543: 8 total, 8 up, 8 in 2024-03-21T13:38:54.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:53 smithi144 ceph-mon[33633]: pgmap v834: 332 pgs: 32 unknown, 32 creating+peering, 268 active+clean; 582 KiB data, 604 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:38:54.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:53 smithi144 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:38:54.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:53 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1961013474' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-107-cache", "mode": "writeback"}]': finished 2024-03-21T13:38:54.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:53 smithi144 ceph-mon[33633]: osdmap e543: 8 total, 8 up, 8 in 2024-03-21T13:38:54.371 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:38:53 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:38:55.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:54 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:38:55.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:54 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1961013474' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-107"}]: dispatch 2024-03-21T13:38:55.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:54 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "313.d", "id": [0, 1]}]: dispatch 2024-03-21T13:38:55.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:54 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "313.d", "id": [0, 1]}]: dispatch 2024-03-21T13:38:55.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:54 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:38:55.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:54 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1961013474' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-107"}]: dispatch 2024-03-21T13:38:55.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:54 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "313.d", "id": [0, 1]}]: dispatch 2024-03-21T13:38:55.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:54 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "313.d", "id": [0, 1]}]: dispatch 2024-03-21T13:38:55.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:54 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:38:55.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:54 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1961013474' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-107"}]: dispatch 2024-03-21T13:38:55.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:54 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "313.d", "id": [0, 1]}]: dispatch 2024-03-21T13:38:55.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:54 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "313.d", "id": [0, 1]}]: dispatch 2024-03-21T13:38:56.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:55 smithi097 ceph-mon[30040]: pgmap v837: 332 pgs: 332 active+clean; 582 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:38:56.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:55 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1961013474' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-107"}]': finished 2024-03-21T13:38:56.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:55 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "313.d", "id": [0, 1]}]': finished 2024-03-21T13:38:56.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:55 smithi097 ceph-mon[30040]: osdmap e544: 8 total, 8 up, 8 in 2024-03-21T13:38:56.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:55 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1961013474' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-107", "tierpool": "test-rados-api-smithi097-79906-107-cache"}]: dispatch 2024-03-21T13:38:56.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:55 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:38:56.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:55 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:38:56.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:55 smithi097 ceph-mon[39416]: pgmap v837: 332 pgs: 332 active+clean; 582 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:38:56.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:55 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1961013474' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-107"}]': finished 2024-03-21T13:38:56.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:55 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "313.d", "id": [0, 1]}]': finished 2024-03-21T13:38:56.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:55 smithi097 ceph-mon[39416]: osdmap e544: 8 total, 8 up, 8 in 2024-03-21T13:38:56.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:55 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1961013474' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-107", "tierpool": "test-rados-api-smithi097-79906-107-cache"}]: dispatch 2024-03-21T13:38:56.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:55 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:38:56.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:55 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:38:56.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:55 smithi144 ceph-mon[33633]: pgmap v837: 332 pgs: 332 active+clean; 582 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:38:56.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:55 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1961013474' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-107"}]': finished 2024-03-21T13:38:56.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:55 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "313.d", "id": [0, 1]}]': finished 2024-03-21T13:38:56.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:55 smithi144 ceph-mon[33633]: osdmap e544: 8 total, 8 up, 8 in 2024-03-21T13:38:56.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:55 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1961013474' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-107", "tierpool": "test-rados-api-smithi097-79906-107-cache"}]: dispatch 2024-03-21T13:38:56.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:55 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:38:56.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:55 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:38:57.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:56 smithi097 ceph-mon[30040]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:38:57.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:56 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1961013474' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-107", "tierpool": "test-rados-api-smithi097-79906-107-cache"}]': finished 2024-03-21T13:38:57.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:56 smithi097 ceph-mon[30040]: osdmap e545: 8 total, 8 up, 8 in 2024-03-21T13:38:57.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:56 smithi097 ceph-mon[39416]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:38:57.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:56 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1961013474' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-107", "tierpool": "test-rados-api-smithi097-79906-107-cache"}]': finished 2024-03-21T13:38:57.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:56 smithi097 ceph-mon[39416]: osdmap e545: 8 total, 8 up, 8 in 2024-03-21T13:38:57.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:56 smithi144 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:38:57.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:56 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1961013474' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-107", "tierpool": "test-rados-api-smithi097-79906-107-cache"}]': finished 2024-03-21T13:38:57.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:56 smithi144 ceph-mon[33633]: osdmap e545: 8 total, 8 up, 8 in 2024-03-21T13:38:58.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:57 smithi097 ceph-mon[39416]: pgmap v840: 332 pgs: 332 active+clean; 582 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:38:58.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:57 smithi097 ceph-mon[39416]: osdmap e546: 8 total, 8 up, 8 in 2024-03-21T13:38:58.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:57 smithi097 ceph-mon[39416]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:38:58.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:57 smithi097 ceph-mon[39416]: osdmap e547: 8 total, 8 up, 8 in 2024-03-21T13:38:58.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:57 smithi097 ceph-mon[30040]: pgmap v840: 332 pgs: 332 active+clean; 582 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:38:58.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:57 smithi097 ceph-mon[30040]: osdmap e546: 8 total, 8 up, 8 in 2024-03-21T13:38:58.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:57 smithi097 ceph-mon[30040]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:38:58.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:57 smithi097 ceph-mon[30040]: osdmap e547: 8 total, 8 up, 8 in 2024-03-21T13:38:58.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:57 smithi144 ceph-mon[33633]: pgmap v840: 332 pgs: 332 active+clean; 582 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:38:58.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:57 smithi144 ceph-mon[33633]: osdmap e546: 8 total, 8 up, 8 in 2024-03-21T13:38:58.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:57 smithi144 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:38:58.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:57 smithi144 ceph-mon[33633]: osdmap e547: 8 total, 8 up, 8 in 2024-03-21T13:38:59.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:58 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1245611706' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi097-79906-109"}]: dispatch 2024-03-21T13:38:59.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:58 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi097-79906-109"}]: dispatch 2024-03-21T13:38:59.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:58 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1245611706' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi097-79906-109"}]: dispatch 2024-03-21T13:38:59.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:58 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi097-79906-109"}]: dispatch 2024-03-21T13:38:59.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:58 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1245611706' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi097-79906-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:38:59.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:38:58 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi097-79906-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:38:59.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:58 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1245611706' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi097-79906-109"}]: dispatch 2024-03-21T13:38:59.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:58 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi097-79906-109"}]: dispatch 2024-03-21T13:38:59.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:58 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1245611706' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi097-79906-109"}]: dispatch 2024-03-21T13:38:59.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:58 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi097-79906-109"}]: dispatch 2024-03-21T13:38:59.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:58 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1245611706' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi097-79906-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:38:59.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:38:58 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi097-79906-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:38:59.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:58 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1245611706' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi097-79906-109"}]: dispatch 2024-03-21T13:38:59.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:58 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi097-79906-109"}]: dispatch 2024-03-21T13:38:59.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:58 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1245611706' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi097-79906-109"}]: dispatch 2024-03-21T13:38:59.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:58 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi097-79906-109"}]: dispatch 2024-03-21T13:38:59.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:58 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1245611706' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi097-79906-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:38:59.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:38:58 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi097-79906-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:39:00.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:00 smithi097 ceph-mon[39416]: pgmap v843: 268 pgs: 268 active+clean; 582 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:39:00.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:00 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi097-79906-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:39:00.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:00 smithi097 ceph-mon[39416]: osdmap e548: 8 total, 8 up, 8 in 2024-03-21T13:39:00.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:00 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1245611706' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi097-79906-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi097-79906-109"}]: dispatch 2024-03-21T13:39:00.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:00 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi097-79906-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi097-79906-109"}]: dispatch 2024-03-21T13:39:00.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:00 smithi097 ceph-mon[30040]: pgmap v843: 268 pgs: 268 active+clean; 582 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:39:00.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:00 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi097-79906-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:39:00.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:00 smithi097 ceph-mon[30040]: osdmap e548: 8 total, 8 up, 8 in 2024-03-21T13:39:00.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:00 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1245611706' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi097-79906-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi097-79906-109"}]: dispatch 2024-03-21T13:39:00.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:00 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi097-79906-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi097-79906-109"}]: dispatch 2024-03-21T13:39:00.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:00 smithi144 ceph-mon[33633]: pgmap v843: 268 pgs: 268 active+clean; 582 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:39:00.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:00 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi097-79906-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:39:00.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:00 smithi144 ceph-mon[33633]: osdmap e548: 8 total, 8 up, 8 in 2024-03-21T13:39:00.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:00 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1245611706' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi097-79906-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi097-79906-109"}]: dispatch 2024-03-21T13:39:00.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:00 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi097-79906-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi097-79906-109"}]: dispatch 2024-03-21T13:39:01.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:01 smithi097 ceph-mon[39416]: osdmap e549: 8 total, 8 up, 8 in 2024-03-21T13:39:01.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:01 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi097-79906-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi097-79906-109"}]': finished 2024-03-21T13:39:01.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:01 smithi097 ceph-mon[39416]: osdmap e550: 8 total, 8 up, 8 in 2024-03-21T13:39:01.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:01 smithi097 ceph-mon[30040]: osdmap e549: 8 total, 8 up, 8 in 2024-03-21T13:39:01.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:01 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi097-79906-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi097-79906-109"}]': finished 2024-03-21T13:39:01.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:01 smithi097 ceph-mon[30040]: osdmap e550: 8 total, 8 up, 8 in 2024-03-21T13:39:01.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:01 smithi144 ceph-mon[33633]: osdmap e549: 8 total, 8 up, 8 in 2024-03-21T13:39:01.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:01 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi097-79906-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi097-79906-109"}]': finished 2024-03-21T13:39:01.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:01 smithi144 ceph-mon[33633]: osdmap e550: 8 total, 8 up, 8 in 2024-03-21T13:39:02.243 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:39:01 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:39:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:39:02.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:02 smithi097 ceph-mon[39416]: pgmap v846: 268 pgs: 268 active+clean; 582 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:39:02.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:02 smithi097 ceph-mon[39416]: osdmap e551: 8 total, 8 up, 8 in 2024-03-21T13:39:02.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:02 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1245611706' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:39:02.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:02 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:39:02.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:02 smithi097 ceph-mon[30040]: pgmap v846: 268 pgs: 268 active+clean; 582 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:39:02.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:02 smithi097 ceph-mon[30040]: osdmap e551: 8 total, 8 up, 8 in 2024-03-21T13:39:02.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:02 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1245611706' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:39:02.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:02 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:39:02.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:02 smithi144 ceph-mon[33633]: pgmap v846: 268 pgs: 268 active+clean; 582 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:39:02.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:02 smithi144 ceph-mon[33633]: osdmap e551: 8 total, 8 up, 8 in 2024-03-21T13:39:02.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:02 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1245611706' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:39:02.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:02 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:39:04.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:04 smithi097 ceph-mon[30040]: pgmap v849: 308 pgs: 40 unknown, 268 active+clean; 582 KiB data, 605 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:39:04.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:04 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:39:04.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:04 smithi097 ceph-mon[30040]: osdmap e552: 8 total, 8 up, 8 in 2024-03-21T13:39:04.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:04 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1245611706' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-109", "tierpool": "test-rados-api-smithi097-79906-109-cache"}]: dispatch 2024-03-21T13:39:04.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:04 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-109", "tierpool": "test-rados-api-smithi097-79906-109-cache"}]: dispatch 2024-03-21T13:39:04.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:04 smithi097 ceph-mon[30040]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:39:04.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:04 smithi097 ceph-mon[39416]: pgmap v849: 308 pgs: 40 unknown, 268 active+clean; 582 KiB data, 605 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:39:04.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:04 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:39:04.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:04 smithi097 ceph-mon[39416]: osdmap e552: 8 total, 8 up, 8 in 2024-03-21T13:39:04.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:04 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1245611706' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-109", "tierpool": "test-rados-api-smithi097-79906-109-cache"}]: dispatch 2024-03-21T13:39:04.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:04 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-109", "tierpool": "test-rados-api-smithi097-79906-109-cache"}]: dispatch 2024-03-21T13:39:04.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:04 smithi097 ceph-mon[39416]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:39:04.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:04 smithi144 ceph-mon[33633]: pgmap v849: 308 pgs: 40 unknown, 268 active+clean; 582 KiB data, 605 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:39:04.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:04 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:39:04.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:04 smithi144 ceph-mon[33633]: osdmap e552: 8 total, 8 up, 8 in 2024-03-21T13:39:04.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:04 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1245611706' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-109", "tierpool": "test-rados-api-smithi097-79906-109-cache"}]: dispatch 2024-03-21T13:39:04.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:04 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-109", "tierpool": "test-rados-api-smithi097-79906-109-cache"}]: dispatch 2024-03-21T13:39:04.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:04 smithi144 ceph-mon[33633]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:39:04.372 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:39:03 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:39:05.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:05 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:39:05.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:05 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-109", "tierpool": "test-rados-api-smithi097-79906-109-cache"}]': finished 2024-03-21T13:39:05.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:05 smithi097 ceph-mon[39416]: osdmap e553: 8 total, 8 up, 8 in 2024-03-21T13:39:05.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:05 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1245611706' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-109", "overlaypool": "test-rados-api-smithi097-79906-109-cache"}]: dispatch 2024-03-21T13:39:05.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:05 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-109", "overlaypool": "test-rados-api-smithi097-79906-109-cache"}]: dispatch 2024-03-21T13:39:05.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:05 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-109", "overlaypool": "test-rados-api-smithi097-79906-109-cache"}]': finished 2024-03-21T13:39:05.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:05 smithi097 ceph-mon[39416]: osdmap e554: 8 total, 8 up, 8 in 2024-03-21T13:39:05.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:05 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1245611706' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-109-cache", "mode": "writeback"}]: dispatch 2024-03-21T13:39:05.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:05 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-109-cache", "mode": "writeback"}]: dispatch 2024-03-21T13:39:05.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:05 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:39:05.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:05 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-109", "tierpool": "test-rados-api-smithi097-79906-109-cache"}]': finished 2024-03-21T13:39:05.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:05 smithi097 ceph-mon[30040]: osdmap e553: 8 total, 8 up, 8 in 2024-03-21T13:39:05.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:05 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1245611706' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-109", "overlaypool": "test-rados-api-smithi097-79906-109-cache"}]: dispatch 2024-03-21T13:39:05.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:05 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-109", "overlaypool": "test-rados-api-smithi097-79906-109-cache"}]: dispatch 2024-03-21T13:39:05.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:05 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-109", "overlaypool": "test-rados-api-smithi097-79906-109-cache"}]': finished 2024-03-21T13:39:05.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:05 smithi097 ceph-mon[30040]: osdmap e554: 8 total, 8 up, 8 in 2024-03-21T13:39:05.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:05 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1245611706' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-109-cache", "mode": "writeback"}]: dispatch 2024-03-21T13:39:05.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:05 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-109-cache", "mode": "writeback"}]: dispatch 2024-03-21T13:39:05.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:05 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:39:05.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:05 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-109", "tierpool": "test-rados-api-smithi097-79906-109-cache"}]': finished 2024-03-21T13:39:05.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:05 smithi144 ceph-mon[33633]: osdmap e553: 8 total, 8 up, 8 in 2024-03-21T13:39:05.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:05 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1245611706' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-109", "overlaypool": "test-rados-api-smithi097-79906-109-cache"}]: dispatch 2024-03-21T13:39:05.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:05 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-109", "overlaypool": "test-rados-api-smithi097-79906-109-cache"}]: dispatch 2024-03-21T13:39:05.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:05 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-109", "overlaypool": "test-rados-api-smithi097-79906-109-cache"}]': finished 2024-03-21T13:39:05.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:05 smithi144 ceph-mon[33633]: osdmap e554: 8 total, 8 up, 8 in 2024-03-21T13:39:05.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:05 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1245611706' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-109-cache", "mode": "writeback"}]: dispatch 2024-03-21T13:39:05.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:05 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-109-cache", "mode": "writeback"}]: dispatch 2024-03-21T13:39:06.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:06 smithi097 ceph-mon[30040]: pgmap v852: 308 pgs: 308 active+clean; 582 KiB data, 606 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:39:06.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:06 smithi097 ceph-mon[30040]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:39:06.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:06 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-109-cache", "mode": "writeback"}]': finished 2024-03-21T13:39:06.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:06 smithi097 ceph-mon[30040]: osdmap e555: 8 total, 8 up, 8 in 2024-03-21T13:39:06.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:06 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1245611706' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:39:06.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:06 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:39:06.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:06 smithi097 ceph-mon[39416]: pgmap v852: 308 pgs: 308 active+clean; 582 KiB data, 606 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:39:06.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:06 smithi097 ceph-mon[39416]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:39:06.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:06 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-109-cache", "mode": "writeback"}]': finished 2024-03-21T13:39:06.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:06 smithi097 ceph-mon[39416]: osdmap e555: 8 total, 8 up, 8 in 2024-03-21T13:39:06.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:06 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1245611706' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:39:06.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:06 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:39:06.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:06 smithi144 ceph-mon[33633]: pgmap v852: 308 pgs: 308 active+clean; 582 KiB data, 606 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:39:06.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:06 smithi144 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:39:06.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:06 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-109-cache", "mode": "writeback"}]': finished 2024-03-21T13:39:06.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:06 smithi144 ceph-mon[33633]: osdmap e555: 8 total, 8 up, 8 in 2024-03-21T13:39:06.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:06 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1245611706' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:39:06.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:06 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:39:08.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:08 smithi097 ceph-mon[30040]: pgmap v855: 308 pgs: 308 active+clean; 582 KiB data, 606 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:39:08.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:08 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-03-21T13:39:08.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:08 smithi097 ceph-mon[30040]: osdmap e556: 8 total, 8 up, 8 in 2024-03-21T13:39:08.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:08 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1245611706' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:39:08.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:08 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:39:08.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:08 smithi097 ceph-mon[39416]: pgmap v855: 308 pgs: 308 active+clean; 582 KiB data, 606 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:39:08.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:08 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-03-21T13:39:08.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:08 smithi097 ceph-mon[39416]: osdmap e556: 8 total, 8 up, 8 in 2024-03-21T13:39:08.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:08 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1245611706' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:39:08.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:08 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:39:08.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:08 smithi144 ceph-mon[33633]: pgmap v855: 308 pgs: 308 active+clean; 582 KiB data, 606 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:39:08.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:08 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-03-21T13:39:08.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:08 smithi144 ceph-mon[33633]: osdmap e556: 8 total, 8 up, 8 in 2024-03-21T13:39:08.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:08 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1245611706' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:39:08.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:08 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:39:09.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:09 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-03-21T13:39:09.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:09 smithi097 ceph-mon[30040]: osdmap e557: 8 total, 8 up, 8 in 2024-03-21T13:39:09.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:09 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1245611706' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-21T13:39:09.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:09 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-21T13:39:09.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:09 smithi097 ceph-mon[30040]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:39:09.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:09 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-21T13:39:09.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:09 smithi097 ceph-mon[30040]: osdmap e558: 8 total, 8 up, 8 in 2024-03-21T13:39:09.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:09 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-03-21T13:39:09.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:09 smithi097 ceph-mon[39416]: osdmap e557: 8 total, 8 up, 8 in 2024-03-21T13:39:09.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:09 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1245611706' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-21T13:39:09.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:09 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-21T13:39:09.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:09 smithi097 ceph-mon[39416]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:39:09.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:09 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-21T13:39:09.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:09 smithi097 ceph-mon[39416]: osdmap e558: 8 total, 8 up, 8 in 2024-03-21T13:39:09.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:09 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-03-21T13:39:09.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:09 smithi144 ceph-mon[33633]: osdmap e557: 8 total, 8 up, 8 in 2024-03-21T13:39:09.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:09 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1245611706' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-21T13:39:09.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:09 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-21T13:39:09.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:09 smithi144 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:39:09.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:09 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-21T13:39:09.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:09 smithi144 ceph-mon[33633]: osdmap e558: 8 total, 8 up, 8 in 2024-03-21T13:39:10.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:10 smithi097 ceph-mon[30040]: pgmap v858: 308 pgs: 308 active+clean; 582 KiB data, 607 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:39:10.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:10 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1245611706' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-03-21T13:39:10.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:10 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-03-21T13:39:10.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:10 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-03-21T13:39:10.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:10 smithi097 ceph-mon[30040]: osdmap e559: 8 total, 8 up, 8 in 2024-03-21T13:39:10.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:10 smithi097 ceph-mon[39416]: pgmap v858: 308 pgs: 308 active+clean; 582 KiB data, 607 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:39:10.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:10 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1245611706' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-03-21T13:39:10.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:10 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-03-21T13:39:10.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:10 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-03-21T13:39:10.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:10 smithi097 ceph-mon[39416]: osdmap e559: 8 total, 8 up, 8 in 2024-03-21T13:39:10.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:10 smithi144 ceph-mon[33633]: pgmap v858: 308 pgs: 308 active+clean; 582 KiB data, 607 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:39:10.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:10 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1245611706' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-03-21T13:39:10.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:10 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-03-21T13:39:10.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:10 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-03-21T13:39:10.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:10 smithi144 ceph-mon[33633]: osdmap e559: 8 total, 8 up, 8 in 2024-03-21T13:39:11.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:11 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1245611706' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-109"}]: dispatch 2024-03-21T13:39:11.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:11 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-109"}]: dispatch 2024-03-21T13:39:11.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:11 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:39:11.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:11 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:39:11.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:11 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1245611706' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-109"}]: dispatch 2024-03-21T13:39:11.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:11 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-109"}]: dispatch 2024-03-21T13:39:11.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:11 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:39:11.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:11 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:39:11.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:11 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1245611706' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-109"}]: dispatch 2024-03-21T13:39:11.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:11 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-109"}]: dispatch 2024-03-21T13:39:11.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:11 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:39:11.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:11 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:39:12.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:39:11 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:39:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:39:12.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:12 smithi097 ceph-mon[30040]: pgmap v861: 308 pgs: 308 active+clean; 582 KiB data, 607 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:39:12.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:12 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-109"}]': finished 2024-03-21T13:39:12.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:12 smithi097 ceph-mon[30040]: osdmap e560: 8 total, 8 up, 8 in 2024-03-21T13:39:12.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:12 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1245611706' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-109", "tierpool": "test-rados-api-smithi097-79906-109-cache"}]: dispatch 2024-03-21T13:39:12.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:12 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-109", "tierpool": "test-rados-api-smithi097-79906-109-cache"}]: dispatch 2024-03-21T13:39:12.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:12 smithi097 ceph-mon[39416]: pgmap v861: 308 pgs: 308 active+clean; 582 KiB data, 607 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:39:12.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:12 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-109"}]': finished 2024-03-21T13:39:12.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:12 smithi097 ceph-mon[39416]: osdmap e560: 8 total, 8 up, 8 in 2024-03-21T13:39:12.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:12 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1245611706' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-109", "tierpool": "test-rados-api-smithi097-79906-109-cache"}]: dispatch 2024-03-21T13:39:12.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:12 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-109", "tierpool": "test-rados-api-smithi097-79906-109-cache"}]: dispatch 2024-03-21T13:39:12.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:12 smithi144 ceph-mon[33633]: pgmap v861: 308 pgs: 308 active+clean; 582 KiB data, 607 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:39:12.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:12 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-109"}]': finished 2024-03-21T13:39:12.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:12 smithi144 ceph-mon[33633]: osdmap e560: 8 total, 8 up, 8 in 2024-03-21T13:39:12.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:12 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1245611706' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-109", "tierpool": "test-rados-api-smithi097-79906-109-cache"}]: dispatch 2024-03-21T13:39:12.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:12 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-109", "tierpool": "test-rados-api-smithi097-79906-109-cache"}]: dispatch 2024-03-21T13:39:13.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:13 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-109", "tierpool": "test-rados-api-smithi097-79906-109-cache"}]': finished 2024-03-21T13:39:13.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:13 smithi097 ceph-mon[39416]: osdmap e561: 8 total, 8 up, 8 in 2024-03-21T13:39:13.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:13 smithi097 ceph-mon[39416]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:39:13.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:13 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-109", "tierpool": "test-rados-api-smithi097-79906-109-cache"}]': finished 2024-03-21T13:39:13.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:13 smithi097 ceph-mon[30040]: osdmap e561: 8 total, 8 up, 8 in 2024-03-21T13:39:13.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:13 smithi097 ceph-mon[30040]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:39:13.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:13 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-109", "tierpool": "test-rados-api-smithi097-79906-109-cache"}]': finished 2024-03-21T13:39:13.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:13 smithi144 ceph-mon[33633]: osdmap e561: 8 total, 8 up, 8 in 2024-03-21T13:39:13.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:13 smithi144 ceph-mon[33633]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:39:14.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:14 smithi144 ceph-mon[33633]: pgmap v864: 308 pgs: 308 active+clean; 582 KiB data, 607 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:39:14.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:14 smithi144 ceph-mon[33633]: osdmap e562: 8 total, 8 up, 8 in 2024-03-21T13:39:14.371 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:39:13 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:39:14.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:14 smithi097 ceph-mon[39416]: pgmap v864: 308 pgs: 308 active+clean; 582 KiB data, 607 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:39:14.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:14 smithi097 ceph-mon[39416]: osdmap e562: 8 total, 8 up, 8 in 2024-03-21T13:39:14.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:14 smithi097 ceph-mon[30040]: pgmap v864: 308 pgs: 308 active+clean; 582 KiB data, 607 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:39:14.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:14 smithi097 ceph-mon[30040]: osdmap e562: 8 total, 8 up, 8 in 2024-03-21T13:39:15.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:15 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:39:15.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:15 smithi144 ceph-mon[33633]: osdmap e563: 8 total, 8 up, 8 in 2024-03-21T13:39:15.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:15 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1245611706' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi097-79906-109"}]: dispatch 2024-03-21T13:39:15.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:15 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi097-79906-109"}]: dispatch 2024-03-21T13:39:15.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:15 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:39:15.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:15 smithi097 ceph-mon[39416]: osdmap e563: 8 total, 8 up, 8 in 2024-03-21T13:39:15.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:15 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1245611706' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi097-79906-109"}]: dispatch 2024-03-21T13:39:15.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:15 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi097-79906-109"}]: dispatch 2024-03-21T13:39:15.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:15 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:39:15.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:15 smithi097 ceph-mon[30040]: osdmap e563: 8 total, 8 up, 8 in 2024-03-21T13:39:15.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:15 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1245611706' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi097-79906-109"}]: dispatch 2024-03-21T13:39:15.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:15 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi097-79906-109"}]: dispatch 2024-03-21T13:39:16.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:16 smithi144 ceph-mon[33633]: pgmap v867: 268 pgs: 268 active+clean; 582 KiB data, 607 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:39:16.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:16 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi097-79906-109"}]': finished 2024-03-21T13:39:16.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:16 smithi144 ceph-mon[33633]: osdmap e564: 8 total, 8 up, 8 in 2024-03-21T13:39:16.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:16 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1245611706' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi097-79906-109"}]: dispatch 2024-03-21T13:39:16.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:16 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi097-79906-109"}]: dispatch 2024-03-21T13:39:16.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:16 smithi097 ceph-mon[30040]: pgmap v867: 268 pgs: 268 active+clean; 582 KiB data, 607 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:39:16.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:16 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi097-79906-109"}]': finished 2024-03-21T13:39:16.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:16 smithi097 ceph-mon[30040]: osdmap e564: 8 total, 8 up, 8 in 2024-03-21T13:39:16.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:16 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1245611706' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi097-79906-109"}]: dispatch 2024-03-21T13:39:16.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:16 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi097-79906-109"}]: dispatch 2024-03-21T13:39:16.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:16 smithi097 ceph-mon[39416]: pgmap v867: 268 pgs: 268 active+clean; 582 KiB data, 607 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:39:16.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:16 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi097-79906-109"}]': finished 2024-03-21T13:39:16.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:16 smithi097 ceph-mon[39416]: osdmap e564: 8 total, 8 up, 8 in 2024-03-21T13:39:16.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:16 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1245611706' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi097-79906-109"}]: dispatch 2024-03-21T13:39:16.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:16 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi097-79906-109"}]: dispatch 2024-03-21T13:39:17.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:17 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi097-79906-109"}]': finished 2024-03-21T13:39:17.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:17 smithi144 ceph-mon[33633]: osdmap e565: 8 total, 8 up, 8 in 2024-03-21T13:39:17.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:17 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi097-79906-109"}]': finished 2024-03-21T13:39:17.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:17 smithi097 ceph-mon[39416]: osdmap e565: 8 total, 8 up, 8 in 2024-03-21T13:39:17.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:17 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi097-79906-109"}]': finished 2024-03-21T13:39:17.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:17 smithi097 ceph-mon[30040]: osdmap e565: 8 total, 8 up, 8 in 2024-03-21T13:39:18.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:18 smithi144 ceph-mon[33633]: pgmap v870: 268 pgs: 268 active+clean; 582 KiB data, 607 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:39:18.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:18 smithi144 ceph-mon[33633]: osdmap e566: 8 total, 8 up, 8 in 2024-03-21T13:39:18.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:18 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/553707059' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi097-79906-104"}]: dispatch 2024-03-21T13:39:18.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:18 smithi144 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:39:18.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:18 smithi097 ceph-mon[39416]: pgmap v870: 268 pgs: 268 active+clean; 582 KiB data, 607 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:39:18.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:18 smithi097 ceph-mon[39416]: osdmap e566: 8 total, 8 up, 8 in 2024-03-21T13:39:18.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:18 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/553707059' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi097-79906-104"}]: dispatch 2024-03-21T13:39:18.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:18 smithi097 ceph-mon[39416]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:39:18.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:18 smithi097 ceph-mon[30040]: pgmap v870: 268 pgs: 268 active+clean; 582 KiB data, 607 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:39:18.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:18 smithi097 ceph-mon[30040]: osdmap e566: 8 total, 8 up, 8 in 2024-03-21T13:39:18.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:18 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/553707059' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi097-79906-104"}]: dispatch 2024-03-21T13:39:18.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:18 smithi097 ceph-mon[30040]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:39:19.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:19 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/553707059' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi097-79906-104"}]': finished 2024-03-21T13:39:19.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:19 smithi144 ceph-mon[33633]: osdmap e567: 8 total, 8 up, 8 in 2024-03-21T13:39:19.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:19 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/553707059' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi097-79906-104"}]: dispatch 2024-03-21T13:39:19.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:19 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/553707059' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi097-79906-104"}]': finished 2024-03-21T13:39:19.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:19 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/553707059' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi097-79906-104"}]': finished 2024-03-21T13:39:19.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:19 smithi097 ceph-mon[39416]: osdmap e567: 8 total, 8 up, 8 in 2024-03-21T13:39:19.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:19 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/553707059' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi097-79906-104"}]: dispatch 2024-03-21T13:39:19.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:19 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/553707059' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi097-79906-104"}]': finished 2024-03-21T13:39:19.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:19 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/553707059' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi097-79906-104"}]': finished 2024-03-21T13:39:19.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:19 smithi097 ceph-mon[30040]: osdmap e567: 8 total, 8 up, 8 in 2024-03-21T13:39:19.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:19 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/553707059' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi097-79906-104"}]: dispatch 2024-03-21T13:39:19.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:19 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/553707059' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi097-79906-104"}]': finished 2024-03-21T13:39:20.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:20 smithi144 ceph-mon[33633]: pgmap v873: 260 pgs: 260 active+clean; 582 KiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-21T13:39:20.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:20 smithi144 ceph-mon[33633]: osdmap e568: 8 total, 8 up, 8 in 2024-03-21T13:39:20.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:20 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:39:20.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:20 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:39:20.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:20 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi097-79906-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:39:20.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:20 smithi097 ceph-mon[39416]: pgmap v873: 260 pgs: 260 active+clean; 582 KiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-21T13:39:20.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:20 smithi097 ceph-mon[39416]: osdmap e568: 8 total, 8 up, 8 in 2024-03-21T13:39:20.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:20 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:39:20.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:20 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:39:20.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:20 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi097-79906-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:39:20.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:20 smithi097 ceph-mon[30040]: pgmap v873: 260 pgs: 260 active+clean; 582 KiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-21T13:39:20.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:20 smithi097 ceph-mon[30040]: osdmap e568: 8 total, 8 up, 8 in 2024-03-21T13:39:20.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:20 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:39:20.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:20 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:39:20.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:20 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi097-79906-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:39:21.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:21 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi097-79906-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:39:21.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:21 smithi144 ceph-mon[33633]: osdmap e569: 8 total, 8 up, 8 in 2024-03-21T13:39:21.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:21 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi097-79906-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:39:21.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:21 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi097-79906-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:39:21.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:21 smithi097 ceph-mon[39416]: osdmap e569: 8 total, 8 up, 8 in 2024-03-21T13:39:21.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:21 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi097-79906-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:39:21.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:21 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi097-79906-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:39:21.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:21 smithi097 ceph-mon[30040]: osdmap e569: 8 total, 8 up, 8 in 2024-03-21T13:39:21.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:21 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi097-79906-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:39:22.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:39:21 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:39:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:39:22.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:22 smithi097 ceph-mon[39416]: pgmap v876: 260 pgs: 260 active+clean; 582 KiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-21T13:39:22.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:22 smithi097 ceph-mon[39416]: osdmap e570: 8 total, 8 up, 8 in 2024-03-21T13:39:22.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:22 smithi097 ceph-mon[30040]: pgmap v876: 260 pgs: 260 active+clean; 582 KiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-21T13:39:22.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:22 smithi097 ceph-mon[30040]: osdmap e570: 8 total, 8 up, 8 in 2024-03-21T13:39:22.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:22 smithi144 ceph-mon[33633]: pgmap v876: 260 pgs: 260 active+clean; 582 KiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-21T13:39:22.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:22 smithi144 ceph-mon[33633]: osdmap e570: 8 total, 8 up, 8 in 2024-03-21T13:39:23.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:23 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi097-79906-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:39:23.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:23 smithi097 ceph-mon[39416]: osdmap e571: 8 total, 8 up, 8 in 2024-03-21T13:39:23.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:23 smithi097 ceph-mon[39416]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:39:23.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:23 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi097-79906-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:39:23.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:23 smithi097 ceph-mon[30040]: osdmap e571: 8 total, 8 up, 8 in 2024-03-21T13:39:23.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:23 smithi097 ceph-mon[30040]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:39:23.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:23 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi097-79906-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:39:23.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:23 smithi144 ceph-mon[33633]: osdmap e571: 8 total, 8 up, 8 in 2024-03-21T13:39:23.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:23 smithi144 ceph-mon[33633]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:39:24.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:24 smithi144 ceph-mon[33633]: pgmap v879: 268 pgs: 8 unknown, 260 active+clean; 582 KiB data, 608 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:39:24.373 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:24 smithi144 ceph-mon[33633]: osdmap e572: 8 total, 8 up, 8 in 2024-03-21T13:39:24.373 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:24 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:39:24.373 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:39:23 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:39:24.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:24 smithi097 ceph-mon[39416]: pgmap v879: 268 pgs: 8 unknown, 260 active+clean; 582 KiB data, 608 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:39:24.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:24 smithi097 ceph-mon[39416]: osdmap e572: 8 total, 8 up, 8 in 2024-03-21T13:39:24.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:24 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:39:24.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:24 smithi097 ceph-mon[30040]: pgmap v879: 268 pgs: 8 unknown, 260 active+clean; 582 KiB data, 608 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:39:24.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:24 smithi097 ceph-mon[30040]: osdmap e572: 8 total, 8 up, 8 in 2024-03-21T13:39:24.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:24 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:39:25.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:25 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:39:25.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:25 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:39:25.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:25 smithi097 ceph-mon[39416]: osdmap e573: 8 total, 8 up, 8 in 2024-03-21T13:39:25.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:25 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:39:25.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:25 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:39:25.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:25 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:39:25.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:25 smithi097 ceph-mon[30040]: osdmap e573: 8 total, 8 up, 8 in 2024-03-21T13:39:25.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:25 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:39:25.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:25 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:39:25.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:25 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:39:25.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:25 smithi144 ceph-mon[33633]: osdmap e573: 8 total, 8 up, 8 in 2024-03-21T13:39:25.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:25 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:39:26.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:26 smithi097 ceph-mon[39416]: pgmap v882: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:39:26.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:26 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-112", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:39:26.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:26 smithi097 ceph-mon[39416]: osdmap e574: 8 total, 8 up, 8 in 2024-03-21T13:39:26.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:26 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-112"}]: dispatch 2024-03-21T13:39:26.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:26 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:39:26.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:26 smithi097 ceph-mon[30040]: pgmap v882: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:39:26.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:26 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-112", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:39:26.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:26 smithi097 ceph-mon[30040]: osdmap e574: 8 total, 8 up, 8 in 2024-03-21T13:39:26.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:26 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-112"}]: dispatch 2024-03-21T13:39:26.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:26 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:39:26.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:26 smithi144 ceph-mon[33633]: pgmap v882: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:39:26.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:26 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-112", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:39:26.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:26 smithi144 ceph-mon[33633]: osdmap e574: 8 total, 8 up, 8 in 2024-03-21T13:39:26.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:26 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-112"}]: dispatch 2024-03-21T13:39:26.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:26 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:39:27.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:27 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-112"}]': finished 2024-03-21T13:39:27.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:27 smithi097 ceph-mon[39416]: osdmap e575: 8 total, 8 up, 8 in 2024-03-21T13:39:27.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:27 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:39:27.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:27 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-112"}]': finished 2024-03-21T13:39:27.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:27 smithi097 ceph-mon[30040]: osdmap e575: 8 total, 8 up, 8 in 2024-03-21T13:39:27.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:27 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:39:27.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:27 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-112"}]': finished 2024-03-21T13:39:27.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:27 smithi144 ceph-mon[33633]: osdmap e575: 8 total, 8 up, 8 in 2024-03-21T13:39:27.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:27 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:39:28.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:28 smithi097 ceph-mon[39416]: pgmap v885: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:39:28.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:28 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:39:28.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:28 smithi097 ceph-mon[39416]: osdmap e576: 8 total, 8 up, 8 in 2024-03-21T13:39:28.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:28 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-112"}]: dispatch 2024-03-21T13:39:28.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:28 smithi097 ceph-mon[39416]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:39:28.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:28 smithi097 ceph-mon[30040]: pgmap v885: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:39:28.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:28 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:39:28.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:28 smithi097 ceph-mon[30040]: osdmap e576: 8 total, 8 up, 8 in 2024-03-21T13:39:28.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:28 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-112"}]: dispatch 2024-03-21T13:39:28.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:28 smithi097 ceph-mon[30040]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:39:28.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:28 smithi144 ceph-mon[33633]: pgmap v885: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:39:28.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:28 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:39:28.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:28 smithi144 ceph-mon[33633]: osdmap e576: 8 total, 8 up, 8 in 2024-03-21T13:39:28.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:28 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-112"}]: dispatch 2024-03-21T13:39:28.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:28 smithi144 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:39:29.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:29 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-112"}]': finished 2024-03-21T13:39:29.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:29 smithi097 ceph-mon[39416]: osdmap e577: 8 total, 8 up, 8 in 2024-03-21T13:39:29.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:29 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-112"}]': finished 2024-03-21T13:39:29.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:29 smithi097 ceph-mon[30040]: osdmap e577: 8 total, 8 up, 8 in 2024-03-21T13:39:29.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:29 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-112"}]': finished 2024-03-21T13:39:29.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:29 smithi144 ceph-mon[33633]: osdmap e577: 8 total, 8 up, 8 in 2024-03-21T13:39:30.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:30 smithi097 ceph-mon[39416]: pgmap v888: 300 pgs: 300 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-21T13:39:30.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:30 smithi097 ceph-mon[39416]: osdmap e578: 8 total, 8 up, 8 in 2024-03-21T13:39:30.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:30 smithi097 ceph-mon[30040]: pgmap v888: 300 pgs: 300 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-21T13:39:30.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:30 smithi097 ceph-mon[30040]: osdmap e578: 8 total, 8 up, 8 in 2024-03-21T13:39:30.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:30 smithi144 ceph-mon[33633]: pgmap v888: 300 pgs: 300 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-21T13:39:30.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:30 smithi144 ceph-mon[33633]: osdmap e578: 8 total, 8 up, 8 in 2024-03-21T13:39:31.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:31 smithi097 ceph-mon[39416]: osdmap e579: 8 total, 8 up, 8 in 2024-03-21T13:39:31.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:31 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:39:31.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:31 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:39:31.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:31 smithi097 ceph-mon[39416]: osdmap e580: 8 total, 8 up, 8 in 2024-03-21T13:39:31.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:31 smithi097 ceph-mon[30040]: osdmap e579: 8 total, 8 up, 8 in 2024-03-21T13:39:31.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:31 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:39:31.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:31 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:39:31.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:31 smithi097 ceph-mon[30040]: osdmap e580: 8 total, 8 up, 8 in 2024-03-21T13:39:31.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:31 smithi144 ceph-mon[33633]: osdmap e579: 8 total, 8 up, 8 in 2024-03-21T13:39:31.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:31 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:39:31.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:31 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:39:31.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:31 smithi144 ceph-mon[33633]: osdmap e580: 8 total, 8 up, 8 in 2024-03-21T13:39:32.208 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:39:31 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:39:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:39:32.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:32 smithi097 ceph-mon[39416]: pgmap v891: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:39:32.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:32 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:39:32.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:32 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-114", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:39:32.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:32 smithi097 ceph-mon[39416]: osdmap e581: 8 total, 8 up, 8 in 2024-03-21T13:39:32.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:32 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-114"}]: dispatch 2024-03-21T13:39:32.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:32 smithi097 ceph-mon[30040]: pgmap v891: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:39:32.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:32 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:39:32.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:32 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-114", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:39:32.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:32 smithi097 ceph-mon[30040]: osdmap e581: 8 total, 8 up, 8 in 2024-03-21T13:39:32.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:32 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-114"}]: dispatch 2024-03-21T13:39:32.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:32 smithi144 ceph-mon[33633]: pgmap v891: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:39:32.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:32 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:39:32.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:32 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-114", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:39:32.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:32 smithi144 ceph-mon[33633]: osdmap e581: 8 total, 8 up, 8 in 2024-03-21T13:39:32.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:32 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-114"}]: dispatch 2024-03-21T13:39:33.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:33 smithi097 ceph-mon[39416]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:39:33.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:33 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-114"}]': finished 2024-03-21T13:39:33.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:33 smithi097 ceph-mon[39416]: osdmap e582: 8 total, 8 up, 8 in 2024-03-21T13:39:33.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:33 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-114", "mode": "writeback"}]: dispatch 2024-03-21T13:39:33.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:33 smithi097 ceph-mon[30040]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:39:33.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:33 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-114"}]': finished 2024-03-21T13:39:33.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:33 smithi097 ceph-mon[30040]: osdmap e582: 8 total, 8 up, 8 in 2024-03-21T13:39:33.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:33 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-114", "mode": "writeback"}]: dispatch 2024-03-21T13:39:33.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:33 smithi144 ceph-mon[33633]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:39:33.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:33 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-114"}]': finished 2024-03-21T13:39:33.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:33 smithi144 ceph-mon[33633]: osdmap e582: 8 total, 8 up, 8 in 2024-03-21T13:39:33.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:33 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-114", "mode": "writeback"}]: dispatch 2024-03-21T13:39:34.371 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:39:34 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:39:34.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:34 smithi144 ceph-mon[33633]: pgmap v894: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:39:34.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:34 smithi144 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:39:34.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:34 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-114", "mode": "writeback"}]': finished 2024-03-21T13:39:34.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:34 smithi144 ceph-mon[33633]: osdmap e583: 8 total, 8 up, 8 in 2024-03-21T13:39:34.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:34 smithi097 ceph-mon[39416]: pgmap v894: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:39:34.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:34 smithi097 ceph-mon[39416]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:39:34.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:34 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-114", "mode": "writeback"}]': finished 2024-03-21T13:39:34.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:34 smithi097 ceph-mon[39416]: osdmap e583: 8 total, 8 up, 8 in 2024-03-21T13:39:34.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:34 smithi097 ceph-mon[30040]: pgmap v894: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:39:34.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:34 smithi097 ceph-mon[30040]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:39:34.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:34 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-114", "mode": "writeback"}]': finished 2024-03-21T13:39:34.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:34 smithi097 ceph-mon[30040]: osdmap e583: 8 total, 8 up, 8 in 2024-03-21T13:39:35.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:35 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:39:35.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:35 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:39:35.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:35 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:39:35.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:35 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:39:35.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:35 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:39:35.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:35 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:39:36.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:36 smithi097 ceph-mon[30040]: pgmap v897: 300 pgs: 300 active+clean; 582 KiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:39:36.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:36 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:39:36.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:36 smithi097 ceph-mon[30040]: osdmap e584: 8 total, 8 up, 8 in 2024-03-21T13:39:36.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:36 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-114"}]: dispatch 2024-03-21T13:39:36.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:36 smithi097 ceph-mon[39416]: pgmap v897: 300 pgs: 300 active+clean; 582 KiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:39:36.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:36 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:39:36.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:36 smithi097 ceph-mon[39416]: osdmap e584: 8 total, 8 up, 8 in 2024-03-21T13:39:36.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:36 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-114"}]: dispatch 2024-03-21T13:39:36.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:36 smithi144 ceph-mon[33633]: pgmap v897: 300 pgs: 300 active+clean; 582 KiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:39:36.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:36 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:39:36.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:36 smithi144 ceph-mon[33633]: osdmap e584: 8 total, 8 up, 8 in 2024-03-21T13:39:36.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:36 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-114"}]: dispatch 2024-03-21T13:39:37.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:37 smithi097 ceph-mon[39416]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:39:37.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:37 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-114"}]': finished 2024-03-21T13:39:37.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:37 smithi097 ceph-mon[39416]: osdmap e585: 8 total, 8 up, 8 in 2024-03-21T13:39:37.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:37 smithi097 ceph-mon[30040]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:39:37.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:37 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-114"}]': finished 2024-03-21T13:39:37.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:37 smithi097 ceph-mon[30040]: osdmap e585: 8 total, 8 up, 8 in 2024-03-21T13:39:37.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:37 smithi144 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:39:37.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:37 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-114"}]': finished 2024-03-21T13:39:37.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:37 smithi144 ceph-mon[33633]: osdmap e585: 8 total, 8 up, 8 in 2024-03-21T13:39:38.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:38 smithi097 ceph-mon[30040]: pgmap v900: 300 pgs: 300 active+clean; 582 KiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:39:38.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:38 smithi097 ceph-mon[30040]: osdmap e586: 8 total, 8 up, 8 in 2024-03-21T13:39:38.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:38 smithi097 ceph-mon[39416]: pgmap v900: 300 pgs: 300 active+clean; 582 KiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:39:38.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:38 smithi097 ceph-mon[39416]: osdmap e586: 8 total, 8 up, 8 in 2024-03-21T13:39:38.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:38 smithi144 ceph-mon[33633]: pgmap v900: 300 pgs: 300 active+clean; 582 KiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:39:38.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:38 smithi144 ceph-mon[33633]: osdmap e586: 8 total, 8 up, 8 in 2024-03-21T13:39:39.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:39 smithi097 ceph-mon[39416]: osdmap e587: 8 total, 8 up, 8 in 2024-03-21T13:39:39.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:39 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:39:39.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:39 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:39:39.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:39 smithi097 ceph-mon[30040]: osdmap e587: 8 total, 8 up, 8 in 2024-03-21T13:39:39.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:39 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:39:39.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:39 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:39:39.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:39 smithi144 ceph-mon[33633]: osdmap e587: 8 total, 8 up, 8 in 2024-03-21T13:39:39.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:39 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:39:39.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:39 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:39:40.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:40 smithi097 ceph-mon[39416]: pgmap v903: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:39:40.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:40 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:39:40.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:40 smithi097 ceph-mon[39416]: osdmap e588: 8 total, 8 up, 8 in 2024-03-21T13:39:40.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:40 smithi097 ceph-mon[30040]: pgmap v903: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:39:40.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:40 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:39:40.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:40 smithi097 ceph-mon[30040]: osdmap e588: 8 total, 8 up, 8 in 2024-03-21T13:39:40.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:40 smithi144 ceph-mon[33633]: pgmap v903: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:39:40.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:40 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:39:40.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:40 smithi144 ceph-mon[33633]: osdmap e588: 8 total, 8 up, 8 in 2024-03-21T13:39:41.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:41 smithi097 ceph-mon[39416]: osdmap e589: 8 total, 8 up, 8 in 2024-03-21T13:39:41.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:41 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:39:41.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:41 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:39:41.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:41 smithi097 ceph-mon[30040]: osdmap e589: 8 total, 8 up, 8 in 2024-03-21T13:39:41.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:41 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:39:41.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:41 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:39:41.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:41 smithi144 ceph-mon[33633]: osdmap e589: 8 total, 8 up, 8 in 2024-03-21T13:39:41.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:41 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:39:41.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:41 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:39:42.053 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:39:41 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:39:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:39:42.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:42 smithi097 ceph-mon[39416]: pgmap v906: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:39:42.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:42 smithi097 ceph-mon[39416]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:39:42.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:42 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-116", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:39:42.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:42 smithi097 ceph-mon[39416]: osdmap e590: 8 total, 8 up, 8 in 2024-03-21T13:39:42.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:42 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-116"}]: dispatch 2024-03-21T13:39:42.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:42 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:39:42.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:42 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:39:42.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:42 smithi097 ceph-mon[30040]: pgmap v906: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:39:42.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:42 smithi097 ceph-mon[30040]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:39:42.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:42 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-116", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:39:42.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:42 smithi097 ceph-mon[30040]: osdmap e590: 8 total, 8 up, 8 in 2024-03-21T13:39:42.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:42 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-116"}]: dispatch 2024-03-21T13:39:42.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:42 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:39:42.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:42 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:39:42.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:42 smithi144 ceph-mon[33633]: pgmap v906: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:39:42.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:42 smithi144 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:39:42.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:42 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-116", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:39:42.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:42 smithi144 ceph-mon[33633]: osdmap e590: 8 total, 8 up, 8 in 2024-03-21T13:39:42.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:42 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-116"}]: dispatch 2024-03-21T13:39:42.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:42 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:39:42.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:42 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:39:43.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:43 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-116"}]': finished 2024-03-21T13:39:43.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:43 smithi097 ceph-mon[39416]: osdmap e591: 8 total, 8 up, 8 in 2024-03-21T13:39:43.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:43 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-116", "mode": "writeback"}]: dispatch 2024-03-21T13:39:43.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:43 smithi097 ceph-mon[39416]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:39:43.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:43 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-116", "mode": "writeback"}]': finished 2024-03-21T13:39:43.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:43 smithi097 ceph-mon[39416]: osdmap e592: 8 total, 8 up, 8 in 2024-03-21T13:39:43.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:43 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-116"}]': finished 2024-03-21T13:39:43.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:43 smithi097 ceph-mon[30040]: osdmap e591: 8 total, 8 up, 8 in 2024-03-21T13:39:43.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:43 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-116", "mode": "writeback"}]: dispatch 2024-03-21T13:39:43.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:43 smithi097 ceph-mon[30040]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:39:43.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:43 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-116", "mode": "writeback"}]': finished 2024-03-21T13:39:43.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:43 smithi097 ceph-mon[30040]: osdmap e592: 8 total, 8 up, 8 in 2024-03-21T13:39:43.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:43 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-116"}]': finished 2024-03-21T13:39:43.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:43 smithi144 ceph-mon[33633]: osdmap e591: 8 total, 8 up, 8 in 2024-03-21T13:39:43.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:43 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-116", "mode": "writeback"}]: dispatch 2024-03-21T13:39:43.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:43 smithi144 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:39:43.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:43 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-116", "mode": "writeback"}]': finished 2024-03-21T13:39:43.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:43 smithi144 ceph-mon[33633]: osdmap e592: 8 total, 8 up, 8 in 2024-03-21T13:39:44.287 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:39:44 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:39:44.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:44 smithi097 ceph-mon[39416]: pgmap v909: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 627 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:39:44.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:44 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "319.0"}]: dispatch 2024-03-21T13:39:44.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:44 smithi097 ceph-mon[39416]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "319.0"}]: dispatch 2024-03-21T13:39:44.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:44 smithi097 ceph-mon[39416]: 319.0 scrub starts 2024-03-21T13:39:44.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:44 smithi097 ceph-mon[39416]: 319.0 scrub ok 2024-03-21T13:39:44.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:44 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:39:44.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:44 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:39:44.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:44 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:39:44.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:44 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:39:44.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:44 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:39:44.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:44 smithi097 ceph-mon[30040]: pgmap v909: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 627 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:39:44.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:44 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "319.0"}]: dispatch 2024-03-21T13:39:44.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:44 smithi097 ceph-mon[30040]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "319.0"}]: dispatch 2024-03-21T13:39:44.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:44 smithi097 ceph-mon[30040]: 319.0 scrub starts 2024-03-21T13:39:44.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:44 smithi097 ceph-mon[30040]: 319.0 scrub ok 2024-03-21T13:39:44.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:44 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:39:44.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:44 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:39:44.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:44 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:39:44.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:44 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:39:44.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:44 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:39:44.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:44 smithi144 ceph-mon[33633]: pgmap v909: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 627 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:39:44.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:44 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "319.0"}]: dispatch 2024-03-21T13:39:44.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:44 smithi144 ceph-mon[33633]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "319.0"}]: dispatch 2024-03-21T13:39:44.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:44 smithi144 ceph-mon[33633]: 319.0 scrub starts 2024-03-21T13:39:44.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:44 smithi144 ceph-mon[33633]: 319.0 scrub ok 2024-03-21T13:39:44.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:44 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:39:44.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:44 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:39:44.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:44 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:39:44.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:44 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:39:44.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:44 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:39:45.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:45 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:39:45.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:45 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:39:45.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:45 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:39:46.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:46 smithi097 ceph-mon[30040]: pgmap v911: 300 pgs: 300 active+clean; 582 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 1.6 KiB/s wr, 3 op/s 2024-03-21T13:39:46.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:46 smithi097 ceph-mon[39416]: pgmap v911: 300 pgs: 300 active+clean; 582 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 1.6 KiB/s wr, 3 op/s 2024-03-21T13:39:46.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:46 smithi144 ceph-mon[33633]: pgmap v911: 300 pgs: 300 active+clean; 582 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 1.6 KiB/s wr, 3 op/s 2024-03-21T13:39:48.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:48 smithi097 ceph-mon[30040]: pgmap v912: 300 pgs: 300 active+clean; 582 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:39:48.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:48 smithi097 ceph-mon[39416]: pgmap v912: 300 pgs: 300 active+clean; 582 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:39:48.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:48 smithi144 ceph-mon[33633]: pgmap v912: 300 pgs: 300 active+clean; 582 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:39:50.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:50 smithi097 ceph-mon[39416]: pgmap v913: 300 pgs: 300 active+clean; 582 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 976 B/s wr, 2 op/s 2024-03-21T13:39:50.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:50 smithi097 ceph-mon[30040]: pgmap v913: 300 pgs: 300 active+clean; 582 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 976 B/s wr, 2 op/s 2024-03-21T13:39:50.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:50 smithi144 ceph-mon[33633]: pgmap v913: 300 pgs: 300 active+clean; 582 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 976 B/s wr, 2 op/s 2024-03-21T13:39:52.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:39:51 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:39:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:39:52.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:52 smithi097 ceph-mon[30040]: pgmap v914: 300 pgs: 300 active+clean; 582 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 859 B/s wr, 2 op/s 2024-03-21T13:39:52.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:52 smithi097 ceph-mon[39416]: pgmap v914: 300 pgs: 300 active+clean; 582 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 859 B/s wr, 2 op/s 2024-03-21T13:39:52.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:52 smithi144 ceph-mon[33633]: pgmap v914: 300 pgs: 300 active+clean; 582 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 859 B/s wr, 2 op/s 2024-03-21T13:39:53.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:53 smithi144 ceph-mon[33633]: pgmap v915: 300 pgs: 300 active+clean; 582 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 716 B/s wr, 2 op/s 2024-03-21T13:39:53.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:53 smithi097 ceph-mon[39416]: pgmap v915: 300 pgs: 300 active+clean; 582 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 716 B/s wr, 2 op/s 2024-03-21T13:39:53.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:53 smithi097 ceph-mon[30040]: pgmap v915: 300 pgs: 300 active+clean; 582 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 716 B/s wr, 2 op/s 2024-03-21T13:39:54.356 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:39:54 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:39:54.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:54 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:39:54.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:54 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:39:54.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:54 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:39:55.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:55 smithi144 ceph-mon[33633]: pgmap v916: 300 pgs: 300 active+clean; 582 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 632 B/s wr, 2 op/s 2024-03-21T13:39:55.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:55 smithi097 ceph-mon[39416]: pgmap v916: 300 pgs: 300 active+clean; 582 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 632 B/s wr, 2 op/s 2024-03-21T13:39:55.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:55 smithi097 ceph-mon[30040]: pgmap v916: 300 pgs: 300 active+clean; 582 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 632 B/s wr, 2 op/s 2024-03-21T13:39:56.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:56 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:39:56.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:56 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:39:57.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:56 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:39:57.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:56 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:39:57.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:56 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:39:57.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:56 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:39:58.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:57 smithi097 ceph-mon[39416]: pgmap v917: 300 pgs: 300 active+clean; 582 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:39:58.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:57 smithi097 ceph-mon[30040]: pgmap v917: 300 pgs: 300 active+clean; 582 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:39:58.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:57 smithi144 ceph-mon[33633]: pgmap v917: 300 pgs: 300 active+clean; 582 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:39:58.961 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.961 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.961 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.961 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.962 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.962 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.962 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.962 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.962 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.962 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.962 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.962 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.962 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.962 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.962 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.962 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.962 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.963 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.963 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.963 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.963 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.963 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.963 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.963 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.963 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.963 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.963 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.963 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.963 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.963 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.963 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.963 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.964 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.964 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.964 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.964 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.964 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.964 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.964 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.964 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.964 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.964 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.964 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.965 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.965 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.965 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.965 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.965 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.965 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.965 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.965 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.965 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.965 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.965 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.965 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.965 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.965 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.965 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.966 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.966 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.966 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.966 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.966 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.966 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.966 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.966 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.966 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.966 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.966 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.966 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.966 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.966 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.966 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.967 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.967 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.967 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.967 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.967 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.967 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.967 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.967 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.967 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.967 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.967 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.967 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.967 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.967 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.968 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.968 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.968 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.968 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.968 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.969 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.969 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.969 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.969 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.969 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.969 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:39:58.969 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:40:00.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:59 smithi097 ceph-mon[30040]: pgmap v918: 300 pgs: 300 active+clean; 583 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-21T13:40:00.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:59 smithi097 ceph-mon[30040]: osdmap e593: 8 total, 8 up, 8 in 2024-03-21T13:40:00.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:39:59 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:40:00.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:59 smithi097 ceph-mon[39416]: pgmap v918: 300 pgs: 300 active+clean; 583 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-21T13:40:00.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:59 smithi097 ceph-mon[39416]: osdmap e593: 8 total, 8 up, 8 in 2024-03-21T13:40:00.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:39:59 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:40:00.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:59 smithi144 ceph-mon[33633]: pgmap v918: 300 pgs: 300 active+clean; 583 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-21T13:40:00.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:59 smithi144 ceph-mon[33633]: osdmap e593: 8 total, 8 up, 8 in 2024-03-21T13:40:00.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:39:59 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:40:00.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:00 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:40:00.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:00 smithi144 ceph-mon[33633]: osdmap e594: 8 total, 8 up, 8 in 2024-03-21T13:40:00.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:00 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-116"}]: dispatch 2024-03-21T13:40:00.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:00 smithi144 ceph-mon[33633]: Health detail: HEALTH_WARN 1 cache pools are missing hit_sets; 5 pool(s) do not have an application enabled 2024-03-21T13:40:00.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:00 smithi144 ceph-mon[33633]: [WRN] CACHE_POOL_NO_HIT_SET: 1 cache pools are missing hit_sets 2024-03-21T13:40:00.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:00 smithi144 ceph-mon[33633]: pool 'test-rados-api-smithi097-79906-116' with cache_mode writeback needs hit_set_type to be set but it is not 2024-03-21T13:40:00.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:00 smithi144 ceph-mon[33633]: [WRN] POOL_APP_NOT_ENABLED: 5 pool(s) do not have an application enabled 2024-03-21T13:40:00.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:00 smithi144 ceph-mon[33633]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-03-21T13:40:00.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:00 smithi144 ceph-mon[33633]: application not enabled on pool 'poolrace.1804289383' 2024-03-21T13:40:00.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:00 smithi144 ceph-mon[33633]: application not enabled on pool 'AssertExistssmithi097-80928-1' 2024-03-21T13:40:00.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:00 smithi144 ceph-mon[33633]: application not enabled on pool 'poolrace2.846930886' 2024-03-21T13:40:00.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:00 smithi144 ceph-mon[33633]: application not enabled on pool 'test-rados-api-smithi097-79906-111' 2024-03-21T13:40:00.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:00 smithi144 ceph-mon[33633]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-03-21T13:40:00.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:00 smithi144 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:40:00.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:00 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-116"}]': finished 2024-03-21T13:40:00.873 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:00 smithi144 ceph-mon[33633]: osdmap e595: 8 total, 8 up, 8 in 2024-03-21T13:40:01.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:00 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:40:01.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:00 smithi097 ceph-mon[39416]: osdmap e594: 8 total, 8 up, 8 in 2024-03-21T13:40:01.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:00 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-116"}]: dispatch 2024-03-21T13:40:01.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:00 smithi097 ceph-mon[39416]: Health detail: HEALTH_WARN 1 cache pools are missing hit_sets; 5 pool(s) do not have an application enabled 2024-03-21T13:40:01.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:00 smithi097 ceph-mon[39416]: [WRN] CACHE_POOL_NO_HIT_SET: 1 cache pools are missing hit_sets 2024-03-21T13:40:01.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:00 smithi097 ceph-mon[39416]: pool 'test-rados-api-smithi097-79906-116' with cache_mode writeback needs hit_set_type to be set but it is not 2024-03-21T13:40:01.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:00 smithi097 ceph-mon[39416]: [WRN] POOL_APP_NOT_ENABLED: 5 pool(s) do not have an application enabled 2024-03-21T13:40:01.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:00 smithi097 ceph-mon[39416]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-03-21T13:40:01.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:00 smithi097 ceph-mon[39416]: application not enabled on pool 'poolrace.1804289383' 2024-03-21T13:40:01.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:00 smithi097 ceph-mon[39416]: application not enabled on pool 'AssertExistssmithi097-80928-1' 2024-03-21T13:40:01.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:00 smithi097 ceph-mon[39416]: application not enabled on pool 'poolrace2.846930886' 2024-03-21T13:40:01.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:00 smithi097 ceph-mon[39416]: application not enabled on pool 'test-rados-api-smithi097-79906-111' 2024-03-21T13:40:01.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:00 smithi097 ceph-mon[39416]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-03-21T13:40:01.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:00 smithi097 ceph-mon[39416]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:40:01.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:00 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-116"}]': finished 2024-03-21T13:40:01.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:00 smithi097 ceph-mon[39416]: osdmap e595: 8 total, 8 up, 8 in 2024-03-21T13:40:01.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:00 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:40:01.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:00 smithi097 ceph-mon[30040]: osdmap e594: 8 total, 8 up, 8 in 2024-03-21T13:40:01.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:00 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-116"}]: dispatch 2024-03-21T13:40:01.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:00 smithi097 ceph-mon[30040]: Health detail: HEALTH_WARN 1 cache pools are missing hit_sets; 5 pool(s) do not have an application enabled 2024-03-21T13:40:01.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:00 smithi097 ceph-mon[30040]: [WRN] CACHE_POOL_NO_HIT_SET: 1 cache pools are missing hit_sets 2024-03-21T13:40:01.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:00 smithi097 ceph-mon[30040]: pool 'test-rados-api-smithi097-79906-116' with cache_mode writeback needs hit_set_type to be set but it is not 2024-03-21T13:40:01.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:00 smithi097 ceph-mon[30040]: [WRN] POOL_APP_NOT_ENABLED: 5 pool(s) do not have an application enabled 2024-03-21T13:40:01.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:00 smithi097 ceph-mon[30040]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-03-21T13:40:01.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:00 smithi097 ceph-mon[30040]: application not enabled on pool 'poolrace.1804289383' 2024-03-21T13:40:01.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:00 smithi097 ceph-mon[30040]: application not enabled on pool 'AssertExistssmithi097-80928-1' 2024-03-21T13:40:01.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:00 smithi097 ceph-mon[30040]: application not enabled on pool 'poolrace2.846930886' 2024-03-21T13:40:01.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:00 smithi097 ceph-mon[30040]: application not enabled on pool 'test-rados-api-smithi097-79906-111' 2024-03-21T13:40:01.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:00 smithi097 ceph-mon[30040]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-03-21T13:40:01.056 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:00 smithi097 ceph-mon[30040]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:40:01.056 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:00 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-116"}]': finished 2024-03-21T13:40:01.056 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:00 smithi097 ceph-mon[30040]: osdmap e595: 8 total, 8 up, 8 in 2024-03-21T13:40:01.949 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:01 smithi097 ceph-mon[39416]: pgmap v921: 300 pgs: 300 active+clean; 583 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 2 op/s 2024-03-21T13:40:01.949 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:01 smithi097 ceph-mon[30040]: pgmap v921: 300 pgs: 300 active+clean; 583 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 2 op/s 2024-03-21T13:40:02.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:01 smithi144 ceph-mon[33633]: pgmap v921: 300 pgs: 300 active+clean; 583 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 2 op/s 2024-03-21T13:40:02.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:40:01 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:40:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:40:03.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:02 smithi097 ceph-mon[30040]: osdmap e596: 8 total, 8 up, 8 in 2024-03-21T13:40:03.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:02 smithi097 ceph-mon[39416]: osdmap e596: 8 total, 8 up, 8 in 2024-03-21T13:40:03.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:02 smithi144 ceph-mon[33633]: osdmap e596: 8 total, 8 up, 8 in 2024-03-21T13:40:04.033 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:03 smithi144 ceph-mon[33633]: pgmap v924: 268 pgs: 268 active+clean; 582 KiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:40:04.033 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:03 smithi144 ceph-mon[33633]: osdmap e597: 8 total, 8 up, 8 in 2024-03-21T13:40:04.033 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:03 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:40:04.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:03 smithi097 ceph-mon[30040]: pgmap v924: 268 pgs: 268 active+clean; 582 KiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:40:04.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:03 smithi097 ceph-mon[30040]: osdmap e597: 8 total, 8 up, 8 in 2024-03-21T13:40:04.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:03 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:40:04.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:03 smithi097 ceph-mon[39416]: pgmap v924: 268 pgs: 268 active+clean; 582 KiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:40:04.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:03 smithi097 ceph-mon[39416]: osdmap e597: 8 total, 8 up, 8 in 2024-03-21T13:40:04.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:03 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:40:04.371 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:40:04 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:40:05.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:04 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:40:05.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:04 smithi097 ceph-mon[39416]: osdmap e598: 8 total, 8 up, 8 in 2024-03-21T13:40:05.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:04 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:40:05.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:04 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:40:05.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:04 smithi097 ceph-mon[30040]: osdmap e598: 8 total, 8 up, 8 in 2024-03-21T13:40:05.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:04 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:40:05.123 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:04 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:40:05.123 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:04 smithi144 ceph-mon[33633]: osdmap e598: 8 total, 8 up, 8 in 2024-03-21T13:40:05.123 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:04 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:40:06.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:05 smithi097 ceph-mon[39416]: pgmap v927: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:40:06.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:05 smithi097 ceph-mon[39416]: osdmap e599: 8 total, 8 up, 8 in 2024-03-21T13:40:06.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:05 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:40:06.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:05 smithi097 ceph-mon[30040]: pgmap v927: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:40:06.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:05 smithi097 ceph-mon[30040]: osdmap e599: 8 total, 8 up, 8 in 2024-03-21T13:40:06.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:05 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:40:06.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:05 smithi144 ceph-mon[33633]: pgmap v927: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:40:06.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:05 smithi144 ceph-mon[33633]: osdmap e599: 8 total, 8 up, 8 in 2024-03-21T13:40:06.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:05 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:40:07.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:06 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-118", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:40:07.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:06 smithi097 ceph-mon[39416]: osdmap e600: 8 total, 8 up, 8 in 2024-03-21T13:40:07.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:06 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-118"}]: dispatch 2024-03-21T13:40:07.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:06 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-118", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:40:07.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:06 smithi097 ceph-mon[30040]: osdmap e600: 8 total, 8 up, 8 in 2024-03-21T13:40:07.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:06 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-118"}]: dispatch 2024-03-21T13:40:07.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:06 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-118", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:40:07.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:06 smithi144 ceph-mon[33633]: osdmap e600: 8 total, 8 up, 8 in 2024-03-21T13:40:07.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:06 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-118"}]: dispatch 2024-03-21T13:40:08.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:07 smithi097 ceph-mon[39416]: pgmap v930: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:40:08.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:07 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-118"}]': finished 2024-03-21T13:40:08.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:07 smithi097 ceph-mon[39416]: osdmap e601: 8 total, 8 up, 8 in 2024-03-21T13:40:08.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:07 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-118", "mode": "writeback"}]: dispatch 2024-03-21T13:40:08.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:07 smithi097 ceph-mon[39416]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:40:08.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:07 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-118", "mode": "writeback"}]': finished 2024-03-21T13:40:08.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:07 smithi097 ceph-mon[39416]: osdmap e602: 8 total, 8 up, 8 in 2024-03-21T13:40:08.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:07 smithi097 ceph-mon[30040]: pgmap v930: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:40:08.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:07 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-118"}]': finished 2024-03-21T13:40:08.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:07 smithi097 ceph-mon[30040]: osdmap e601: 8 total, 8 up, 8 in 2024-03-21T13:40:08.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:07 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-118", "mode": "writeback"}]: dispatch 2024-03-21T13:40:08.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:07 smithi097 ceph-mon[30040]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:40:08.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:07 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-118", "mode": "writeback"}]': finished 2024-03-21T13:40:08.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:07 smithi097 ceph-mon[30040]: osdmap e602: 8 total, 8 up, 8 in 2024-03-21T13:40:08.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:07 smithi144 ceph-mon[33633]: pgmap v930: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:40:08.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:07 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-118"}]': finished 2024-03-21T13:40:08.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:07 smithi144 ceph-mon[33633]: osdmap e601: 8 total, 8 up, 8 in 2024-03-21T13:40:08.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:07 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-118", "mode": "writeback"}]: dispatch 2024-03-21T13:40:08.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:07 smithi144 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:40:08.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:07 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-118", "mode": "writeback"}]': finished 2024-03-21T13:40:08.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:07 smithi144 ceph-mon[33633]: osdmap e602: 8 total, 8 up, 8 in 2024-03-21T13:40:09.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:09 smithi097 ceph-mon[39416]: osdmap e603: 8 total, 8 up, 8 in 2024-03-21T13:40:09.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:09 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:40:09.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:09 smithi097 ceph-mon[39416]: pgmap v934: 300 pgs: 300 active+clean; 582 KiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 520 B/s wr, 1 op/s 2024-03-21T13:40:09.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:09 smithi097 ceph-mon[30040]: osdmap e603: 8 total, 8 up, 8 in 2024-03-21T13:40:09.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:09 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:40:09.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:09 smithi097 ceph-mon[30040]: pgmap v934: 300 pgs: 300 active+clean; 582 KiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 520 B/s wr, 1 op/s 2024-03-21T13:40:09.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:09 smithi144 ceph-mon[33633]: osdmap e603: 8 total, 8 up, 8 in 2024-03-21T13:40:09.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:09 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:40:09.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:09 smithi144 ceph-mon[33633]: pgmap v934: 300 pgs: 300 active+clean; 582 KiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 520 B/s wr, 1 op/s 2024-03-21T13:40:10.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:10 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:40:10.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:10 smithi097 ceph-mon[39416]: osdmap e604: 8 total, 8 up, 8 in 2024-03-21T13:40:10.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:10 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-118"}]: dispatch 2024-03-21T13:40:10.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:10 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:40:10.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:10 smithi097 ceph-mon[30040]: osdmap e604: 8 total, 8 up, 8 in 2024-03-21T13:40:10.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:10 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-118"}]: dispatch 2024-03-21T13:40:10.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:10 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:40:10.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:10 smithi144 ceph-mon[33633]: osdmap e604: 8 total, 8 up, 8 in 2024-03-21T13:40:10.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:10 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-118"}]: dispatch 2024-03-21T13:40:11.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:11 smithi097 ceph-mon[39416]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:40:11.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:11 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-118"}]': finished 2024-03-21T13:40:11.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:11 smithi097 ceph-mon[39416]: osdmap e605: 8 total, 8 up, 8 in 2024-03-21T13:40:11.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:11 smithi097 ceph-mon[39416]: pgmap v937: 300 pgs: 300 active+clean; 582 KiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 521 B/s wr, 1 op/s 2024-03-21T13:40:11.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:11 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:40:11.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:11 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:40:11.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:11 smithi097 ceph-mon[30040]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:40:11.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:11 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-118"}]': finished 2024-03-21T13:40:11.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:11 smithi097 ceph-mon[30040]: osdmap e605: 8 total, 8 up, 8 in 2024-03-21T13:40:11.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:11 smithi097 ceph-mon[30040]: pgmap v937: 300 pgs: 300 active+clean; 582 KiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 521 B/s wr, 1 op/s 2024-03-21T13:40:11.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:11 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:40:11.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:11 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:40:11.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:11 smithi144 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:40:11.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:11 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-118"}]': finished 2024-03-21T13:40:11.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:11 smithi144 ceph-mon[33633]: osdmap e605: 8 total, 8 up, 8 in 2024-03-21T13:40:11.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:11 smithi144 ceph-mon[33633]: pgmap v937: 300 pgs: 300 active+clean; 582 KiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 521 B/s wr, 1 op/s 2024-03-21T13:40:11.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:11 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:40:11.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:11 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:40:12.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:40:11 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:40:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:40:12.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:12 smithi097 ceph-mon[30040]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:40:12.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:12 smithi097 ceph-mon[30040]: osdmap e606: 8 total, 8 up, 8 in 2024-03-21T13:40:12.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:12 smithi097 ceph-mon[39416]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:40:12.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:12 smithi097 ceph-mon[39416]: osdmap e606: 8 total, 8 up, 8 in 2024-03-21T13:40:12.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:12 smithi144 ceph-mon[33633]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:40:12.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:12 smithi144 ceph-mon[33633]: osdmap e606: 8 total, 8 up, 8 in 2024-03-21T13:40:13.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:13 smithi097 ceph-mon[30040]: osdmap e607: 8 total, 8 up, 8 in 2024-03-21T13:40:13.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:13 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:40:13.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:13 smithi097 ceph-mon[30040]: pgmap v940: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 637 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:40:13.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:13 smithi097 ceph-mon[39416]: osdmap e607: 8 total, 8 up, 8 in 2024-03-21T13:40:13.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:13 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:40:13.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:13 smithi097 ceph-mon[39416]: pgmap v940: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 637 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:40:13.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:13 smithi144 ceph-mon[33633]: osdmap e607: 8 total, 8 up, 8 in 2024-03-21T13:40:13.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:13 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:40:13.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:13 smithi144 ceph-mon[33633]: pgmap v940: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 637 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:40:14.371 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:40:14 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:40:14.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:14 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:40:14.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:14 smithi097 ceph-mon[30040]: osdmap e608: 8 total, 8 up, 8 in 2024-03-21T13:40:14.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:14 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:40:14.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:14 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:40:14.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:14 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:40:14.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:14 smithi097 ceph-mon[39416]: osdmap e608: 8 total, 8 up, 8 in 2024-03-21T13:40:14.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:14 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:40:14.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:14 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:40:14.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:14 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:40:14.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:14 smithi144 ceph-mon[33633]: osdmap e608: 8 total, 8 up, 8 in 2024-03-21T13:40:14.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:14 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:40:14.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:14 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:40:15.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:15 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-120", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:40:15.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:15 smithi097 ceph-mon[30040]: osdmap e609: 8 total, 8 up, 8 in 2024-03-21T13:40:15.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:15 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-120"}]: dispatch 2024-03-21T13:40:15.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:15 smithi097 ceph-mon[30040]: pgmap v943: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:40:15.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:15 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-120", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:40:15.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:15 smithi097 ceph-mon[39416]: osdmap e609: 8 total, 8 up, 8 in 2024-03-21T13:40:15.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:15 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-120"}]: dispatch 2024-03-21T13:40:15.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:15 smithi097 ceph-mon[39416]: pgmap v943: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:40:15.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:15 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-120", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:40:15.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:15 smithi144 ceph-mon[33633]: osdmap e609: 8 total, 8 up, 8 in 2024-03-21T13:40:15.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:15 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-120"}]: dispatch 2024-03-21T13:40:15.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:15 smithi144 ceph-mon[33633]: pgmap v943: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:40:16.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:16 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-120"}]': finished 2024-03-21T13:40:16.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:16 smithi097 ceph-mon[30040]: osdmap e610: 8 total, 8 up, 8 in 2024-03-21T13:40:16.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:16 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-120", "mode": "writeback"}]: dispatch 2024-03-21T13:40:16.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:16 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-120"}]': finished 2024-03-21T13:40:16.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:16 smithi097 ceph-mon[39416]: osdmap e610: 8 total, 8 up, 8 in 2024-03-21T13:40:16.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:16 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-120", "mode": "writeback"}]: dispatch 2024-03-21T13:40:16.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:16 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-120"}]': finished 2024-03-21T13:40:16.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:16 smithi144 ceph-mon[33633]: osdmap e610: 8 total, 8 up, 8 in 2024-03-21T13:40:16.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:16 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-120", "mode": "writeback"}]: dispatch 2024-03-21T13:40:17.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:17 smithi144 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:40:17.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:17 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-120", "mode": "writeback"}]': finished 2024-03-21T13:40:17.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:17 smithi144 ceph-mon[33633]: osdmap e611: 8 total, 8 up, 8 in 2024-03-21T13:40:17.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:17 smithi144 ceph-mon[33633]: pgmap v946: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:40:17.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:17 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:40:17.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:17 smithi144 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:40:17.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:17 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:40:17.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:17 smithi144 ceph-mon[33633]: osdmap e612: 8 total, 8 up, 8 in 2024-03-21T13:40:17.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:17 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-120"}]: dispatch 2024-03-21T13:40:18.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:17 smithi097 ceph-mon[30040]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:40:18.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:17 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-120", "mode": "writeback"}]': finished 2024-03-21T13:40:18.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:17 smithi097 ceph-mon[30040]: osdmap e611: 8 total, 8 up, 8 in 2024-03-21T13:40:18.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:17 smithi097 ceph-mon[30040]: pgmap v946: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:40:18.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:17 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:40:18.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:17 smithi097 ceph-mon[30040]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:40:18.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:17 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:40:18.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:17 smithi097 ceph-mon[30040]: osdmap e612: 8 total, 8 up, 8 in 2024-03-21T13:40:18.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:17 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-120"}]: dispatch 2024-03-21T13:40:18.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:17 smithi097 ceph-mon[39416]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:40:18.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:17 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-120", "mode": "writeback"}]': finished 2024-03-21T13:40:18.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:17 smithi097 ceph-mon[39416]: osdmap e611: 8 total, 8 up, 8 in 2024-03-21T13:40:18.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:17 smithi097 ceph-mon[39416]: pgmap v946: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:40:18.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:17 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:40:18.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:17 smithi097 ceph-mon[39416]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:40:18.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:17 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:40:18.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:17 smithi097 ceph-mon[39416]: osdmap e612: 8 total, 8 up, 8 in 2024-03-21T13:40:18.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:17 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-120"}]: dispatch 2024-03-21T13:40:18.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:18 smithi144 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:40:18.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:18 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-120"}]': finished 2024-03-21T13:40:18.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:18 smithi144 ceph-mon[33633]: osdmap e613: 8 total, 8 up, 8 in 2024-03-21T13:40:19.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:18 smithi097 ceph-mon[39416]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:40:19.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:18 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-120"}]': finished 2024-03-21T13:40:19.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:18 smithi097 ceph-mon[39416]: osdmap e613: 8 total, 8 up, 8 in 2024-03-21T13:40:19.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:18 smithi097 ceph-mon[30040]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:40:19.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:18 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-120"}]': finished 2024-03-21T13:40:19.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:18 smithi097 ceph-mon[30040]: osdmap e613: 8 total, 8 up, 8 in 2024-03-21T13:40:20.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:19 smithi097 ceph-mon[39416]: pgmap v949: 300 pgs: 300 active+clean; 582 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-03-21T13:40:20.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:19 smithi097 ceph-mon[39416]: osdmap e614: 8 total, 8 up, 8 in 2024-03-21T13:40:20.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:19 smithi097 ceph-mon[30040]: pgmap v949: 300 pgs: 300 active+clean; 582 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-03-21T13:40:20.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:19 smithi097 ceph-mon[30040]: osdmap e614: 8 total, 8 up, 8 in 2024-03-21T13:40:20.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:19 smithi144 ceph-mon[33633]: pgmap v949: 300 pgs: 300 active+clean; 582 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-03-21T13:40:20.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:19 smithi144 ceph-mon[33633]: osdmap e614: 8 total, 8 up, 8 in 2024-03-21T13:40:21.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:21 smithi097 ceph-mon[39416]: osdmap e615: 8 total, 8 up, 8 in 2024-03-21T13:40:21.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:21 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:40:21.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:21 smithi097 ceph-mon[39416]: pgmap v952: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T13:40:21.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:21 smithi097 ceph-mon[30040]: osdmap e615: 8 total, 8 up, 8 in 2024-03-21T13:40:21.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:21 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:40:21.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:21 smithi097 ceph-mon[30040]: pgmap v952: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T13:40:21.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:21 smithi144 ceph-mon[33633]: osdmap e615: 8 total, 8 up, 8 in 2024-03-21T13:40:21.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:21 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:40:21.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:21 smithi144 ceph-mon[33633]: pgmap v952: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T13:40:22.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:40:21 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:40:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:40:22.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:22 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:40:22.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:22 smithi097 ceph-mon[30040]: osdmap e616: 8 total, 8 up, 8 in 2024-03-21T13:40:22.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:22 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:40:22.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:22 smithi097 ceph-mon[30040]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:40:22.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:22 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:40:22.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:22 smithi097 ceph-mon[39416]: osdmap e616: 8 total, 8 up, 8 in 2024-03-21T13:40:22.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:22 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:40:22.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:22 smithi097 ceph-mon[39416]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:40:22.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:22 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:40:22.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:22 smithi144 ceph-mon[33633]: osdmap e616: 8 total, 8 up, 8 in 2024-03-21T13:40:22.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:22 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:40:22.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:22 smithi144 ceph-mon[33633]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:40:23.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:23 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-122", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:40:23.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:23 smithi097 ceph-mon[30040]: osdmap e617: 8 total, 8 up, 8 in 2024-03-21T13:40:23.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:23 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-122"}]: dispatch 2024-03-21T13:40:23.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:23 smithi097 ceph-mon[30040]: pgmap v955: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 660 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:40:23.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:23 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-122", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:40:23.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:23 smithi097 ceph-mon[39416]: osdmap e617: 8 total, 8 up, 8 in 2024-03-21T13:40:23.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:23 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-122"}]: dispatch 2024-03-21T13:40:23.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:23 smithi097 ceph-mon[39416]: pgmap v955: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 660 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:40:23.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:23 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-122", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:40:23.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:23 smithi144 ceph-mon[33633]: osdmap e617: 8 total, 8 up, 8 in 2024-03-21T13:40:23.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:23 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-122"}]: dispatch 2024-03-21T13:40:23.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:23 smithi144 ceph-mon[33633]: pgmap v955: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 660 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:40:24.371 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:40:24 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:40:24.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:24 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-122"}]': finished 2024-03-21T13:40:24.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:24 smithi097 ceph-mon[39416]: osdmap e618: 8 total, 8 up, 8 in 2024-03-21T13:40:24.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:24 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-122", "mode": "writeback"}]: dispatch 2024-03-21T13:40:24.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:24 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:40:24.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:24 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-122"}]': finished 2024-03-21T13:40:24.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:24 smithi097 ceph-mon[30040]: osdmap e618: 8 total, 8 up, 8 in 2024-03-21T13:40:24.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:24 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-122", "mode": "writeback"}]: dispatch 2024-03-21T13:40:24.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:24 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:40:24.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:24 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-122"}]': finished 2024-03-21T13:40:24.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:24 smithi144 ceph-mon[33633]: osdmap e618: 8 total, 8 up, 8 in 2024-03-21T13:40:24.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:24 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-122", "mode": "writeback"}]: dispatch 2024-03-21T13:40:24.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:24 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:40:25.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:25 smithi097 ceph-mon[39416]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:40:25.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:25 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-122", "mode": "writeback"}]': finished 2024-03-21T13:40:25.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:25 smithi097 ceph-mon[39416]: osdmap e619: 8 total, 8 up, 8 in 2024-03-21T13:40:25.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:25 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:40:25.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:25 smithi097 ceph-mon[39416]: pgmap v958: 300 pgs: 300 active+clean; 582 KiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:40:25.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:25 smithi097 ceph-mon[30040]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:40:25.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:25 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-122", "mode": "writeback"}]': finished 2024-03-21T13:40:25.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:25 smithi097 ceph-mon[30040]: osdmap e619: 8 total, 8 up, 8 in 2024-03-21T13:40:25.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:25 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:40:25.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:25 smithi097 ceph-mon[30040]: pgmap v958: 300 pgs: 300 active+clean; 582 KiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:40:25.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:25 smithi144 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:40:25.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:25 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-122", "mode": "writeback"}]': finished 2024-03-21T13:40:25.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:25 smithi144 ceph-mon[33633]: osdmap e619: 8 total, 8 up, 8 in 2024-03-21T13:40:25.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:25 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:40:25.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:25 smithi144 ceph-mon[33633]: pgmap v958: 300 pgs: 300 active+clean; 582 KiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:40:26.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:26 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:40:26.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:26 smithi097 ceph-mon[39416]: osdmap e620: 8 total, 8 up, 8 in 2024-03-21T13:40:26.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:26 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-122"}]: dispatch 2024-03-21T13:40:26.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:26 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:40:26.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:26 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:40:26.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:26 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:40:26.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:26 smithi097 ceph-mon[30040]: osdmap e620: 8 total, 8 up, 8 in 2024-03-21T13:40:26.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:26 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-122"}]: dispatch 2024-03-21T13:40:26.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:26 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:40:26.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:26 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:40:26.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:26 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:40:26.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:26 smithi144 ceph-mon[33633]: osdmap e620: 8 total, 8 up, 8 in 2024-03-21T13:40:26.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:26 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-122"}]: dispatch 2024-03-21T13:40:26.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:26 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:40:26.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:26 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:40:27.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:27 smithi144 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:40:27.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:27 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-122"}]': finished 2024-03-21T13:40:27.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:27 smithi144 ceph-mon[33633]: osdmap e621: 8 total, 8 up, 8 in 2024-03-21T13:40:27.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:27 smithi144 ceph-mon[33633]: pgmap v961: 300 pgs: 300 active+clean; 582 KiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:40:27.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:27 smithi144 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:40:28.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:27 smithi097 ceph-mon[39416]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:40:28.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:27 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-122"}]': finished 2024-03-21T13:40:28.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:27 smithi097 ceph-mon[39416]: osdmap e621: 8 total, 8 up, 8 in 2024-03-21T13:40:28.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:27 smithi097 ceph-mon[39416]: pgmap v961: 300 pgs: 300 active+clean; 582 KiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:40:28.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:27 smithi097 ceph-mon[39416]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:40:28.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:27 smithi097 ceph-mon[30040]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:40:28.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:27 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-122"}]': finished 2024-03-21T13:40:28.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:27 smithi097 ceph-mon[30040]: osdmap e621: 8 total, 8 up, 8 in 2024-03-21T13:40:28.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:27 smithi097 ceph-mon[30040]: pgmap v961: 300 pgs: 300 active+clean; 582 KiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:40:28.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:27 smithi097 ceph-mon[30040]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:40:28.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:28 smithi144 ceph-mon[33633]: osdmap e622: 8 total, 8 up, 8 in 2024-03-21T13:40:29.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:28 smithi097 ceph-mon[39416]: osdmap e622: 8 total, 8 up, 8 in 2024-03-21T13:40:29.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:28 smithi097 ceph-mon[30040]: osdmap e622: 8 total, 8 up, 8 in 2024-03-21T13:40:29.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:29 smithi144 ceph-mon[33633]: osdmap e623: 8 total, 8 up, 8 in 2024-03-21T13:40:29.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:29 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:40:29.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:29 smithi144 ceph-mon[33633]: pgmap v964: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:40:30.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:29 smithi097 ceph-mon[39416]: osdmap e623: 8 total, 8 up, 8 in 2024-03-21T13:40:30.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:29 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:40:30.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:29 smithi097 ceph-mon[39416]: pgmap v964: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:40:30.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:29 smithi097 ceph-mon[30040]: osdmap e623: 8 total, 8 up, 8 in 2024-03-21T13:40:30.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:29 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:40:30.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:29 smithi097 ceph-mon[30040]: pgmap v964: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:40:30.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:30 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:40:30.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:30 smithi144 ceph-mon[33633]: osdmap e624: 8 total, 8 up, 8 in 2024-03-21T13:40:31.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:30 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:40:31.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:30 smithi097 ceph-mon[39416]: osdmap e624: 8 total, 8 up, 8 in 2024-03-21T13:40:31.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:30 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:40:31.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:30 smithi097 ceph-mon[30040]: osdmap e624: 8 total, 8 up, 8 in 2024-03-21T13:40:31.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:31 smithi144 ceph-mon[33633]: osdmap e625: 8 total, 8 up, 8 in 2024-03-21T13:40:31.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:31 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:40:31.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:31 smithi144 ceph-mon[33633]: pgmap v967: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:40:31.949 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:31 smithi097 ceph-mon[39416]: osdmap e625: 8 total, 8 up, 8 in 2024-03-21T13:40:31.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:31 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:40:31.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:31 smithi097 ceph-mon[39416]: pgmap v967: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:40:31.950 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:31 smithi097 ceph-mon[30040]: osdmap e625: 8 total, 8 up, 8 in 2024-03-21T13:40:31.950 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:31 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:40:31.950 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:31 smithi097 ceph-mon[30040]: pgmap v967: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:40:32.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:40:31 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:40:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:40:32.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:32 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-124", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:40:32.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:32 smithi144 ceph-mon[33633]: osdmap e626: 8 total, 8 up, 8 in 2024-03-21T13:40:32.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:32 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-124"}]: dispatch 2024-03-21T13:40:33.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:32 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-124", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:40:33.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:32 smithi097 ceph-mon[39416]: osdmap e626: 8 total, 8 up, 8 in 2024-03-21T13:40:33.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:32 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-124"}]: dispatch 2024-03-21T13:40:33.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:32 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-124", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:40:33.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:32 smithi097 ceph-mon[30040]: osdmap e626: 8 total, 8 up, 8 in 2024-03-21T13:40:33.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:32 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-124"}]: dispatch 2024-03-21T13:40:33.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:33 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-124"}]': finished 2024-03-21T13:40:33.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:33 smithi144 ceph-mon[33633]: osdmap e627: 8 total, 8 up, 8 in 2024-03-21T13:40:33.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:33 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-124", "mode": "writeback"}]: dispatch 2024-03-21T13:40:33.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:33 smithi144 ceph-mon[33633]: pgmap v970: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 665 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:40:33.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:33 smithi144 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:40:33.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:33 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-124", "mode": "writeback"}]': finished 2024-03-21T13:40:34.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:33 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-124"}]': finished 2024-03-21T13:40:34.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:33 smithi097 ceph-mon[39416]: osdmap e627: 8 total, 8 up, 8 in 2024-03-21T13:40:34.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:33 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-124", "mode": "writeback"}]: dispatch 2024-03-21T13:40:34.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:33 smithi097 ceph-mon[39416]: pgmap v970: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 665 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:40:34.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:33 smithi097 ceph-mon[39416]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:40:34.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:33 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-124", "mode": "writeback"}]': finished 2024-03-21T13:40:34.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:33 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-124"}]': finished 2024-03-21T13:40:34.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:33 smithi097 ceph-mon[30040]: osdmap e627: 8 total, 8 up, 8 in 2024-03-21T13:40:34.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:33 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-124", "mode": "writeback"}]: dispatch 2024-03-21T13:40:34.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:33 smithi097 ceph-mon[30040]: pgmap v970: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 665 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:40:34.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:33 smithi097 ceph-mon[30040]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:40:34.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:33 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-124", "mode": "writeback"}]': finished 2024-03-21T13:40:34.371 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:40:34 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:40:34.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:34 smithi144 ceph-mon[33633]: osdmap e628: 8 total, 8 up, 8 in 2024-03-21T13:40:34.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:34 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:40:35.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:34 smithi097 ceph-mon[39416]: osdmap e628: 8 total, 8 up, 8 in 2024-03-21T13:40:35.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:34 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:40:35.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:34 smithi097 ceph-mon[30040]: osdmap e628: 8 total, 8 up, 8 in 2024-03-21T13:40:35.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:34 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:40:35.873 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:35 smithi144 ceph-mon[33633]: osdmap e629: 8 total, 8 up, 8 in 2024-03-21T13:40:35.873 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:35 smithi144 ceph-mon[33633]: pgmap v973: 300 pgs: 300 active+clean; 583 KiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-21T13:40:35.873 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:35 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:40:36.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:35 smithi097 ceph-mon[39416]: osdmap e629: 8 total, 8 up, 8 in 2024-03-21T13:40:36.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:35 smithi097 ceph-mon[39416]: pgmap v973: 300 pgs: 300 active+clean; 583 KiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-21T13:40:36.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:35 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:40:36.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:35 smithi097 ceph-mon[30040]: osdmap e629: 8 total, 8 up, 8 in 2024-03-21T13:40:36.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:35 smithi097 ceph-mon[30040]: pgmap v973: 300 pgs: 300 active+clean; 583 KiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-21T13:40:36.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:35 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:40:36.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:36 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:40:36.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:36 smithi144 ceph-mon[33633]: osdmap e630: 8 total, 8 up, 8 in 2024-03-21T13:40:36.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:36 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-124"}]: dispatch 2024-03-21T13:40:37.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:36 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:40:37.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:36 smithi097 ceph-mon[39416]: osdmap e630: 8 total, 8 up, 8 in 2024-03-21T13:40:37.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:36 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-124"}]: dispatch 2024-03-21T13:40:37.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:36 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:40:37.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:36 smithi097 ceph-mon[30040]: osdmap e630: 8 total, 8 up, 8 in 2024-03-21T13:40:37.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:36 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-124"}]: dispatch 2024-03-21T13:40:37.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:37 smithi144 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:40:37.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:37 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-124"}]': finished 2024-03-21T13:40:37.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:37 smithi144 ceph-mon[33633]: osdmap e631: 8 total, 8 up, 8 in 2024-03-21T13:40:37.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:37 smithi144 ceph-mon[33633]: pgmap v976: 300 pgs: 300 active+clean; 583 KiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-21T13:40:38.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:37 smithi097 ceph-mon[39416]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:40:38.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:37 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-124"}]': finished 2024-03-21T13:40:38.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:37 smithi097 ceph-mon[39416]: osdmap e631: 8 total, 8 up, 8 in 2024-03-21T13:40:38.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:37 smithi097 ceph-mon[39416]: pgmap v976: 300 pgs: 300 active+clean; 583 KiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-21T13:40:38.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:37 smithi097 ceph-mon[30040]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:40:38.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:37 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-124"}]': finished 2024-03-21T13:40:38.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:37 smithi097 ceph-mon[30040]: osdmap e631: 8 total, 8 up, 8 in 2024-03-21T13:40:38.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:37 smithi097 ceph-mon[30040]: pgmap v976: 300 pgs: 300 active+clean; 583 KiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-21T13:40:38.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:38 smithi144 ceph-mon[33633]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:40:38.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:38 smithi144 ceph-mon[33633]: osdmap e632: 8 total, 8 up, 8 in 2024-03-21T13:40:39.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:38 smithi097 ceph-mon[39416]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:40:39.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:38 smithi097 ceph-mon[39416]: osdmap e632: 8 total, 8 up, 8 in 2024-03-21T13:40:39.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:38 smithi097 ceph-mon[30040]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:40:39.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:38 smithi097 ceph-mon[30040]: osdmap e632: 8 total, 8 up, 8 in 2024-03-21T13:40:40.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:39 smithi097 ceph-mon[39416]: osdmap e633: 8 total, 8 up, 8 in 2024-03-21T13:40:40.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:39 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:40:40.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:39 smithi097 ceph-mon[39416]: pgmap v979: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:40:40.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:39 smithi097 ceph-mon[30040]: osdmap e633: 8 total, 8 up, 8 in 2024-03-21T13:40:40.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:39 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:40:40.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:39 smithi097 ceph-mon[30040]: pgmap v979: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:40:40.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:39 smithi144 ceph-mon[33633]: osdmap e633: 8 total, 8 up, 8 in 2024-03-21T13:40:40.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:39 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:40:40.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:39 smithi144 ceph-mon[33633]: pgmap v979: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:40:40.954 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:40 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:40:40.954 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:40 smithi097 ceph-mon[39416]: osdmap e634: 8 total, 8 up, 8 in 2024-03-21T13:40:40.954 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:40 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:40:40.954 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:40 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-126", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:40:40.954 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:40 smithi097 ceph-mon[39416]: osdmap e635: 8 total, 8 up, 8 in 2024-03-21T13:40:40.954 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:40 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-126"}]: dispatch 2024-03-21T13:40:40.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:40 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:40:40.955 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:40 smithi097 ceph-mon[30040]: osdmap e634: 8 total, 8 up, 8 in 2024-03-21T13:40:40.955 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:40 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:40:40.955 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:40 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-126", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:40:40.955 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:40 smithi097 ceph-mon[30040]: osdmap e635: 8 total, 8 up, 8 in 2024-03-21T13:40:40.955 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:40 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-126"}]: dispatch 2024-03-21T13:40:41.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:40 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:40:41.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:40 smithi144 ceph-mon[33633]: osdmap e634: 8 total, 8 up, 8 in 2024-03-21T13:40:41.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:40 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:40:41.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:40 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-126", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:40:41.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:40 smithi144 ceph-mon[33633]: osdmap e635: 8 total, 8 up, 8 in 2024-03-21T13:40:41.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:40 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-126"}]: dispatch 2024-03-21T13:40:41.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:41 smithi097 ceph-mon[39416]: pgmap v981: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:40:41.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:41 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:40:41.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:41 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:40:41.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:41 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-126"}]': finished 2024-03-21T13:40:41.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:41 smithi097 ceph-mon[39416]: osdmap e636: 8 total, 8 up, 8 in 2024-03-21T13:40:41.951 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:41 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-126", "mode": "writeback"}]: dispatch 2024-03-21T13:40:41.951 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:41 smithi097 ceph-mon[30040]: pgmap v981: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:40:41.951 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:41 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:40:41.951 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:41 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:40:41.951 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:41 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-126"}]': finished 2024-03-21T13:40:41.951 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:41 smithi097 ceph-mon[30040]: osdmap e636: 8 total, 8 up, 8 in 2024-03-21T13:40:41.951 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:41 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-126", "mode": "writeback"}]: dispatch 2024-03-21T13:40:42.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:41 smithi144 ceph-mon[33633]: pgmap v981: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:40:42.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:41 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:40:42.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:41 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:40:42.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:41 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-126"}]': finished 2024-03-21T13:40:42.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:41 smithi144 ceph-mon[33633]: osdmap e636: 8 total, 8 up, 8 in 2024-03-21T13:40:42.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:41 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-126", "mode": "writeback"}]: dispatch 2024-03-21T13:40:42.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:40:41 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:40:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:40:43.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:42 smithi097 ceph-mon[39416]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:40:43.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:42 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-126", "mode": "writeback"}]': finished 2024-03-21T13:40:43.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:42 smithi097 ceph-mon[39416]: osdmap e637: 8 total, 8 up, 8 in 2024-03-21T13:40:43.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:42 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:40:43.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:42 smithi097 ceph-mon[30040]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:40:43.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:42 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-126", "mode": "writeback"}]': finished 2024-03-21T13:40:43.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:42 smithi097 ceph-mon[30040]: osdmap e637: 8 total, 8 up, 8 in 2024-03-21T13:40:43.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:42 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:40:43.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:42 smithi144 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:40:43.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:42 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-126", "mode": "writeback"}]': finished 2024-03-21T13:40:43.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:42 smithi144 ceph-mon[33633]: osdmap e637: 8 total, 8 up, 8 in 2024-03-21T13:40:43.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:42 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:40:44.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:43 smithi097 ceph-mon[39416]: pgmap v985: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 650 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:40:44.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:43 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:40:44.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:43 smithi097 ceph-mon[39416]: osdmap e638: 8 total, 8 up, 8 in 2024-03-21T13:40:44.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:43 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-126"}]: dispatch 2024-03-21T13:40:44.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:43 smithi097 ceph-mon[30040]: pgmap v985: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 650 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:40:44.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:43 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:40:44.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:43 smithi097 ceph-mon[30040]: osdmap e638: 8 total, 8 up, 8 in 2024-03-21T13:40:44.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:43 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-126"}]: dispatch 2024-03-21T13:40:44.076 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:43 smithi144 ceph-mon[33633]: pgmap v985: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 650 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:40:44.076 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:43 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:40:44.076 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:43 smithi144 ceph-mon[33633]: osdmap e638: 8 total, 8 up, 8 in 2024-03-21T13:40:44.076 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:43 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-126"}]: dispatch 2024-03-21T13:40:44.371 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:40:44 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:40:45.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:44 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:40:45.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:44 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:40:45.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:44 smithi097 ceph-mon[39416]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:40:45.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:44 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-126"}]': finished 2024-03-21T13:40:45.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:44 smithi097 ceph-mon[39416]: osdmap e639: 8 total, 8 up, 8 in 2024-03-21T13:40:45.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:44 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:40:45.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:44 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:40:45.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:44 smithi097 ceph-mon[30040]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:40:45.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:44 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-126"}]': finished 2024-03-21T13:40:45.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:44 smithi097 ceph-mon[30040]: osdmap e639: 8 total, 8 up, 8 in 2024-03-21T13:40:45.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:44 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:40:45.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:44 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:40:45.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:44 smithi144 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:40:45.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:44 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-126"}]': finished 2024-03-21T13:40:45.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:44 smithi144 ceph-mon[33633]: osdmap e639: 8 total, 8 up, 8 in 2024-03-21T13:40:45.523 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 seapi_tier_pp: require_osd_release = squid 2024-03-21T13:40:45.523 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringFlush (9058 ms) 2024-03-21T13:40:45.523 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapHasChunk 2024-03-21T13:40:45.523 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: require_osd_release = squid 2024-03-21T13:40:45.523 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapHasChunk (6057 ms) 2024-03-21T13:40:45.523 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollback 2024-03-21T13:40:45.524 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: require_osd_release = squid 2024-03-21T13:40:45.524 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollback (5052 ms) 2024-03-21T13:40:45.524 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollbackRefcount 2024-03-21T13:40:45.524 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: require_osd_release = squid 2024-03-21T13:40:45.524 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollbackRefcount (25635 ms) 2024-03-21T13:40:45.524 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictRollback 2024-03-21T13:40:45.524 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: require_osd_release = squid 2024-03-21T13:40:45.524 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictRollback (13290 ms) 2024-03-21T13:40:45.524 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PropagateBaseTierError 2024-03-21T13:40:45.524 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PropagateBaseTierError (12111 ms) 2024-03-21T13:40:45.524 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HelloWriteReturn 2024-03-21T13:40:45.524 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: 00000000 79 6f 75 20 6d 69 67 68 74 20 73 65 65 20 74 68 |you might see th| 2024-03-21T13:40:45.524 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: 00000010 69 73 |is| 2024-03-21T13:40:45.524 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: 00000012 2024-03-21T13:40:45.525 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HelloWriteReturn (12084 ms) 2024-03-21T13:40:45.525 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier 2024-03-21T13:40:45.525 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: require_osd_release = squid 2024-03-21T13:40:45.525 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier (6048 ms) 2024-03-21T13:40:45.525 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP (552720 ms total) 2024-03-21T13:40:45.525 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: 2024-03-21T13:40:45.525 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP 2024-03-21T13:40:45.525 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.Dirty 2024-03-21T13:40:45.525 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.Dirty (1639 ms) 2024-03-21T13:40:45.525 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.FlushWriteRaces 2024-03-21T13:40:45.525 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.FlushWriteRaces (10409 ms) 2024-03-21T13:40:45.525 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.CallForcesPromote 2024-03-21T13:40:45.525 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.CallForcesPromote (18104 ms) 2024-03-21T13:40:45.525 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.HitSetNone 2024-03-21T13:40:45.526 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.HitSetNone (3 ms) 2024-03-21T13:40:45.526 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP (30155 ms total) 2024-03-21T13:40:45.526 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: 2024-03-21T13:40:45.526 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP 2024-03-21T13:40:45.526 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Overlay 2024-03-21T13:40:45.526 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Overlay (7055 ms) 2024-03-21T13:40:45.526 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Promote 2024-03-21T13:40:45.526 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Promote (8062 ms) 2024-03-21T13:40:45.526 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnap 2024-03-21T13:40:45.526 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: waiting for scrub... 2024-03-21T13:40:45.526 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: done waiting 2024-03-21T13:40:45.526 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnap (24422 ms) 2024-03-21T13:40:45.526 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace 2024-03-21T13:40:45.526 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace (9860 ms) 2024-03-21T13:40:45.526 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Whiteout 2024-03-21T13:40:45.527 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Whiteout (7978 ms) 2024-03-21T13:40:45.527 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Evict 2024-03-21T13:40:45.527 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Evict (8061 ms) 2024-03-21T13:40:45.527 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.EvictSnap 2024-03-21T13:40:45.527 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.EvictSnap (10055 ms) 2024-03-21T13:40:45.527 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlush 2024-03-21T13:40:46.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:45 smithi097 ceph-mon[30040]: pgmap v988: 300 pgs: 300 active+clean; 582 KiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-21T13:40:46.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:45 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:40:46.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:45 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:40:46.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:45 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:40:46.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:45 smithi097 ceph-mon[30040]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:40:46.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:45 smithi097 ceph-mon[30040]: osdmap e640: 8 total, 8 up, 8 in 2024-03-21T13:40:46.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:45 smithi097 ceph-mon[39416]: pgmap v988: 300 pgs: 300 active+clean; 582 KiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-21T13:40:46.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:45 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:40:46.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:45 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:40:46.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:45 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:40:46.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:45 smithi097 ceph-mon[39416]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:40:46.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:45 smithi097 ceph-mon[39416]: osdmap e640: 8 total, 8 up, 8 in 2024-03-21T13:40:46.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:45 smithi144 ceph-mon[33633]: pgmap v988: 300 pgs: 300 active+clean; 582 KiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-21T13:40:46.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:45 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:40:46.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:45 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:40:46.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:45 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:40:46.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:45 smithi144 ceph-mon[33633]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:40:46.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:45 smithi144 ceph-mon[33633]: osdmap e640: 8 total, 8 up, 8 in 2024-03-21T13:40:47.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:47 smithi097 ceph-mon[39416]: osdmap e641: 8 total, 8 up, 8 in 2024-03-21T13:40:47.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:47 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:40:47.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:47 smithi097 ceph-mon[39416]: pgmap v991: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:40:47.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:47 smithi097 ceph-mon[30040]: osdmap e641: 8 total, 8 up, 8 in 2024-03-21T13:40:47.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:47 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:40:47.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:47 smithi097 ceph-mon[30040]: pgmap v991: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:40:47.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:47 smithi144 ceph-mon[33633]: osdmap e641: 8 total, 8 up, 8 in 2024-03-21T13:40:47.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:47 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:40:47.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:47 smithi144 ceph-mon[33633]: pgmap v991: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:40:48.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:48 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:40:48.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:48 smithi097 ceph-mon[30040]: osdmap e642: 8 total, 8 up, 8 in 2024-03-21T13:40:48.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:48 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:40:48.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:48 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:40:48.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:48 smithi097 ceph-mon[39416]: osdmap e642: 8 total, 8 up, 8 in 2024-03-21T13:40:48.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:48 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:40:48.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:48 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:40:48.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:48 smithi144 ceph-mon[33633]: osdmap e642: 8 total, 8 up, 8 in 2024-03-21T13:40:48.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:48 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:40:49.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:49 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-128", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:40:49.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:49 smithi097 ceph-mon[39416]: osdmap e643: 8 total, 8 up, 8 in 2024-03-21T13:40:49.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:49 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-128"}]: dispatch 2024-03-21T13:40:49.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:49 smithi097 ceph-mon[39416]: pgmap v994: 300 pgs: 300 active+clean; 582 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-21T13:40:49.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:49 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-128"}]': finished 2024-03-21T13:40:49.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:49 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-128", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:40:49.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:49 smithi097 ceph-mon[30040]: osdmap e643: 8 total, 8 up, 8 in 2024-03-21T13:40:49.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:49 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-128"}]: dispatch 2024-03-21T13:40:49.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:49 smithi097 ceph-mon[30040]: pgmap v994: 300 pgs: 300 active+clean; 582 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-21T13:40:49.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:49 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-128"}]': finished 2024-03-21T13:40:49.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:49 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-128", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:40:49.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:49 smithi144 ceph-mon[33633]: osdmap e643: 8 total, 8 up, 8 in 2024-03-21T13:40:49.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:49 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-128"}]: dispatch 2024-03-21T13:40:49.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:49 smithi144 ceph-mon[33633]: pgmap v994: 300 pgs: 300 active+clean; 582 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-21T13:40:49.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:49 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-128"}]': finished 2024-03-21T13:40:50.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:50 smithi097 ceph-mon[30040]: osdmap e644: 8 total, 8 up, 8 in 2024-03-21T13:40:50.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:50 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-128", "mode": "writeback"}]: dispatch 2024-03-21T13:40:50.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:50 smithi097 ceph-mon[30040]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:40:50.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:50 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-128", "mode": "writeback"}]': finished 2024-03-21T13:40:50.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:50 smithi097 ceph-mon[39416]: osdmap e644: 8 total, 8 up, 8 in 2024-03-21T13:40:50.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:50 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-128", "mode": "writeback"}]: dispatch 2024-03-21T13:40:50.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:50 smithi097 ceph-mon[39416]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:40:50.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:50 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-128", "mode": "writeback"}]': finished 2024-03-21T13:40:50.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:50 smithi144 ceph-mon[33633]: osdmap e644: 8 total, 8 up, 8 in 2024-03-21T13:40:50.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:50 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-128", "mode": "writeback"}]: dispatch 2024-03-21T13:40:50.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:50 smithi144 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:40:50.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:50 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-128", "mode": "writeback"}]': finished 2024-03-21T13:40:51.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:51 smithi144 ceph-mon[33633]: osdmap e645: 8 total, 8 up, 8 in 2024-03-21T13:40:51.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:51 smithi144 ceph-mon[33633]: pgmap v997: 300 pgs: 300 active+clean; 582 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:40:51.949 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:51 smithi097 ceph-mon[39416]: osdmap e645: 8 total, 8 up, 8 in 2024-03-21T13:40:51.949 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:51 smithi097 ceph-mon[39416]: pgmap v997: 300 pgs: 300 active+clean; 582 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:40:51.950 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:51 smithi097 ceph-mon[30040]: osdmap e645: 8 total, 8 up, 8 in 2024-03-21T13:40:51.950 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:51 smithi097 ceph-mon[30040]: pgmap v997: 300 pgs: 300 active+clean; 582 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:40:52.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:40:51 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:40:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:40:52.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:52 smithi144 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:40:53.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:52 smithi097 ceph-mon[39416]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:40:53.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:52 smithi097 ceph-mon[30040]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:40:53.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:53 smithi144 ceph-mon[33633]: pgmap v998: 300 pgs: 300 active+clean; 582 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1002 B/s rd, 0 op/s 2024-03-21T13:40:54.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:53 smithi097 ceph-mon[39416]: pgmap v998: 300 pgs: 300 active+clean; 582 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1002 B/s rd, 0 op/s 2024-03-21T13:40:54.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:53 smithi097 ceph-mon[30040]: pgmap v998: 300 pgs: 300 active+clean; 582 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1002 B/s rd, 0 op/s 2024-03-21T13:40:54.371 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:40:54 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:40:54.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:54 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:40:55.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:54 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:40:55.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:54 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:40:55.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:55 smithi144 ceph-mon[33633]: pgmap v999: 300 pgs: 300 active+clean; 582 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1006 B/s rd, 335 B/s wr, 1 op/s 2024-03-21T13:40:56.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:55 smithi097 ceph-mon[39416]: pgmap v999: 300 pgs: 300 active+clean; 582 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1006 B/s rd, 335 B/s wr, 1 op/s 2024-03-21T13:40:56.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:55 smithi097 ceph-mon[30040]: pgmap v999: 300 pgs: 300 active+clean; 582 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1006 B/s rd, 335 B/s wr, 1 op/s 2024-03-21T13:40:56.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:56 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:40:56.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:56 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:40:56.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:56 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:40:57.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:56 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:40:57.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:56 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:40:57.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:56 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:40:57.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:56 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:40:57.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:56 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:40:57.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:56 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:40:57.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:57 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:40:57.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:57 smithi144 ceph-mon[33633]: osdmap e646: 8 total, 8 up, 8 in 2024-03-21T13:40:57.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:57 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-128"}]: dispatch 2024-03-21T13:40:57.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:57 smithi144 ceph-mon[33633]: pgmap v1001: 300 pgs: 300 active+clean; 582 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 864 B/s rd, 288 B/s wr, 1 op/s 2024-03-21T13:40:58.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:57 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:40:58.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:57 smithi097 ceph-mon[39416]: osdmap e646: 8 total, 8 up, 8 in 2024-03-21T13:40:58.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:57 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-128"}]: dispatch 2024-03-21T13:40:58.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:57 smithi097 ceph-mon[39416]: pgmap v1001: 300 pgs: 300 active+clean; 582 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 864 B/s rd, 288 B/s wr, 1 op/s 2024-03-21T13:40:58.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:57 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:40:58.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:57 smithi097 ceph-mon[30040]: osdmap e646: 8 total, 8 up, 8 in 2024-03-21T13:40:58.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:57 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-128"}]: dispatch 2024-03-21T13:40:58.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:57 smithi097 ceph-mon[30040]: pgmap v1001: 300 pgs: 300 active+clean; 582 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 864 B/s rd, 288 B/s wr, 1 op/s 2024-03-21T13:40:59.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:58 smithi144 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:40:59.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:58 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-128"}]': finished 2024-03-21T13:40:59.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:58 smithi144 ceph-mon[33633]: osdmap e647: 8 total, 8 up, 8 in 2024-03-21T13:40:59.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:58 smithi097 ceph-mon[39416]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:40:59.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:58 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-128"}]': finished 2024-03-21T13:40:59.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:58 smithi097 ceph-mon[39416]: osdmap e647: 8 total, 8 up, 8 in 2024-03-21T13:40:59.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:58 smithi097 ceph-mon[30040]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:40:59.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:58 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-128"}]': finished 2024-03-21T13:40:59.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:58 smithi097 ceph-mon[30040]: osdmap e647: 8 total, 8 up, 8 in 2024-03-21T13:41:00.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:59 smithi144 ceph-mon[33633]: osdmap e648: 8 total, 8 up, 8 in 2024-03-21T13:41:00.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:59 smithi144 ceph-mon[33633]: pgmap v1004: 268 pgs: 268 active+clean; 582 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-21T13:41:00.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:59 smithi144 ceph-mon[33633]: osdmap e649: 8 total, 8 up, 8 in 2024-03-21T13:41:00.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:40:59 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:41:00.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:59 smithi097 ceph-mon[39416]: osdmap e648: 8 total, 8 up, 8 in 2024-03-21T13:41:00.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:59 smithi097 ceph-mon[39416]: pgmap v1004: 268 pgs: 268 active+clean; 582 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-21T13:41:00.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:59 smithi097 ceph-mon[39416]: osdmap e649: 8 total, 8 up, 8 in 2024-03-21T13:41:00.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:40:59 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:41:00.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:59 smithi097 ceph-mon[30040]: osdmap e648: 8 total, 8 up, 8 in 2024-03-21T13:41:00.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:59 smithi097 ceph-mon[30040]: pgmap v1004: 268 pgs: 268 active+clean; 582 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-21T13:41:00.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:59 smithi097 ceph-mon[30040]: osdmap e649: 8 total, 8 up, 8 in 2024-03-21T13:41:00.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:40:59 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:41:01.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:01 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:41:01.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:01 smithi144 ceph-mon[33633]: osdmap e650: 8 total, 8 up, 8 in 2024-03-21T13:41:01.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:01 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:41:01.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:01 smithi144 ceph-mon[33633]: pgmap v1007: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:41:01.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:01 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:41:01.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:01 smithi097 ceph-mon[39416]: osdmap e650: 8 total, 8 up, 8 in 2024-03-21T13:41:01.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:01 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:41:01.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:01 smithi097 ceph-mon[39416]: pgmap v1007: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:41:01.950 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:01 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:41:01.950 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:01 smithi097 ceph-mon[30040]: osdmap e650: 8 total, 8 up, 8 in 2024-03-21T13:41:01.950 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:01 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:41:01.950 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:01 smithi097 ceph-mon[30040]: pgmap v1007: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:41:02.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:41:01 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:41:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:41:02.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:02 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-130", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:41:02.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:02 smithi144 ceph-mon[33633]: osdmap e651: 8 total, 8 up, 8 in 2024-03-21T13:41:02.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:02 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-130"}]: dispatch 2024-03-21T13:41:03.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:02 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-130", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:41:03.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:02 smithi097 ceph-mon[39416]: osdmap e651: 8 total, 8 up, 8 in 2024-03-21T13:41:03.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:02 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-130"}]: dispatch 2024-03-21T13:41:03.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:02 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-130", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:41:03.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:02 smithi097 ceph-mon[30040]: osdmap e651: 8 total, 8 up, 8 in 2024-03-21T13:41:03.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:02 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-130"}]: dispatch 2024-03-21T13:41:03.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:03 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-130"}]': finished 2024-03-21T13:41:03.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:03 smithi144 ceph-mon[33633]: osdmap e652: 8 total, 8 up, 8 in 2024-03-21T13:41:03.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:03 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-130", "mode": "writeback"}]: dispatch 2024-03-21T13:41:03.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:03 smithi144 ceph-mon[33633]: pgmap v1010: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 652 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:41:03.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:03 smithi144 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:41:03.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:03 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-130", "mode": "writeback"}]': finished 2024-03-21T13:41:03.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:03 smithi144 ceph-mon[33633]: osdmap e653: 8 total, 8 up, 8 in 2024-03-21T13:41:04.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:03 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-130"}]': finished 2024-03-21T13:41:04.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:03 smithi097 ceph-mon[39416]: osdmap e652: 8 total, 8 up, 8 in 2024-03-21T13:41:04.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:03 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-130", "mode": "writeback"}]: dispatch 2024-03-21T13:41:04.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:03 smithi097 ceph-mon[39416]: pgmap v1010: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 652 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:41:04.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:03 smithi097 ceph-mon[39416]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:41:04.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:03 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-130", "mode": "writeback"}]': finished 2024-03-21T13:41:04.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:03 smithi097 ceph-mon[39416]: osdmap e653: 8 total, 8 up, 8 in 2024-03-21T13:41:04.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:03 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-130"}]': finished 2024-03-21T13:41:04.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:03 smithi097 ceph-mon[30040]: osdmap e652: 8 total, 8 up, 8 in 2024-03-21T13:41:04.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:03 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-130", "mode": "writeback"}]: dispatch 2024-03-21T13:41:04.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:03 smithi097 ceph-mon[30040]: pgmap v1010: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 652 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:41:04.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:03 smithi097 ceph-mon[30040]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:41:04.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:03 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-130", "mode": "writeback"}]': finished 2024-03-21T13:41:04.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:03 smithi097 ceph-mon[30040]: osdmap e653: 8 total, 8 up, 8 in 2024-03-21T13:41:04.371 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:41:04 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:41:05.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:04 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:41:05.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:04 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:41:05.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:04 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:41:05.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:04 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:41:05.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:04 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:41:05.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:04 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:41:06.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:05 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:41:06.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:05 smithi097 ceph-mon[39416]: osdmap e654: 8 total, 8 up, 8 in 2024-03-21T13:41:06.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:05 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-130"}]: dispatch 2024-03-21T13:41:06.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:05 smithi097 ceph-mon[39416]: pgmap v1013: 300 pgs: 300 active+clean; 582 KiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:41:06.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:05 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:41:06.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:05 smithi097 ceph-mon[30040]: osdmap e654: 8 total, 8 up, 8 in 2024-03-21T13:41:06.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:05 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-130"}]: dispatch 2024-03-21T13:41:06.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:05 smithi097 ceph-mon[30040]: pgmap v1013: 300 pgs: 300 active+clean; 582 KiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:41:06.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:05 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:41:06.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:05 smithi144 ceph-mon[33633]: osdmap e654: 8 total, 8 up, 8 in 2024-03-21T13:41:06.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:05 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-130"}]: dispatch 2024-03-21T13:41:06.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:05 smithi144 ceph-mon[33633]: pgmap v1013: 300 pgs: 300 active+clean; 582 KiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:41:07.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:06 smithi097 ceph-mon[30040]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:41:07.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:06 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-130"}]': finished 2024-03-21T13:41:07.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:06 smithi097 ceph-mon[30040]: osdmap e655: 8 total, 8 up, 8 in 2024-03-21T13:41:07.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:06 smithi097 ceph-mon[39416]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:41:07.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:06 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-130"}]': finished 2024-03-21T13:41:07.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:06 smithi097 ceph-mon[39416]: osdmap e655: 8 total, 8 up, 8 in 2024-03-21T13:41:07.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:06 smithi144 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:41:07.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:06 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-130"}]': finished 2024-03-21T13:41:07.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:06 smithi144 ceph-mon[33633]: osdmap e655: 8 total, 8 up, 8 in 2024-03-21T13:41:08.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:07 smithi097 ceph-mon[39416]: pgmap v1015: 300 pgs: 300 active+clean; 582 KiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:41:08.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:07 smithi097 ceph-mon[39416]: osdmap e656: 8 total, 8 up, 8 in 2024-03-21T13:41:08.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:07 smithi097 ceph-mon[39416]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:41:08.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:07 smithi097 ceph-mon[39416]: osdmap e657: 8 total, 8 up, 8 in 2024-03-21T13:41:08.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:07 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:41:08.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:07 smithi097 ceph-mon[30040]: pgmap v1015: 300 pgs: 300 active+clean; 582 KiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:41:08.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:07 smithi097 ceph-mon[30040]: osdmap e656: 8 total, 8 up, 8 in 2024-03-21T13:41:08.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:07 smithi097 ceph-mon[30040]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:41:08.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:07 smithi097 ceph-mon[30040]: osdmap e657: 8 total, 8 up, 8 in 2024-03-21T13:41:08.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:07 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:41:08.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:07 smithi144 ceph-mon[33633]: pgmap v1015: 300 pgs: 300 active+clean; 582 KiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:41:08.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:07 smithi144 ceph-mon[33633]: osdmap e656: 8 total, 8 up, 8 in 2024-03-21T13:41:08.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:07 smithi144 ceph-mon[33633]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:41:08.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:07 smithi144 ceph-mon[33633]: osdmap e657: 8 total, 8 up, 8 in 2024-03-21T13:41:08.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:07 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:41:10.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:09 smithi097 ceph-mon[30040]: pgmap v1018: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:41:10.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:09 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:41:10.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:09 smithi097 ceph-mon[30040]: osdmap e658: 8 total, 8 up, 8 in 2024-03-21T13:41:10.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:09 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:41:10.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:09 smithi097 ceph-mon[39416]: pgmap v1018: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:41:10.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:09 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:41:10.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:09 smithi097 ceph-mon[39416]: osdmap e658: 8 total, 8 up, 8 in 2024-03-21T13:41:10.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:09 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:41:10.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:09 smithi144 ceph-mon[33633]: pgmap v1018: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:41:10.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:09 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:41:10.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:09 smithi144 ceph-mon[33633]: osdmap e658: 8 total, 8 up, 8 in 2024-03-21T13:41:10.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:09 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:41:11.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:10 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-132", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:41:11.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:10 smithi097 ceph-mon[39416]: osdmap e659: 8 total, 8 up, 8 in 2024-03-21T13:41:11.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:10 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-132"}]: dispatch 2024-03-21T13:41:11.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:10 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-132"}]': finished 2024-03-21T13:41:11.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:10 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-132", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:41:11.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:10 smithi097 ceph-mon[30040]: osdmap e659: 8 total, 8 up, 8 in 2024-03-21T13:41:11.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:10 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-132"}]: dispatch 2024-03-21T13:41:11.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:10 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-132"}]': finished 2024-03-21T13:41:11.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:10 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-132", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:41:11.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:10 smithi144 ceph-mon[33633]: osdmap e659: 8 total, 8 up, 8 in 2024-03-21T13:41:11.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:10 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-132"}]: dispatch 2024-03-21T13:41:11.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:10 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-132"}]': finished 2024-03-21T13:41:11.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:11 smithi097 ceph-mon[39416]: pgmap v1021: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:41:11.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:11 smithi097 ceph-mon[39416]: osdmap e660: 8 total, 8 up, 8 in 2024-03-21T13:41:11.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:11 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-132", "mode": "writeback"}]: dispatch 2024-03-21T13:41:11.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:11 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:41:11.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:11 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:41:11.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:11 smithi097 ceph-mon[39416]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:41:11.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:11 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-132", "mode": "writeback"}]': finished 2024-03-21T13:41:11.951 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:11 smithi097 ceph-mon[39416]: osdmap e661: 8 total, 8 up, 8 in 2024-03-21T13:41:11.951 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:11 smithi097 ceph-mon[30040]: pgmap v1021: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:41:11.951 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:11 smithi097 ceph-mon[30040]: osdmap e660: 8 total, 8 up, 8 in 2024-03-21T13:41:11.951 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:11 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-132", "mode": "writeback"}]: dispatch 2024-03-21T13:41:11.951 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:11 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:41:11.951 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:11 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:41:11.951 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:11 smithi097 ceph-mon[30040]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:41:11.951 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:11 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-132", "mode": "writeback"}]': finished 2024-03-21T13:41:11.951 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:11 smithi097 ceph-mon[30040]: osdmap e661: 8 total, 8 up, 8 in 2024-03-21T13:41:12.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:11 smithi144 ceph-mon[33633]: pgmap v1021: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:41:12.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:11 smithi144 ceph-mon[33633]: osdmap e660: 8 total, 8 up, 8 in 2024-03-21T13:41:12.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:11 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-132", "mode": "writeback"}]: dispatch 2024-03-21T13:41:12.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:11 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:41:12.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:11 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:41:12.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:11 smithi144 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:41:12.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:11 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-132", "mode": "writeback"}]': finished 2024-03-21T13:41:12.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:11 smithi144 ceph-mon[33633]: osdmap e661: 8 total, 8 up, 8 in 2024-03-21T13:41:12.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:41:11 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:41:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:41:14.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:13 smithi097 ceph-mon[39416]: pgmap v1024: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:41:14.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:13 smithi097 ceph-mon[39416]: osdmap e662: 8 total, 8 up, 8 in 2024-03-21T13:41:14.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:13 smithi097 ceph-mon[30040]: pgmap v1024: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:41:14.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:13 smithi097 ceph-mon[30040]: osdmap e662: 8 total, 8 up, 8 in 2024-03-21T13:41:14.100 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:13 smithi144 ceph-mon[33633]: pgmap v1024: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:41:14.101 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:13 smithi144 ceph-mon[33633]: osdmap e662: 8 total, 8 up, 8 in 2024-03-21T13:41:14.371 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:41:14 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:41:15.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:14 smithi097 ceph-mon[30040]: osdmap e663: 8 total, 8 up, 8 in 2024-03-21T13:41:15.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:14 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:41:15.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:14 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:41:15.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:14 smithi097 ceph-mon[39416]: osdmap e663: 8 total, 8 up, 8 in 2024-03-21T13:41:15.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:14 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:41:15.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:14 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:41:15.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:14 smithi144 ceph-mon[33633]: osdmap e663: 8 total, 8 up, 8 in 2024-03-21T13:41:15.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:14 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:41:15.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:14 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:41:16.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:15 smithi097 ceph-mon[39416]: pgmap v1027: 300 pgs: 300 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T13:41:16.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:15 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:41:16.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:15 smithi097 ceph-mon[39416]: osdmap e664: 8 total, 8 up, 8 in 2024-03-21T13:41:16.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:15 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-132"}]: dispatch 2024-03-21T13:41:16.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:15 smithi097 ceph-mon[30040]: pgmap v1027: 300 pgs: 300 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T13:41:16.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:15 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:41:16.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:15 smithi097 ceph-mon[30040]: osdmap e664: 8 total, 8 up, 8 in 2024-03-21T13:41:16.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:15 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-132"}]: dispatch 2024-03-21T13:41:16.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:15 smithi144 ceph-mon[33633]: pgmap v1027: 300 pgs: 300 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T13:41:16.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:15 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:41:16.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:15 smithi144 ceph-mon[33633]: osdmap e664: 8 total, 8 up, 8 in 2024-03-21T13:41:16.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:15 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-132"}]: dispatch 2024-03-21T13:41:17.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:16 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-132"}]': finished 2024-03-21T13:41:17.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:16 smithi097 ceph-mon[30040]: osdmap e665: 8 total, 8 up, 8 in 2024-03-21T13:41:17.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:16 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-132"}]': finished 2024-03-21T13:41:17.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:16 smithi097 ceph-mon[39416]: osdmap e665: 8 total, 8 up, 8 in 2024-03-21T13:41:17.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:16 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-132"}]': finished 2024-03-21T13:41:17.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:16 smithi144 ceph-mon[33633]: osdmap e665: 8 total, 8 up, 8 in 2024-03-21T13:41:18.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:17 smithi097 ceph-mon[30040]: pgmap v1030: 300 pgs: 300 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T13:41:18.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:17 smithi097 ceph-mon[30040]: osdmap e666: 8 total, 8 up, 8 in 2024-03-21T13:41:18.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:17 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:41:18.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:17 smithi097 ceph-mon[30040]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:41:18.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:17 smithi097 ceph-mon[39416]: pgmap v1030: 300 pgs: 300 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T13:41:18.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:17 smithi097 ceph-mon[39416]: osdmap e666: 8 total, 8 up, 8 in 2024-03-21T13:41:18.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:17 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:41:18.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:17 smithi097 ceph-mon[39416]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:41:18.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:17 smithi144 ceph-mon[33633]: pgmap v1030: 300 pgs: 300 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T13:41:18.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:17 smithi144 ceph-mon[33633]: osdmap e666: 8 total, 8 up, 8 in 2024-03-21T13:41:18.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:17 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:41:18.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:17 smithi144 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:41:19.057 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:18 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:41:19.057 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:18 smithi097 ceph-mon[39416]: osdmap e667: 8 total, 8 up, 8 in 2024-03-21T13:41:19.057 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:18 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-132"}]: dispatch 2024-03-21T13:41:19.058 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:18 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:41:19.058 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:18 smithi097 ceph-mon[30040]: osdmap e667: 8 total, 8 up, 8 in 2024-03-21T13:41:19.058 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:18 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-132"}]: dispatch 2024-03-21T13:41:19.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:18 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:41:19.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:18 smithi144 ceph-mon[33633]: osdmap e667: 8 total, 8 up, 8 in 2024-03-21T13:41:19.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:18 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-132"}]: dispatch 2024-03-21T13:41:20.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:19 smithi097 ceph-mon[39416]: pgmap v1033: 300 pgs: 300 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1023 B/s wr, 3 op/s 2024-03-21T13:41:20.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:19 smithi097 ceph-mon[39416]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:41:20.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:19 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-132"}]': finished 2024-03-21T13:41:20.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:19 smithi097 ceph-mon[39416]: osdmap e668: 8 total, 8 up, 8 in 2024-03-21T13:41:20.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:19 smithi097 ceph-mon[30040]: pgmap v1033: 300 pgs: 300 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1023 B/s wr, 3 op/s 2024-03-21T13:41:20.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:19 smithi097 ceph-mon[30040]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:41:20.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:19 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-132"}]': finished 2024-03-21T13:41:20.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:19 smithi097 ceph-mon[30040]: osdmap e668: 8 total, 8 up, 8 in 2024-03-21T13:41:20.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:19 smithi144 ceph-mon[33633]: pgmap v1033: 300 pgs: 300 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1023 B/s wr, 3 op/s 2024-03-21T13:41:20.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:19 smithi144 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:41:20.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:19 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-132"}]': finished 2024-03-21T13:41:20.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:19 smithi144 ceph-mon[33633]: osdmap e668: 8 total, 8 up, 8 in 2024-03-21T13:41:21.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:20 smithi097 ceph-mon[30040]: osdmap e669: 8 total, 8 up, 8 in 2024-03-21T13:41:21.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:20 smithi097 ceph-mon[39416]: osdmap e669: 8 total, 8 up, 8 in 2024-03-21T13:41:21.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:20 smithi144 ceph-mon[33633]: osdmap e669: 8 total, 8 up, 8 in 2024-03-21T13:41:22.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:21 smithi097 ceph-mon[30040]: pgmap v1036: 268 pgs: 268 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-21T13:41:22.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:21 smithi097 ceph-mon[30040]: osdmap e670: 8 total, 8 up, 8 in 2024-03-21T13:41:22.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:21 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:41:22.054 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:41:21 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:41:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:41:22.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:21 smithi097 ceph-mon[39416]: pgmap v1036: 268 pgs: 268 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-21T13:41:22.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:21 smithi097 ceph-mon[39416]: osdmap e670: 8 total, 8 up, 8 in 2024-03-21T13:41:22.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:21 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:41:22.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:21 smithi144 ceph-mon[33633]: pgmap v1036: 268 pgs: 268 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-21T13:41:22.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:21 smithi144 ceph-mon[33633]: osdmap e670: 8 total, 8 up, 8 in 2024-03-21T13:41:22.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:21 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:41:23.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:22 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:41:23.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:22 smithi097 ceph-mon[39416]: osdmap e671: 8 total, 8 up, 8 in 2024-03-21T13:41:23.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:22 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:41:23.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:22 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-134", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:41:23.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:22 smithi097 ceph-mon[39416]: osdmap e672: 8 total, 8 up, 8 in 2024-03-21T13:41:23.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:22 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-134"}]: dispatch 2024-03-21T13:41:23.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:22 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:41:23.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:22 smithi097 ceph-mon[30040]: osdmap e671: 8 total, 8 up, 8 in 2024-03-21T13:41:23.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:22 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:41:23.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:22 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-134", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:41:23.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:22 smithi097 ceph-mon[30040]: osdmap e672: 8 total, 8 up, 8 in 2024-03-21T13:41:23.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:22 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-134"}]: dispatch 2024-03-21T13:41:23.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:22 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:41:23.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:22 smithi144 ceph-mon[33633]: osdmap e671: 8 total, 8 up, 8 in 2024-03-21T13:41:23.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:22 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:41:23.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:22 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-134", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:41:23.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:22 smithi144 ceph-mon[33633]: osdmap e672: 8 total, 8 up, 8 in 2024-03-21T13:41:23.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:22 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-134"}]: dispatch 2024-03-21T13:41:24.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:23 smithi097 ceph-mon[30040]: pgmap v1040: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:41:24.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:23 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-134"}]': finished 2024-03-21T13:41:24.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:23 smithi097 ceph-mon[30040]: osdmap e673: 8 total, 8 up, 8 in 2024-03-21T13:41:24.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:23 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-134", "mode": "writeback"}]: dispatch 2024-03-21T13:41:24.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:23 smithi097 ceph-mon[39416]: pgmap v1040: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:41:24.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:23 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-134"}]': finished 2024-03-21T13:41:24.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:23 smithi097 ceph-mon[39416]: osdmap e673: 8 total, 8 up, 8 in 2024-03-21T13:41:24.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:23 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-134", "mode": "writeback"}]: dispatch 2024-03-21T13:41:24.103 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:23 smithi144 ceph-mon[33633]: pgmap v1040: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:41:24.103 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:23 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-134"}]': finished 2024-03-21T13:41:24.103 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:23 smithi144 ceph-mon[33633]: osdmap e673: 8 total, 8 up, 8 in 2024-03-21T13:41:24.103 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:23 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-134", "mode": "writeback"}]: dispatch 2024-03-21T13:41:24.371 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:41:24 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:41:25.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:24 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:41:25.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:24 smithi097 ceph-mon[30040]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:41:25.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:24 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-134", "mode": "writeback"}]': finished 2024-03-21T13:41:25.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:24 smithi097 ceph-mon[30040]: osdmap e674: 8 total, 8 up, 8 in 2024-03-21T13:41:25.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:24 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:41:25.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:24 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:41:25.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:24 smithi097 ceph-mon[39416]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:41:25.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:24 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-134", "mode": "writeback"}]': finished 2024-03-21T13:41:25.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:24 smithi097 ceph-mon[39416]: osdmap e674: 8 total, 8 up, 8 in 2024-03-21T13:41:25.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:24 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:41:25.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:24 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:41:25.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:24 smithi144 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:41:25.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:24 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-134", "mode": "writeback"}]': finished 2024-03-21T13:41:25.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:24 smithi144 ceph-mon[33633]: osdmap e674: 8 total, 8 up, 8 in 2024-03-21T13:41:25.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:24 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:41:26.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:25 smithi097 ceph-mon[30040]: pgmap v1043: 300 pgs: 300 active+clean; 582 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-21T13:41:26.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:25 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:41:26.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:25 smithi097 ceph-mon[30040]: osdmap e675: 8 total, 8 up, 8 in 2024-03-21T13:41:26.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:25 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-134"}]: dispatch 2024-03-21T13:41:26.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:25 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:41:26.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:25 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:41:26.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:25 smithi097 ceph-mon[39416]: pgmap v1043: 300 pgs: 300 active+clean; 582 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-21T13:41:26.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:25 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:41:26.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:25 smithi097 ceph-mon[39416]: osdmap e675: 8 total, 8 up, 8 in 2024-03-21T13:41:26.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:25 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-134"}]: dispatch 2024-03-21T13:41:26.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:25 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:41:26.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:25 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:41:26.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:25 smithi144 ceph-mon[33633]: pgmap v1043: 300 pgs: 300 active+clean; 582 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-21T13:41:26.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:25 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:41:26.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:25 smithi144 ceph-mon[33633]: osdmap e675: 8 total, 8 up, 8 in 2024-03-21T13:41:26.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:25 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-134"}]: dispatch 2024-03-21T13:41:26.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:25 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:41:26.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:25 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:41:27.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:26 smithi097 ceph-mon[30040]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:41:27.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:26 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-134"}]': finished 2024-03-21T13:41:27.056 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:26 smithi097 ceph-mon[30040]: osdmap e676: 8 total, 8 up, 8 in 2024-03-21T13:41:27.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:26 smithi097 ceph-mon[39416]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:41:27.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:26 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-134"}]': finished 2024-03-21T13:41:27.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:26 smithi097 ceph-mon[39416]: osdmap e676: 8 total, 8 up, 8 in 2024-03-21T13:41:27.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:26 smithi144 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:41:27.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:26 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-134"}]': finished 2024-03-21T13:41:27.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:26 smithi144 ceph-mon[33633]: osdmap e676: 8 total, 8 up, 8 in 2024-03-21T13:41:28.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:27 smithi097 ceph-mon[30040]: pgmap v1046: 300 pgs: 300 active+clean; 582 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:41:28.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:27 smithi097 ceph-mon[30040]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:41:28.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:27 smithi097 ceph-mon[30040]: osdmap e677: 8 total, 8 up, 8 in 2024-03-21T13:41:28.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:27 smithi097 ceph-mon[39416]: pgmap v1046: 300 pgs: 300 active+clean; 582 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:41:28.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:27 smithi097 ceph-mon[39416]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:41:28.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:27 smithi097 ceph-mon[39416]: osdmap e677: 8 total, 8 up, 8 in 2024-03-21T13:41:28.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:27 smithi144 ceph-mon[33633]: pgmap v1046: 300 pgs: 300 active+clean; 582 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:41:28.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:27 smithi144 ceph-mon[33633]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:41:28.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:27 smithi144 ceph-mon[33633]: osdmap e677: 8 total, 8 up, 8 in 2024-03-21T13:41:29.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:29 smithi097 ceph-mon[30040]: osdmap e678: 8 total, 8 up, 8 in 2024-03-21T13:41:29.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:29 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:41:29.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:29 smithi097 ceph-mon[30040]: pgmap v1049: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:41:29.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:29 smithi097 ceph-mon[39416]: osdmap e678: 8 total, 8 up, 8 in 2024-03-21T13:41:29.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:29 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:41:29.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:29 smithi097 ceph-mon[39416]: pgmap v1049: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:41:29.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:29 smithi144 ceph-mon[33633]: osdmap e678: 8 total, 8 up, 8 in 2024-03-21T13:41:29.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:29 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:41:29.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:29 smithi144 ceph-mon[33633]: pgmap v1049: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:41:30.806 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:30 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:41:30.807 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:30 smithi097 ceph-mon[30040]: osdmap e679: 8 total, 8 up, 8 in 2024-03-21T13:41:30.807 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:30 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:41:30.807 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:30 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:41:30.807 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:30 smithi097 ceph-mon[39416]: osdmap e679: 8 total, 8 up, 8 in 2024-03-21T13:41:30.807 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:30 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:41:30.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:30 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:41:30.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:30 smithi144 ceph-mon[33633]: osdmap e679: 8 total, 8 up, 8 in 2024-03-21T13:41:30.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:30 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:41:31.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:31 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-136", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:41:31.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:31 smithi144 ceph-mon[33633]: osdmap e680: 8 total, 8 up, 8 in 2024-03-21T13:41:31.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:31 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-136"}]: dispatch 2024-03-21T13:41:31.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:31 smithi144 ceph-mon[33633]: pgmap v1052: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:41:31.949 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:31 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-136", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:41:31.949 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:31 smithi097 ceph-mon[39416]: osdmap e680: 8 total, 8 up, 8 in 2024-03-21T13:41:31.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:31 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-136"}]: dispatch 2024-03-21T13:41:31.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:31 smithi097 ceph-mon[39416]: pgmap v1052: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:41:31.950 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:31 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-136", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:41:31.950 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:31 smithi097 ceph-mon[30040]: osdmap e680: 8 total, 8 up, 8 in 2024-03-21T13:41:31.950 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:31 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-136"}]: dispatch 2024-03-21T13:41:31.950 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:31 smithi097 ceph-mon[30040]: pgmap v1052: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:41:32.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:41:31 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:41:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:41:32.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:32 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-136"}]': finished 2024-03-21T13:41:32.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:32 smithi144 ceph-mon[33633]: osdmap e681: 8 total, 8 up, 8 in 2024-03-21T13:41:32.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:32 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-136", "mode": "writeback"}]: dispatch 2024-03-21T13:41:33.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:32 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-136"}]': finished 2024-03-21T13:41:33.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:32 smithi097 ceph-mon[30040]: osdmap e681: 8 total, 8 up, 8 in 2024-03-21T13:41:33.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:32 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-136", "mode": "writeback"}]: dispatch 2024-03-21T13:41:33.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:32 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-136"}]': finished 2024-03-21T13:41:33.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:32 smithi097 ceph-mon[39416]: osdmap e681: 8 total, 8 up, 8 in 2024-03-21T13:41:33.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:32 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-136", "mode": "writeback"}]: dispatch 2024-03-21T13:41:33.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:33 smithi144 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:41:33.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:33 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-136", "mode": "writeback"}]': finished 2024-03-21T13:41:33.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:33 smithi144 ceph-mon[33633]: osdmap e682: 8 total, 8 up, 8 in 2024-03-21T13:41:33.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:33 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:41:33.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:33 smithi144 ceph-mon[33633]: pgmap v1055: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 656 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:41:34.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:33 smithi097 ceph-mon[30040]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:41:34.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:33 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-136", "mode": "writeback"}]': finished 2024-03-21T13:41:34.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:33 smithi097 ceph-mon[30040]: osdmap e682: 8 total, 8 up, 8 in 2024-03-21T13:41:34.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:33 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:41:34.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:33 smithi097 ceph-mon[30040]: pgmap v1055: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 656 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:41:34.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:33 smithi097 ceph-mon[39416]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:41:34.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:33 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-136", "mode": "writeback"}]': finished 2024-03-21T13:41:34.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:33 smithi097 ceph-mon[39416]: osdmap e682: 8 total, 8 up, 8 in 2024-03-21T13:41:34.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:33 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:41:34.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:33 smithi097 ceph-mon[39416]: pgmap v1055: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 656 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:41:34.371 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:41:34 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:41:34.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:34 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:41:34.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:34 smithi144 ceph-mon[33633]: osdmap e683: 8 total, 8 up, 8 in 2024-03-21T13:41:34.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:34 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-136"}]: dispatch 2024-03-21T13:41:34.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:34 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:41:35.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:34 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:41:35.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:34 smithi097 ceph-mon[39416]: osdmap e683: 8 total, 8 up, 8 in 2024-03-21T13:41:35.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:34 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-136"}]: dispatch 2024-03-21T13:41:35.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:34 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:41:35.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:34 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:41:35.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:34 smithi097 ceph-mon[30040]: osdmap e683: 8 total, 8 up, 8 in 2024-03-21T13:41:35.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:34 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-136"}]: dispatch 2024-03-21T13:41:35.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:34 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:41:35.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:35 smithi144 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:41:35.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:35 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-136"}]': finished 2024-03-21T13:41:35.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:35 smithi144 ceph-mon[33633]: osdmap e684: 8 total, 8 up, 8 in 2024-03-21T13:41:35.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:35 smithi144 ceph-mon[33633]: pgmap v1058: 300 pgs: 300 active+clean; 4.4 MiB data, 688 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 1 op/s 2024-03-21T13:41:36.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:35 smithi097 ceph-mon[39416]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:41:36.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:35 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-136"}]': finished 2024-03-21T13:41:36.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:35 smithi097 ceph-mon[39416]: osdmap e684: 8 total, 8 up, 8 in 2024-03-21T13:41:36.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:35 smithi097 ceph-mon[39416]: pgmap v1058: 300 pgs: 300 active+clean; 4.4 MiB data, 688 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 1 op/s 2024-03-21T13:41:36.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:35 smithi097 ceph-mon[30040]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:41:36.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:35 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-136"}]': finished 2024-03-21T13:41:36.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:35 smithi097 ceph-mon[30040]: osdmap e684: 8 total, 8 up, 8 in 2024-03-21T13:41:36.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:35 smithi097 ceph-mon[30040]: pgmap v1058: 300 pgs: 300 active+clean; 4.4 MiB data, 688 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 1 op/s 2024-03-21T13:41:36.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:36 smithi144 ceph-mon[33633]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:41:36.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:36 smithi144 ceph-mon[33633]: osdmap e685: 8 total, 8 up, 8 in 2024-03-21T13:41:37.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:36 smithi097 ceph-mon[39416]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:41:37.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:36 smithi097 ceph-mon[39416]: osdmap e685: 8 total, 8 up, 8 in 2024-03-21T13:41:37.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:36 smithi097 ceph-mon[30040]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:41:37.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:36 smithi097 ceph-mon[30040]: osdmap e685: 8 total, 8 up, 8 in 2024-03-21T13:41:37.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:37 smithi144 ceph-mon[33633]: osdmap e686: 8 total, 8 up, 8 in 2024-03-21T13:41:37.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:37 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:41:37.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:37 smithi144 ceph-mon[33633]: pgmap v1061: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 688 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-03-21T13:41:38.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:37 smithi097 ceph-mon[39416]: osdmap e686: 8 total, 8 up, 8 in 2024-03-21T13:41:38.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:37 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:41:38.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:37 smithi097 ceph-mon[39416]: pgmap v1061: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 688 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-03-21T13:41:38.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:37 smithi097 ceph-mon[30040]: osdmap e686: 8 total, 8 up, 8 in 2024-03-21T13:41:38.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:37 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:41:38.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:37 smithi097 ceph-mon[30040]: pgmap v1061: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 688 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-03-21T13:41:38.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:38 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:41:38.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:38 smithi144 ceph-mon[33633]: osdmap e687: 8 total, 8 up, 8 in 2024-03-21T13:41:38.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:38 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:41:38.973 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] c 2024-03-21T13:41:38.973 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.973 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.973 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.973 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.973 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.973 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.973 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.973 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.973 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.973 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.974 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.974 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.974 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.974 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.974 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.974 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.974 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.974 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.974 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.974 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.974 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.974 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.974 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.974 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.975 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.975 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.975 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.975 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.975 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.975 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.975 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.975 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.975 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.975 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.975 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.975 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.975 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.975 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.976 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.976 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.976 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.976 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.976 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.976 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.976 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.976 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.976 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.976 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.976 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.976 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.976 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.976 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.976 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.977 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.977 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.977 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.977 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.977 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.977 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.977 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.977 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.977 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.977 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.977 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.977 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.977 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.977 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.977 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.978 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.978 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.978 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.978 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.978 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.978 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.978 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.978 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.978 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.978 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.978 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.978 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.978 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.978 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.978 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.979 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.979 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.979 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.979 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.979 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.979 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.979 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.979 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.979 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.979 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.979 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.979 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.979 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.979 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.979 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:38.980 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:41:39.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:38 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:41:39.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:38 smithi097 ceph-mon[39416]: osdmap e687: 8 total, 8 up, 8 in 2024-03-21T13:41:39.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:38 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:41:39.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:38 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:41:39.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:38 smithi097 ceph-mon[30040]: osdmap e687: 8 total, 8 up, 8 in 2024-03-21T13:41:39.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:38 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:41:39.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:39 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-138", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:41:39.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:39 smithi144 ceph-mon[33633]: osdmap e688: 8 total, 8 up, 8 in 2024-03-21T13:41:39.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:39 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-138","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:41:39.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:39 smithi144 ceph-mon[33633]: pgmap v1064: 300 pgs: 300 active+clean; 4.4 MiB data, 689 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:41:39.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:39 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-138","var": "hit_set_count","val": "2"}]': finished 2024-03-21T13:41:39.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:39 smithi144 ceph-mon[33633]: osdmap e689: 8 total, 8 up, 8 in 2024-03-21T13:41:39.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:39 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-138","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:41:40.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:39 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-138", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:41:40.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:39 smithi097 ceph-mon[39416]: osdmap e688: 8 total, 8 up, 8 in 2024-03-21T13:41:40.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:39 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-138","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:41:40.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:39 smithi097 ceph-mon[39416]: pgmap v1064: 300 pgs: 300 active+clean; 4.4 MiB data, 689 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:41:40.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:39 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-138","var": "hit_set_count","val": "2"}]': finished 2024-03-21T13:41:40.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:39 smithi097 ceph-mon[39416]: osdmap e689: 8 total, 8 up, 8 in 2024-03-21T13:41:40.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:39 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-138","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:41:40.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:39 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-138", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:41:40.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:39 smithi097 ceph-mon[30040]: osdmap e688: 8 total, 8 up, 8 in 2024-03-21T13:41:40.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:39 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-138","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:41:40.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:39 smithi097 ceph-mon[30040]: pgmap v1064: 300 pgs: 300 active+clean; 4.4 MiB data, 689 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:41:40.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:39 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-138","var": "hit_set_count","val": "2"}]': finished 2024-03-21T13:41:40.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:39 smithi097 ceph-mon[30040]: osdmap e689: 8 total, 8 up, 8 in 2024-03-21T13:41:40.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:39 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-138","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:41:41.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:41 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-138","var": "hit_set_period","val": "600"}]': finished 2024-03-21T13:41:41.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:41 smithi144 ceph-mon[33633]: osdmap e690: 8 total, 8 up, 8 in 2024-03-21T13:41:41.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:41 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-21T13:41:41.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:41 smithi144 ceph-mon[33633]: pgmap v1067: 300 pgs: 300 active+clean; 4.4 MiB data, 689 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:41:41.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:41 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:41:41.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:41 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:41:41.949 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:41 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-138","var": "hit_set_period","val": "600"}]': finished 2024-03-21T13:41:41.949 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:41 smithi097 ceph-mon[30040]: osdmap e690: 8 total, 8 up, 8 in 2024-03-21T13:41:41.949 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:41 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-21T13:41:41.950 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:41 smithi097 ceph-mon[30040]: pgmap v1067: 300 pgs: 300 active+clean; 4.4 MiB data, 689 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:41:41.950 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:41 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:41:41.950 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:41 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:41:41.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:41 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-138","var": "hit_set_period","val": "600"}]': finished 2024-03-21T13:41:41.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:41 smithi097 ceph-mon[39416]: osdmap e690: 8 total, 8 up, 8 in 2024-03-21T13:41:41.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:41 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-21T13:41:41.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:41 smithi097 ceph-mon[39416]: pgmap v1067: 300 pgs: 300 active+clean; 4.4 MiB data, 689 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:41:41.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:41 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:41:41.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:41 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:41:42.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:41:41 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:41:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:41:42.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:42 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-21T13:41:42.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:42 smithi144 ceph-mon[33633]: osdmap e691: 8 total, 8 up, 8 in 2024-03-21T13:41:42.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:42 smithi144 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:41:43.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:42 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-21T13:41:43.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:42 smithi097 ceph-mon[39416]: osdmap e691: 8 total, 8 up, 8 in 2024-03-21T13:41:43.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:42 smithi097 ceph-mon[39416]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:41:43.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:42 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-21T13:41:43.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:42 smithi097 ceph-mon[30040]: osdmap e691: 8 total, 8 up, 8 in 2024-03-21T13:41:43.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:42 smithi097 ceph-mon[30040]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:41:43.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:43 smithi144 ceph-mon[33633]: pgmap v1069: 300 pgs: 300 active+clean; 4.4 MiB data, 689 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:41:43.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:43 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:41:43.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:43 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-138"}]: dispatch 2024-03-21T13:41:44.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:43 smithi097 ceph-mon[39416]: pgmap v1069: 300 pgs: 300 active+clean; 4.4 MiB data, 689 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:41:44.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:43 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:41:44.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:43 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-138"}]: dispatch 2024-03-21T13:41:44.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:43 smithi097 ceph-mon[30040]: pgmap v1069: 300 pgs: 300 active+clean; 4.4 MiB data, 689 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:41:44.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:43 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:41:44.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:43 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-138"}]: dispatch 2024-03-21T13:41:44.371 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:41:44 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:41:45.272 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:44 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-138"}]': finished 2024-03-21T13:41:45.272 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:44 smithi144 ceph-mon[33633]: osdmap e692: 8 total, 8 up, 8 in 2024-03-21T13:41:45.272 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:44 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:41:45.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:44 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-138"}]': finished 2024-03-21T13:41:45.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:44 smithi097 ceph-mon[39416]: osdmap e692: 8 total, 8 up, 8 in 2024-03-21T13:41:45.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:44 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:41:45.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:44 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-138"}]': finished 2024-03-21T13:41:45.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:44 smithi097 ceph-mon[30040]: osdmap e692: 8 total, 8 up, 8 in 2024-03-21T13:41:45.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:44 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:41:46.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:45 smithi097 ceph-mon[39416]: osdmap e693: 8 total, 8 up, 8 in 2024-03-21T13:41:46.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:45 smithi097 ceph-mon[39416]: pgmap v1072: 268 pgs: 268 active+clean; 4.4 MiB data, 689 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:41:46.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:45 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:41:46.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:45 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:41:46.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:45 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:41:46.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:45 smithi097 ceph-mon[39416]: osdmap e694: 8 total, 8 up, 8 in 2024-03-21T13:41:46.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:45 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:41:46.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:45 smithi097 ceph-mon[30040]: osdmap e693: 8 total, 8 up, 8 in 2024-03-21T13:41:46.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:45 smithi097 ceph-mon[30040]: pgmap v1072: 268 pgs: 268 active+clean; 4.4 MiB data, 689 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:41:46.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:45 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:41:46.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:45 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:41:46.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:45 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:41:46.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:45 smithi097 ceph-mon[30040]: osdmap e694: 8 total, 8 up, 8 in 2024-03-21T13:41:46.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:45 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:41:46.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:45 smithi144 ceph-mon[33633]: osdmap e693: 8 total, 8 up, 8 in 2024-03-21T13:41:46.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:45 smithi144 ceph-mon[33633]: pgmap v1072: 268 pgs: 268 active+clean; 4.4 MiB data, 689 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:41:46.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:45 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:41:46.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:45 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:41:46.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:45 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:41:46.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:45 smithi144 ceph-mon[33633]: osdmap e694: 8 total, 8 up, 8 in 2024-03-21T13:41:46.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:45 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:41:47.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:46 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:41:47.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:46 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:41:47.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:46 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:41:47.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:46 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:41:47.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:46 smithi097 ceph-mon[39416]: osdmap e695: 8 total, 8 up, 8 in 2024-03-21T13:41:47.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:46 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:41:47.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:46 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:41:47.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:46 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:41:47.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:46 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:41:47.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:46 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:41:47.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:46 smithi097 ceph-mon[30040]: osdmap e695: 8 total, 8 up, 8 in 2024-03-21T13:41:47.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:46 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:41:47.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:46 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:41:47.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:46 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:41:47.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:46 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:41:47.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:46 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:41:47.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:46 smithi144 ceph-mon[33633]: osdmap e695: 8 total, 8 up, 8 in 2024-03-21T13:41:47.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:46 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:41:48.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:47 smithi097 ceph-mon[39416]: pgmap v1075: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 689 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:41:48.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:47 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-140", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:41:48.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:47 smithi097 ceph-mon[39416]: osdmap e696: 8 total, 8 up, 8 in 2024-03-21T13:41:48.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:47 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-140","var": "hit_set_count","val": "3"}]: dispatch 2024-03-21T13:41:48.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:47 smithi097 ceph-mon[30040]: pgmap v1075: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 689 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:41:48.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:47 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-140", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:41:48.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:47 smithi097 ceph-mon[30040]: osdmap e696: 8 total, 8 up, 8 in 2024-03-21T13:41:48.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:47 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-140","var": "hit_set_count","val": "3"}]: dispatch 2024-03-21T13:41:48.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:47 smithi144 ceph-mon[33633]: pgmap v1075: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 689 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:41:48.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:47 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-140", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:41:48.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:47 smithi144 ceph-mon[33633]: osdmap e696: 8 total, 8 up, 8 in 2024-03-21T13:41:48.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:47 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-140","var": "hit_set_count","val": "3"}]: dispatch 2024-03-21T13:41:50.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:49 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-140","var": "hit_set_count","val": "3"}]': finished 2024-03-21T13:41:50.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:49 smithi097 ceph-mon[39416]: osdmap e697: 8 total, 8 up, 8 in 2024-03-21T13:41:50.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:49 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-140","var": "hit_set_period","val": "3"}]: dispatch 2024-03-21T13:41:50.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:49 smithi097 ceph-mon[39416]: pgmap v1078: 300 pgs: 300 active+clean; 4.4 MiB data, 712 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:41:50.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:49 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-140","var": "hit_set_count","val": "3"}]': finished 2024-03-21T13:41:50.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:49 smithi097 ceph-mon[30040]: osdmap e697: 8 total, 8 up, 8 in 2024-03-21T13:41:50.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:49 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-140","var": "hit_set_period","val": "3"}]: dispatch 2024-03-21T13:41:50.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:49 smithi097 ceph-mon[30040]: pgmap v1078: 300 pgs: 300 active+clean; 4.4 MiB data, 712 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:41:50.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:49 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-140","var": "hit_set_count","val": "3"}]': finished 2024-03-21T13:41:50.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:49 smithi144 ceph-mon[33633]: osdmap e697: 8 total, 8 up, 8 in 2024-03-21T13:41:50.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:49 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-140","var": "hit_set_period","val": "3"}]: dispatch 2024-03-21T13:41:50.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:49 smithi144 ceph-mon[33633]: pgmap v1078: 300 pgs: 300 active+clean; 4.4 MiB data, 712 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:41:51.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:50 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-140","var": "hit_set_period","val": "3"}]': finished 2024-03-21T13:41:51.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:50 smithi097 ceph-mon[39416]: osdmap e698: 8 total, 8 up, 8 in 2024-03-21T13:41:51.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:50 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:41:51.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:50 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-140","var": "hit_set_period","val": "3"}]': finished 2024-03-21T13:41:51.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:50 smithi097 ceph-mon[30040]: osdmap e698: 8 total, 8 up, 8 in 2024-03-21T13:41:51.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:50 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:41:51.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:50 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-140","var": "hit_set_period","val": "3"}]': finished 2024-03-21T13:41:51.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:50 smithi144 ceph-mon[33633]: osdmap e698: 8 total, 8 up, 8 in 2024-03-21T13:41:51.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:50 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:41:51.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:51 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-140","var": "hit_set_type","val": "bloom"}]': finished 2024-03-21T13:41:51.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:51 smithi097 ceph-mon[39416]: osdmap e699: 8 total, 8 up, 8 in 2024-03-21T13:41:51.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:51 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-03-21T13:41:51.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:51 smithi097 ceph-mon[39416]: pgmap v1081: 300 pgs: 300 active+clean; 4.4 MiB data, 712 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:41:51.950 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:51 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-03-21T13:41:51.950 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:51 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-140","var": "hit_set_type","val": "bloom"}]': finished 2024-03-21T13:41:51.950 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:51 smithi097 ceph-mon[30040]: osdmap e699: 8 total, 8 up, 8 in 2024-03-21T13:41:51.950 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:51 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-03-21T13:41:51.951 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:51 smithi097 ceph-mon[30040]: pgmap v1081: 300 pgs: 300 active+clean; 4.4 MiB data, 712 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:41:51.951 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:51 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-03-21T13:41:52.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:51 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-140","var": "hit_set_type","val": "bloom"}]': finished 2024-03-21T13:41:52.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:51 smithi144 ceph-mon[33633]: osdmap e699: 8 total, 8 up, 8 in 2024-03-21T13:41:52.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:51 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-03-21T13:41:52.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:51 smithi144 ceph-mon[33633]: pgmap v1081: 300 pgs: 300 active+clean; 4.4 MiB data, 712 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:41:52.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:51 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-03-21T13:41:52.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:41:51 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:41:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:41:53.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:52 smithi097 ceph-mon[39416]: osdmap e700: 8 total, 8 up, 8 in 2024-03-21T13:41:53.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:52 smithi097 ceph-mon[30040]: osdmap e700: 8 total, 8 up, 8 in 2024-03-21T13:41:53.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:52 smithi144 ceph-mon[33633]: osdmap e700: 8 total, 8 up, 8 in 2024-03-21T13:41:54.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:53 smithi097 ceph-mon[39416]: pgmap v1083: 300 pgs: 300 active+clean; 4.4 MiB data, 712 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:41:54.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:53 smithi097 ceph-mon[30040]: pgmap v1083: 300 pgs: 300 active+clean; 4.4 MiB data, 712 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:41:54.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:53 smithi144 ceph-mon[33633]: pgmap v1083: 300 pgs: 300 active+clean; 4.4 MiB data, 712 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:41:54.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:41:54 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:41:55.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:54 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:41:55.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:54 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:41:55.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:54 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:41:56.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:55 smithi097 ceph-mon[39416]: pgmap v1084: 300 pgs: 300 active+clean; 4.4 MiB data, 712 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 2.7 KiB/s wr, 1 op/s 2024-03-21T13:41:56.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:55 smithi097 ceph-mon[30040]: pgmap v1084: 300 pgs: 300 active+clean; 4.4 MiB data, 712 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 2.7 KiB/s wr, 1 op/s 2024-03-21T13:41:56.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:55 smithi144 ceph-mon[33633]: pgmap v1084: 300 pgs: 300 active+clean; 4.4 MiB data, 712 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 2.7 KiB/s wr, 1 op/s 2024-03-21T13:41:57.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:56 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:41:57.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:56 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:41:57.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:56 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:41:58.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:57 smithi097 ceph-mon[39416]: pgmap v1085: 300 pgs: 300 active+clean; 4.4 MiB data, 712 MiB used, 715 GiB / 715 GiB avail; 730 B/s rd, 2.3 KiB/s wr, 0 op/s 2024-03-21T13:41:58.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:57 smithi097 ceph-mon[30040]: pgmap v1085: 300 pgs: 300 active+clean; 4.4 MiB data, 712 MiB used, 715 GiB / 715 GiB avail; 730 B/s rd, 2.3 KiB/s wr, 0 op/s 2024-03-21T13:41:58.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:57 smithi144 ceph-mon[33633]: pgmap v1085: 300 pgs: 300 active+clean; 4.4 MiB data, 712 MiB used, 715 GiB / 715 GiB avail; 730 B/s rd, 2.3 KiB/s wr, 0 op/s 2024-03-21T13:42:00.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:41:59 smithi097 ceph-mon[39416]: pgmap v1086: 300 pgs: 300 active+clean; 4.4 MiB data, 712 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 7.2 KiB/s wr, 2 op/s 2024-03-21T13:42:00.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:41:59 smithi097 ceph-mon[30040]: pgmap v1086: 300 pgs: 300 active+clean; 4.4 MiB data, 712 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 7.2 KiB/s wr, 2 op/s 2024-03-21T13:42:00.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:41:59 smithi144 ceph-mon[33633]: pgmap v1086: 300 pgs: 300 active+clean; 4.4 MiB data, 712 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 7.2 KiB/s wr, 2 op/s 2024-03-21T13:42:02.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:01 smithi097 ceph-mon[39416]: pgmap v1087: 300 pgs: 300 active+clean; 4.4 MiB data, 712 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 5.8 KiB/s wr, 1 op/s 2024-03-21T13:42:02.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:01 smithi097 ceph-mon[30040]: pgmap v1087: 300 pgs: 300 active+clean; 4.4 MiB data, 712 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 5.8 KiB/s wr, 1 op/s 2024-03-21T13:42:02.053 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:42:01 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:42:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:42:02.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:01 smithi144 ceph-mon[33633]: pgmap v1087: 300 pgs: 300 active+clean; 4.4 MiB data, 712 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 5.8 KiB/s wr, 1 op/s 2024-03-21T13:42:04.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:03 smithi097 ceph-mon[39416]: pgmap v1088: 300 pgs: 300 active+clean; 4.4 MiB data, 712 MiB used, 715 GiB / 715 GiB avail; 930 B/s rd, 5.3 KiB/s wr, 1 op/s 2024-03-21T13:42:04.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:03 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:42:04.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:03 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-140"}]: dispatch 2024-03-21T13:42:04.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:03 smithi097 ceph-mon[30040]: pgmap v1088: 300 pgs: 300 active+clean; 4.4 MiB data, 712 MiB used, 715 GiB / 715 GiB avail; 930 B/s rd, 5.3 KiB/s wr, 1 op/s 2024-03-21T13:42:04.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:03 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:42:04.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:03 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-140"}]: dispatch 2024-03-21T13:42:04.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:03 smithi144 ceph-mon[33633]: pgmap v1088: 300 pgs: 300 active+clean; 4.4 MiB data, 712 MiB used, 715 GiB / 715 GiB avail; 930 B/s rd, 5.3 KiB/s wr, 1 op/s 2024-03-21T13:42:04.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:03 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:42:04.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:03 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-140"}]: dispatch 2024-03-21T13:42:04.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:42:04 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:42:05.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:04 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-140"}]': finished 2024-03-21T13:42:05.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:04 smithi097 ceph-mon[39416]: osdmap e701: 8 total, 8 up, 8 in 2024-03-21T13:42:05.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:04 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:42:05.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:04 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-140"}]': finished 2024-03-21T13:42:05.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:04 smithi097 ceph-mon[30040]: osdmap e701: 8 total, 8 up, 8 in 2024-03-21T13:42:05.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:04 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:42:05.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:04 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-140"}]': finished 2024-03-21T13:42:05.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:04 smithi144 ceph-mon[33633]: osdmap e701: 8 total, 8 up, 8 in 2024-03-21T13:42:05.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:04 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:42:06.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:05 smithi097 ceph-mon[39416]: pgmap v1090: 300 pgs: 300 active+clean; 4.5 MiB data, 713 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 8.3 KiB/s wr, 2 op/s 2024-03-21T13:42:06.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:05 smithi097 ceph-mon[39416]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:42:06.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:05 smithi097 ceph-mon[39416]: osdmap e702: 8 total, 8 up, 8 in 2024-03-21T13:42:06.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:05 smithi097 ceph-mon[30040]: pgmap v1090: 300 pgs: 300 active+clean; 4.5 MiB data, 713 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 8.3 KiB/s wr, 2 op/s 2024-03-21T13:42:06.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:05 smithi097 ceph-mon[30040]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:42:06.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:05 smithi097 ceph-mon[30040]: osdmap e702: 8 total, 8 up, 8 in 2024-03-21T13:42:06.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:05 smithi144 ceph-mon[33633]: pgmap v1090: 300 pgs: 300 active+clean; 4.5 MiB data, 713 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 8.3 KiB/s wr, 2 op/s 2024-03-21T13:42:06.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:05 smithi144 ceph-mon[33633]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:42:06.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:05 smithi144 ceph-mon[33633]: osdmap e702: 8 total, 8 up, 8 in 2024-03-21T13:42:07.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:06 smithi097 ceph-mon[30040]: osdmap e703: 8 total, 8 up, 8 in 2024-03-21T13:42:07.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:06 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:42:07.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:06 smithi097 ceph-mon[39416]: osdmap e703: 8 total, 8 up, 8 in 2024-03-21T13:42:07.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:06 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:42:07.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:06 smithi144 ceph-mon[33633]: osdmap e703: 8 total, 8 up, 8 in 2024-03-21T13:42:07.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:06 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:42:08.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:07 smithi097 ceph-mon[39416]: pgmap v1093: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 713 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:42:08.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:07 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:42:08.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:07 smithi097 ceph-mon[39416]: osdmap e704: 8 total, 8 up, 8 in 2024-03-21T13:42:08.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:07 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:42:08.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:07 smithi097 ceph-mon[30040]: pgmap v1093: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 713 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:42:08.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:07 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:42:08.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:07 smithi097 ceph-mon[30040]: osdmap e704: 8 total, 8 up, 8 in 2024-03-21T13:42:08.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:07 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:42:08.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:07 smithi144 ceph-mon[33633]: pgmap v1093: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 713 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:42:08.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:07 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:42:08.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:07 smithi144 ceph-mon[33633]: osdmap e704: 8 total, 8 up, 8 in 2024-03-21T13:42:08.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:07 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:42:09.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:08 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-142", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:42:09.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:08 smithi097 ceph-mon[39416]: osdmap e705: 8 total, 8 up, 8 in 2024-03-21T13:42:09.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:08 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-142"}]: dispatch 2024-03-21T13:42:09.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:08 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-142", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:42:09.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:08 smithi097 ceph-mon[30040]: osdmap e705: 8 total, 8 up, 8 in 2024-03-21T13:42:09.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:08 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-142"}]: dispatch 2024-03-21T13:42:09.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:08 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-142", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:42:09.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:08 smithi144 ceph-mon[33633]: osdmap e705: 8 total, 8 up, 8 in 2024-03-21T13:42:09.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:08 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-142"}]: dispatch 2024-03-21T13:42:10.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:09 smithi097 ceph-mon[39416]: pgmap v1096: 300 pgs: 300 active+clean; 4.4 MiB data, 743 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-21T13:42:10.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:09 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-142"}]': finished 2024-03-21T13:42:10.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:09 smithi097 ceph-mon[39416]: osdmap e706: 8 total, 8 up, 8 in 2024-03-21T13:42:10.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:09 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-142", "mode": "writeback"}]: dispatch 2024-03-21T13:42:10.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:09 smithi097 ceph-mon[30040]: pgmap v1096: 300 pgs: 300 active+clean; 4.4 MiB data, 743 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-21T13:42:10.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:09 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-142"}]': finished 2024-03-21T13:42:10.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:09 smithi097 ceph-mon[30040]: osdmap e706: 8 total, 8 up, 8 in 2024-03-21T13:42:10.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:09 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-142", "mode": "writeback"}]: dispatch 2024-03-21T13:42:10.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:09 smithi144 ceph-mon[33633]: pgmap v1096: 300 pgs: 300 active+clean; 4.4 MiB data, 743 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-21T13:42:10.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:09 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-142"}]': finished 2024-03-21T13:42:10.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:09 smithi144 ceph-mon[33633]: osdmap e706: 8 total, 8 up, 8 in 2024-03-21T13:42:10.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:09 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-142", "mode": "writeback"}]: dispatch 2024-03-21T13:42:11.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:10 smithi097 ceph-mon[39416]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:42:11.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:10 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-142", "mode": "writeback"}]': finished 2024-03-21T13:42:11.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:10 smithi097 ceph-mon[39416]: osdmap e707: 8 total, 8 up, 8 in 2024-03-21T13:42:11.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:10 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-142","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:42:11.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:10 smithi097 ceph-mon[30040]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:42:11.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:10 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-142", "mode": "writeback"}]': finished 2024-03-21T13:42:11.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:10 smithi097 ceph-mon[30040]: osdmap e707: 8 total, 8 up, 8 in 2024-03-21T13:42:11.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:10 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-142","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:42:11.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:10 smithi144 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:42:11.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:10 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-142", "mode": "writeback"}]': finished 2024-03-21T13:42:11.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:10 smithi144 ceph-mon[33633]: osdmap e707: 8 total, 8 up, 8 in 2024-03-21T13:42:11.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:10 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-142","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:42:12.053 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:42:11 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:42:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:42:12.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:11 smithi097 ceph-mon[30040]: pgmap v1099: 300 pgs: 300 active+clean; 4.4 MiB data, 743 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-03-21T13:42:12.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:11 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-142","var": "hit_set_count","val": "2"}]': finished 2024-03-21T13:42:12.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:11 smithi097 ceph-mon[30040]: osdmap e708: 8 total, 8 up, 8 in 2024-03-21T13:42:12.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:11 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-142","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:42:12.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:11 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:42:12.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:11 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:42:12.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:11 smithi097 ceph-mon[39416]: pgmap v1099: 300 pgs: 300 active+clean; 4.4 MiB data, 743 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-03-21T13:42:12.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:11 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-142","var": "hit_set_count","val": "2"}]': finished 2024-03-21T13:42:12.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:11 smithi097 ceph-mon[39416]: osdmap e708: 8 total, 8 up, 8 in 2024-03-21T13:42:12.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:11 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-142","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:42:12.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:11 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:42:12.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:11 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:42:12.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:11 smithi144 ceph-mon[33633]: pgmap v1099: 300 pgs: 300 active+clean; 4.4 MiB data, 743 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-03-21T13:42:12.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:11 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-142","var": "hit_set_count","val": "2"}]': finished 2024-03-21T13:42:12.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:11 smithi144 ceph-mon[33633]: osdmap e708: 8 total, 8 up, 8 in 2024-03-21T13:42:12.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:11 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-142","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:42:12.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:11 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:42:12.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:11 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:42:13.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:12 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-142","var": "hit_set_period","val": "600"}]': finished 2024-03-21T13:42:13.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:12 smithi097 ceph-mon[39416]: osdmap e709: 8 total, 8 up, 8 in 2024-03-21T13:42:13.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:12 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:42:13.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:12 smithi097 ceph-mon[39416]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:42:13.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:12 smithi097 ceph-mon[39416]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:42:13.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:12 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-142","var": "hit_set_type","val": "bloom"}]': finished 2024-03-21T13:42:13.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:12 smithi097 ceph-mon[39416]: osdmap e710: 8 total, 8 up, 8 in 2024-03-21T13:42:13.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:12 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-21T13:42:13.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:12 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-142","var": "hit_set_period","val": "600"}]': finished 2024-03-21T13:42:13.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:12 smithi097 ceph-mon[30040]: osdmap e709: 8 total, 8 up, 8 in 2024-03-21T13:42:13.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:12 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:42:13.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:12 smithi097 ceph-mon[30040]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:42:13.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:12 smithi097 ceph-mon[30040]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:42:13.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:12 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-142","var": "hit_set_type","val": "bloom"}]': finished 2024-03-21T13:42:13.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:12 smithi097 ceph-mon[30040]: osdmap e710: 8 total, 8 up, 8 in 2024-03-21T13:42:13.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:12 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-21T13:42:13.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:12 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-142","var": "hit_set_period","val": "600"}]': finished 2024-03-21T13:42:13.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:12 smithi144 ceph-mon[33633]: osdmap e709: 8 total, 8 up, 8 in 2024-03-21T13:42:13.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:12 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:42:13.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:12 smithi144 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:42:13.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:12 smithi144 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:42:13.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:12 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-142","var": "hit_set_type","val": "bloom"}]': finished 2024-03-21T13:42:13.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:12 smithi144 ceph-mon[33633]: osdmap e710: 8 total, 8 up, 8 in 2024-03-21T13:42:13.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:12 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-21T13:42:14.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:13 smithi097 ceph-mon[39416]: pgmap v1102: 300 pgs: 300 active+clean; 4.4 MiB data, 743 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:42:14.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:13 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-21T13:42:14.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:13 smithi097 ceph-mon[39416]: osdmap e711: 8 total, 8 up, 8 in 2024-03-21T13:42:14.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:13 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-03-21T13:42:14.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:13 smithi097 ceph-mon[30040]: pgmap v1102: 300 pgs: 300 active+clean; 4.4 MiB data, 743 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:42:14.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:13 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-21T13:42:14.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:13 smithi097 ceph-mon[30040]: osdmap e711: 8 total, 8 up, 8 in 2024-03-21T13:42:14.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:13 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-03-21T13:42:14.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:13 smithi144 ceph-mon[33633]: pgmap v1102: 300 pgs: 300 active+clean; 4.4 MiB data, 743 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:42:14.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:13 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-21T13:42:14.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:13 smithi144 ceph-mon[33633]: osdmap e711: 8 total, 8 up, 8 in 2024-03-21T13:42:14.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:13 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-03-21T13:42:14.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:42:14 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:42:15.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:14 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:42:15.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:14 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-03-21T13:42:15.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:14 smithi097 ceph-mon[30040]: osdmap e712: 8 total, 8 up, 8 in 2024-03-21T13:42:15.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:14 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-03-21T13:42:15.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:14 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:42:15.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:14 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-03-21T13:42:15.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:14 smithi097 ceph-mon[39416]: osdmap e712: 8 total, 8 up, 8 in 2024-03-21T13:42:15.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:14 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-03-21T13:42:15.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:14 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:42:15.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:14 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-03-21T13:42:15.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:14 smithi144 ceph-mon[33633]: osdmap e712: 8 total, 8 up, 8 in 2024-03-21T13:42:15.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:14 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-03-21T13:42:16.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:15 smithi097 ceph-mon[39416]: pgmap v1105: 300 pgs: 300 active+clean; 4.4 MiB data, 743 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:42:16.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:15 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-03-21T13:42:16.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:15 smithi097 ceph-mon[39416]: osdmap e713: 8 total, 8 up, 8 in 2024-03-21T13:42:16.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:15 smithi097 ceph-mon[30040]: pgmap v1105: 300 pgs: 300 active+clean; 4.4 MiB data, 743 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:42:16.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:15 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-03-21T13:42:16.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:15 smithi097 ceph-mon[30040]: osdmap e713: 8 total, 8 up, 8 in 2024-03-21T13:42:16.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:15 smithi144 ceph-mon[33633]: pgmap v1105: 300 pgs: 300 active+clean; 4.4 MiB data, 743 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:42:16.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:15 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-03-21T13:42:16.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:15 smithi144 ceph-mon[33633]: osdmap e713: 8 total, 8 up, 8 in 2024-03-21T13:42:18.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:17 smithi144 ceph-mon[33633]: pgmap v1108: 300 pgs: 300 active+clean; 4.4 MiB data, 743 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:42:18.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:17 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:42:18.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:17 smithi097 ceph-mon[39416]: pgmap v1108: 300 pgs: 300 active+clean; 4.4 MiB data, 743 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:42:18.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:17 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:42:18.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:17 smithi097 ceph-mon[30040]: pgmap v1108: 300 pgs: 300 active+clean; 4.4 MiB data, 743 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:42:18.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:17 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:42:19.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:18 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:42:19.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:18 smithi144 ceph-mon[33633]: osdmap e714: 8 total, 8 up, 8 in 2024-03-21T13:42:19.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:18 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-142"}]: dispatch 2024-03-21T13:42:19.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:18 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:42:19.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:18 smithi097 ceph-mon[30040]: osdmap e714: 8 total, 8 up, 8 in 2024-03-21T13:42:19.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:18 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-142"}]: dispatch 2024-03-21T13:42:19.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:18 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:42:19.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:18 smithi097 ceph-mon[39416]: osdmap e714: 8 total, 8 up, 8 in 2024-03-21T13:42:19.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:18 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-142"}]: dispatch 2024-03-21T13:42:20.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:20 smithi144 ceph-mon[33633]: pgmap v1110: 300 pgs: 300 active+clean; 4.4 MiB data, 744 MiB used, 714 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-03-21T13:42:20.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:20 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-142"}]': finished 2024-03-21T13:42:20.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:20 smithi144 ceph-mon[33633]: osdmap e715: 8 total, 8 up, 8 in 2024-03-21T13:42:20.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:20 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:42:20.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:20 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-142"}]: dispatch 2024-03-21T13:42:20.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:20 smithi097 ceph-mon[30040]: pgmap v1110: 300 pgs: 300 active+clean; 4.4 MiB data, 744 MiB used, 714 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-03-21T13:42:20.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:20 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-142"}]': finished 2024-03-21T13:42:20.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:20 smithi097 ceph-mon[30040]: osdmap e715: 8 total, 8 up, 8 in 2024-03-21T13:42:20.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:20 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:42:20.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:20 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-142"}]: dispatch 2024-03-21T13:42:20.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:20 smithi097 ceph-mon[39416]: pgmap v1110: 300 pgs: 300 active+clean; 4.4 MiB data, 744 MiB used, 714 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-03-21T13:42:20.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:20 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-142"}]': finished 2024-03-21T13:42:20.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:20 smithi097 ceph-mon[39416]: osdmap e715: 8 total, 8 up, 8 in 2024-03-21T13:42:20.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:20 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:42:20.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:20 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-142"}]: dispatch 2024-03-21T13:42:21.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:21 smithi144 ceph-mon[33633]: osdmap e716: 8 total, 8 up, 8 in 2024-03-21T13:42:21.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:21 smithi144 ceph-mon[33633]: osdmap e717: 8 total, 8 up, 8 in 2024-03-21T13:42:21.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:21 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:42:21.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:21 smithi097 ceph-mon[39416]: osdmap e716: 8 total, 8 up, 8 in 2024-03-21T13:42:21.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:21 smithi097 ceph-mon[39416]: osdmap e717: 8 total, 8 up, 8 in 2024-03-21T13:42:21.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:21 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:42:21.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:21 smithi097 ceph-mon[30040]: osdmap e716: 8 total, 8 up, 8 in 2024-03-21T13:42:21.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:21 smithi097 ceph-mon[30040]: osdmap e717: 8 total, 8 up, 8 in 2024-03-21T13:42:21.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:21 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:42:22.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:22 smithi097 ceph-mon[30040]: pgmap v1113: 268 pgs: 268 active+clean; 4.4 MiB data, 744 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-03-21T13:42:22.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:22 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:42:22.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:22 smithi097 ceph-mon[30040]: osdmap e718: 8 total, 8 up, 8 in 2024-03-21T13:42:22.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:22 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:42:22.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:42:21 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:42:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:42:22.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:22 smithi097 ceph-mon[39416]: pgmap v1113: 268 pgs: 268 active+clean; 4.4 MiB data, 744 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-03-21T13:42:22.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:22 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:42:22.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:22 smithi097 ceph-mon[39416]: osdmap e718: 8 total, 8 up, 8 in 2024-03-21T13:42:22.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:22 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:42:22.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:22 smithi144 ceph-mon[33633]: pgmap v1113: 268 pgs: 268 active+clean; 4.4 MiB data, 744 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-03-21T13:42:22.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:22 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:42:22.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:22 smithi144 ceph-mon[33633]: osdmap e718: 8 total, 8 up, 8 in 2024-03-21T13:42:22.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:22 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:42:24.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:23 smithi144 ceph-mon[33633]: pgmap v1116: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 744 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:42:24.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:23 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-144", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:42:24.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:23 smithi144 ceph-mon[33633]: osdmap e719: 8 total, 8 up, 8 in 2024-03-21T13:42:24.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:23 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-144"}]: dispatch 2024-03-21T13:42:24.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:23 smithi097 ceph-mon[39416]: pgmap v1116: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 744 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:42:24.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:23 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-144", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:42:24.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:23 smithi097 ceph-mon[39416]: osdmap e719: 8 total, 8 up, 8 in 2024-03-21T13:42:24.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:23 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-144"}]: dispatch 2024-03-21T13:42:24.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:23 smithi097 ceph-mon[30040]: pgmap v1116: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 744 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:42:24.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:23 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-144", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:42:24.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:23 smithi097 ceph-mon[30040]: osdmap e719: 8 total, 8 up, 8 in 2024-03-21T13:42:24.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:23 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-144"}]: dispatch 2024-03-21T13:42:24.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:42:24 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:42:25.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:24 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-144"}]': finished 2024-03-21T13:42:25.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:24 smithi144 ceph-mon[33633]: osdmap e720: 8 total, 8 up, 8 in 2024-03-21T13:42:25.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:24 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-144", "mode": "readproxy"}]: dispatch 2024-03-21T13:42:25.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:24 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:42:25.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:24 smithi144 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:42:25.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:24 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-144", "mode": "readproxy"}]': finished 2024-03-21T13:42:25.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:24 smithi144 ceph-mon[33633]: osdmap e721: 8 total, 8 up, 8 in 2024-03-21T13:42:25.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:24 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-144"}]': finished 2024-03-21T13:42:25.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:24 smithi097 ceph-mon[39416]: osdmap e720: 8 total, 8 up, 8 in 2024-03-21T13:42:25.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:24 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-144", "mode": "readproxy"}]: dispatch 2024-03-21T13:42:25.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:24 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:42:25.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:24 smithi097 ceph-mon[39416]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:42:25.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:24 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-144", "mode": "readproxy"}]': finished 2024-03-21T13:42:25.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:24 smithi097 ceph-mon[39416]: osdmap e721: 8 total, 8 up, 8 in 2024-03-21T13:42:25.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:24 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-144"}]': finished 2024-03-21T13:42:25.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:24 smithi097 ceph-mon[30040]: osdmap e720: 8 total, 8 up, 8 in 2024-03-21T13:42:25.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:24 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-144", "mode": "readproxy"}]: dispatch 2024-03-21T13:42:25.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:24 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:42:25.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:24 smithi097 ceph-mon[30040]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:42:25.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:24 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-144", "mode": "readproxy"}]': finished 2024-03-21T13:42:25.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:24 smithi097 ceph-mon[30040]: osdmap e721: 8 total, 8 up, 8 in 2024-03-21T13:42:26.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:25 smithi144 ceph-mon[33633]: pgmap v1119: 300 pgs: 300 active+clean; 4.4 MiB data, 767 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:42:26.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:25 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:42:26.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:25 smithi097 ceph-mon[39416]: pgmap v1119: 300 pgs: 300 active+clean; 4.4 MiB data, 767 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:42:26.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:25 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:42:26.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:25 smithi097 ceph-mon[30040]: pgmap v1119: 300 pgs: 300 active+clean; 4.4 MiB data, 767 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:42:26.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:25 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:42:28.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:27 smithi144 ceph-mon[33633]: pgmap v1121: 300 pgs: 300 active+clean; 4.4 MiB data, 767 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 211 B/s wr, 1 op/s 2024-03-21T13:42:28.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:27 smithi097 ceph-mon[39416]: pgmap v1121: 300 pgs: 300 active+clean; 4.4 MiB data, 767 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 211 B/s wr, 1 op/s 2024-03-21T13:42:28.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:27 smithi097 ceph-mon[30040]: pgmap v1121: 300 pgs: 300 active+clean; 4.4 MiB data, 767 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 211 B/s wr, 1 op/s 2024-03-21T13:42:30.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:29 smithi144 ceph-mon[33633]: pgmap v1122: 300 pgs: 300 active+clean; 4.4 MiB data, 767 MiB used, 714 GiB / 715 GiB avail; 1.8 KiB/s rd, 170 B/s wr, 1 op/s 2024-03-21T13:42:30.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:29 smithi097 ceph-mon[30040]: pgmap v1122: 300 pgs: 300 active+clean; 4.4 MiB data, 767 MiB used, 714 GiB / 715 GiB avail; 1.8 KiB/s rd, 170 B/s wr, 1 op/s 2024-03-21T13:42:30.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:29 smithi097 ceph-mon[39416]: pgmap v1122: 300 pgs: 300 active+clean; 4.4 MiB data, 767 MiB used, 714 GiB / 715 GiB avail; 1.8 KiB/s rd, 170 B/s wr, 1 op/s 2024-03-21T13:42:32.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:31 smithi144 ceph-mon[33633]: pgmap v1123: 300 pgs: 300 active+clean; 4.4 MiB data, 767 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 130 B/s wr, 1 op/s 2024-03-21T13:42:32.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:31 smithi097 ceph-mon[30040]: pgmap v1123: 300 pgs: 300 active+clean; 4.4 MiB data, 767 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 130 B/s wr, 1 op/s 2024-03-21T13:42:32.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:42:31 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:42:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:42:32.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:31 smithi097 ceph-mon[39416]: pgmap v1123: 300 pgs: 300 active+clean; 4.4 MiB data, 767 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 130 B/s wr, 1 op/s 2024-03-21T13:42:34.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:33 smithi144 ceph-mon[33633]: pgmap v1124: 300 pgs: 300 active+clean; 4.4 MiB data, 767 MiB used, 714 GiB / 715 GiB avail; 694 B/s rd, 0 op/s 2024-03-21T13:42:34.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:33 smithi097 ceph-mon[30040]: pgmap v1124: 300 pgs: 300 active+clean; 4.4 MiB data, 767 MiB used, 714 GiB / 715 GiB avail; 694 B/s rd, 0 op/s 2024-03-21T13:42:34.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:33 smithi097 ceph-mon[39416]: pgmap v1124: 300 pgs: 300 active+clean; 4.4 MiB data, 767 MiB used, 714 GiB / 715 GiB avail; 694 B/s rd, 0 op/s 2024-03-21T13:42:34.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:42:34 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:42:35.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:34 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:42:35.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:34 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:42:35.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:34 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:42:36.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:35 smithi144 ceph-mon[33633]: pgmap v1125: 300 pgs: 300 active+clean; 4.4 MiB data, 767 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-21T13:42:36.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:35 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:42:36.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:35 smithi097 ceph-mon[30040]: pgmap v1125: 300 pgs: 300 active+clean; 4.4 MiB data, 767 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-21T13:42:36.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:35 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:42:36.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:35 smithi097 ceph-mon[39416]: pgmap v1125: 300 pgs: 300 active+clean; 4.4 MiB data, 767 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-21T13:42:36.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:35 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:42:37.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:36 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:42:37.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:36 smithi097 ceph-mon[39416]: osdmap e722: 8 total, 8 up, 8 in 2024-03-21T13:42:37.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:36 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-144"}]: dispatch 2024-03-21T13:42:37.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:36 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:42:37.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:36 smithi097 ceph-mon[30040]: osdmap e722: 8 total, 8 up, 8 in 2024-03-21T13:42:37.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:36 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-144"}]: dispatch 2024-03-21T13:42:37.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:36 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:42:37.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:36 smithi144 ceph-mon[33633]: osdmap e722: 8 total, 8 up, 8 in 2024-03-21T13:42:37.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:36 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-144"}]: dispatch 2024-03-21T13:42:38.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:38 smithi097 ceph-mon[39416]: pgmap v1127: 300 pgs: 300 active+clean; 4.4 MiB data, 767 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-21T13:42:38.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:38 smithi097 ceph-mon[39416]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:42:38.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:38 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-144"}]': finished 2024-03-21T13:42:38.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:38 smithi097 ceph-mon[39416]: osdmap e723: 8 total, 8 up, 8 in 2024-03-21T13:42:38.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:38 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:42:38.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:38 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-144"}]: dispatch 2024-03-21T13:42:38.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:38 smithi097 ceph-mon[30040]: pgmap v1127: 300 pgs: 300 active+clean; 4.4 MiB data, 767 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-21T13:42:38.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:38 smithi097 ceph-mon[30040]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:42:38.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:38 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-144"}]': finished 2024-03-21T13:42:38.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:38 smithi097 ceph-mon[30040]: osdmap e723: 8 total, 8 up, 8 in 2024-03-21T13:42:38.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:38 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:42:38.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:38 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-144"}]: dispatch 2024-03-21T13:42:38.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:38 smithi144 ceph-mon[33633]: pgmap v1127: 300 pgs: 300 active+clean; 4.4 MiB data, 767 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-21T13:42:38.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:38 smithi144 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:42:38.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:38 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-144"}]': finished 2024-03-21T13:42:38.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:38 smithi144 ceph-mon[33633]: osdmap e723: 8 total, 8 up, 8 in 2024-03-21T13:42:38.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:38 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:42:38.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:38 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-144"}]: dispatch 2024-03-21T13:42:39.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:39 smithi097 ceph-mon[39416]: osdmap e724: 8 total, 8 up, 8 in 2024-03-21T13:42:39.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:39 smithi097 ceph-mon[39416]: osdmap e725: 8 total, 8 up, 8 in 2024-03-21T13:42:39.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:39 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:42:39.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:39 smithi097 ceph-mon[30040]: osdmap e724: 8 total, 8 up, 8 in 2024-03-21T13:42:39.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:39 smithi097 ceph-mon[30040]: osdmap e725: 8 total, 8 up, 8 in 2024-03-21T13:42:39.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:39 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:42:39.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:39 smithi144 ceph-mon[33633]: osdmap e724: 8 total, 8 up, 8 in 2024-03-21T13:42:39.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:39 smithi144 ceph-mon[33633]: osdmap e725: 8 total, 8 up, 8 in 2024-03-21T13:42:39.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:39 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:42:40.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:40 smithi097 ceph-mon[39416]: pgmap v1130: 268 pgs: 268 active+clean; 4.4 MiB data, 767 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-21T13:42:40.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:40 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:42:40.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:40 smithi097 ceph-mon[39416]: osdmap e726: 8 total, 8 up, 8 in 2024-03-21T13:42:40.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:40 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:42:40.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:40 smithi097 ceph-mon[30040]: pgmap v1130: 268 pgs: 268 active+clean; 4.4 MiB data, 767 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-21T13:42:40.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:40 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:42:40.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:40 smithi097 ceph-mon[30040]: osdmap e726: 8 total, 8 up, 8 in 2024-03-21T13:42:40.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:40 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:42:40.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:40 smithi144 ceph-mon[33633]: pgmap v1130: 268 pgs: 268 active+clean; 4.4 MiB data, 767 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-21T13:42:40.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:40 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:42:40.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:40 smithi144 ceph-mon[33633]: osdmap e726: 8 total, 8 up, 8 in 2024-03-21T13:42:40.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:40 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:42:42.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:41 smithi144 ceph-mon[33633]: pgmap v1133: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 767 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:42:42.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:41 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:42:42.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:41 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:42:42.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:41 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-146", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:42:42.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:41 smithi144 ceph-mon[33633]: osdmap e727: 8 total, 8 up, 8 in 2024-03-21T13:42:42.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:41 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-146"}]: dispatch 2024-03-21T13:42:42.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:42:41 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:42:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:42:42.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:41 smithi097 ceph-mon[30040]: pgmap v1133: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 767 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:42:42.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:41 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:42:42.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:41 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:42:42.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:41 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-146", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:42:42.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:41 smithi097 ceph-mon[30040]: osdmap e727: 8 total, 8 up, 8 in 2024-03-21T13:42:42.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:41 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-146"}]: dispatch 2024-03-21T13:42:42.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:41 smithi097 ceph-mon[39416]: pgmap v1133: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 767 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:42:42.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:41 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:42:42.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:41 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:42:42.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:41 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-146", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:42:42.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:41 smithi097 ceph-mon[39416]: osdmap e727: 8 total, 8 up, 8 in 2024-03-21T13:42:42.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:41 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-146"}]: dispatch 2024-03-21T13:42:43.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:42 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-146"}]': finished 2024-03-21T13:42:43.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:42 smithi097 ceph-mon[39416]: osdmap e728: 8 total, 8 up, 8 in 2024-03-21T13:42:43.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:42 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-146", "mode": "writeback"}]: dispatch 2024-03-21T13:42:43.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:42 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-146"}]': finished 2024-03-21T13:42:43.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:42 smithi097 ceph-mon[30040]: osdmap e728: 8 total, 8 up, 8 in 2024-03-21T13:42:43.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:42 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-146", "mode": "writeback"}]: dispatch 2024-03-21T13:42:43.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:42 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi097-79906-111", "overlaypool": "test-rados-api-smithi097-79906-146"}]': finished 2024-03-21T13:42:43.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:42 smithi144 ceph-mon[33633]: osdmap e728: 8 total, 8 up, 8 in 2024-03-21T13:42:43.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:42 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-146", "mode": "writeback"}]: dispatch 2024-03-21T13:42:44.181 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:43 smithi144 ceph-mon[33633]: pgmap v1136: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 767 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:42:44.181 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:43 smithi144 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:42:44.181 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:43 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-146", "mode": "writeback"}]': finished 2024-03-21T13:42:44.181 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:43 smithi144 ceph-mon[33633]: osdmap e729: 8 total, 8 up, 8 in 2024-03-21T13:42:44.181 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:43 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-146","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:42:44.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:43 smithi097 ceph-mon[39416]: pgmap v1136: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 767 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:42:44.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:43 smithi097 ceph-mon[39416]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:42:44.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:43 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-146", "mode": "writeback"}]': finished 2024-03-21T13:42:44.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:43 smithi097 ceph-mon[39416]: osdmap e729: 8 total, 8 up, 8 in 2024-03-21T13:42:44.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:43 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-146","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:42:44.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:43 smithi097 ceph-mon[30040]: pgmap v1136: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 767 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:42:44.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:43 smithi097 ceph-mon[30040]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:42:44.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:43 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi097-79906-146", "mode": "writeback"}]': finished 2024-03-21T13:42:44.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:43 smithi097 ceph-mon[30040]: osdmap e729: 8 total, 8 up, 8 in 2024-03-21T13:42:44.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:43 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-146","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:42:44.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:42:44 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:42:45.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:44 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-146","var": "hit_set_count","val": "2"}]': finished 2024-03-21T13:42:45.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:44 smithi097 ceph-mon[30040]: osdmap e730: 8 total, 8 up, 8 in 2024-03-21T13:42:45.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:44 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-146","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:42:45.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:44 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:42:45.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:44 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-146","var": "hit_set_count","val": "2"}]': finished 2024-03-21T13:42:45.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:44 smithi097 ceph-mon[39416]: osdmap e730: 8 total, 8 up, 8 in 2024-03-21T13:42:45.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:44 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-146","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:42:45.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:44 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:42:45.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:44 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-146","var": "hit_set_count","val": "2"}]': finished 2024-03-21T13:42:45.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:44 smithi144 ceph-mon[33633]: osdmap e730: 8 total, 8 up, 8 in 2024-03-21T13:42:45.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:44 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-146","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:42:45.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:44 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:42:46.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:45 smithi097 ceph-mon[30040]: pgmap v1139: 300 pgs: 300 active+clean; 4.4 MiB data, 772 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-03-21T13:42:46.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:45 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-146","var": "hit_set_period","val": "600"}]': finished 2024-03-21T13:42:46.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:45 smithi097 ceph-mon[30040]: osdmap e731: 8 total, 8 up, 8 in 2024-03-21T13:42:46.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:45 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:42:46.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:45 smithi097 ceph-mon[39416]: pgmap v1139: 300 pgs: 300 active+clean; 4.4 MiB data, 772 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-03-21T13:42:46.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:45 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-146","var": "hit_set_period","val": "600"}]': finished 2024-03-21T13:42:46.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:45 smithi097 ceph-mon[39416]: osdmap e731: 8 total, 8 up, 8 in 2024-03-21T13:42:46.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:45 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:42:46.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:45 smithi144 ceph-mon[33633]: pgmap v1139: 300 pgs: 300 active+clean; 4.4 MiB data, 772 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-03-21T13:42:46.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:45 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-146","var": "hit_set_period","val": "600"}]': finished 2024-03-21T13:42:46.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:45 smithi144 ceph-mon[33633]: osdmap e731: 8 total, 8 up, 8 in 2024-03-21T13:42:46.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:45 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:42:47.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:46 smithi097 ceph-mon[39416]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:42:47.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:46 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-146","var": "hit_set_type","val": "bloom"}]': finished 2024-03-21T13:42:47.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:46 smithi097 ceph-mon[39416]: osdmap e732: 8 total, 8 up, 8 in 2024-03-21T13:42:47.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:46 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-21T13:42:47.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:46 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:42:47.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:46 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:42:47.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:46 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:42:47.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:46 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-21T13:42:47.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:46 smithi097 ceph-mon[30040]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:42:47.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:46 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-146","var": "hit_set_type","val": "bloom"}]': finished 2024-03-21T13:42:47.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:46 smithi097 ceph-mon[30040]: osdmap e732: 8 total, 8 up, 8 in 2024-03-21T13:42:47.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:46 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-21T13:42:47.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:46 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:42:47.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:46 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:42:47.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:46 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:42:47.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:46 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-21T13:42:47.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:46 smithi144 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:42:47.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:46 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-146","var": "hit_set_type","val": "bloom"}]': finished 2024-03-21T13:42:47.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:46 smithi144 ceph-mon[33633]: osdmap e732: 8 total, 8 up, 8 in 2024-03-21T13:42:47.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:46 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-21T13:42:47.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:46 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:42:47.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:46 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:42:47.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:46 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:42:47.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:46 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-21T13:42:48.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:47 smithi097 ceph-mon[39416]: pgmap v1142: 300 pgs: 300 active+clean; 4.4 MiB data, 772 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-03-21T13:42:48.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:47 smithi097 ceph-mon[39416]: osdmap e733: 8 total, 8 up, 8 in 2024-03-21T13:42:48.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:47 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-146","var": "target_max_objects","val": "1"}]: dispatch 2024-03-21T13:42:48.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:47 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:42:48.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:47 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:42:48.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:47 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:42:48.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:47 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-146","var": "target_max_objects","val": "1"}]': finished 2024-03-21T13:42:48.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:47 smithi097 ceph-mon[39416]: osdmap e734: 8 total, 8 up, 8 in 2024-03-21T13:42:48.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:47 smithi097 ceph-mon[30040]: pgmap v1142: 300 pgs: 300 active+clean; 4.4 MiB data, 772 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-03-21T13:42:48.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:47 smithi097 ceph-mon[30040]: osdmap e733: 8 total, 8 up, 8 in 2024-03-21T13:42:48.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:47 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-146","var": "target_max_objects","val": "1"}]: dispatch 2024-03-21T13:42:48.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:47 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:42:48.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:47 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:42:48.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:47 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:42:48.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:47 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-146","var": "target_max_objects","val": "1"}]': finished 2024-03-21T13:42:48.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:47 smithi097 ceph-mon[30040]: osdmap e734: 8 total, 8 up, 8 in 2024-03-21T13:42:48.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:47 smithi144 ceph-mon[33633]: pgmap v1142: 300 pgs: 300 active+clean; 4.4 MiB data, 772 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-03-21T13:42:48.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:47 smithi144 ceph-mon[33633]: osdmap e733: 8 total, 8 up, 8 in 2024-03-21T13:42:48.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:47 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-146","var": "target_max_objects","val": "1"}]: dispatch 2024-03-21T13:42:48.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:47 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:42:48.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:47 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:42:48.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:47 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:42:48.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:47 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-146","var": "target_max_objects","val": "1"}]': finished 2024-03-21T13:42:48.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:47 smithi144 ceph-mon[33633]: osdmap e734: 8 total, 8 up, 8 in 2024-03-21T13:42:49.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:48 smithi097 ceph-mon[30040]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-03-21T13:42:49.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:48 smithi097 ceph-mon[39416]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-03-21T13:42:49.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:48 smithi144 ceph-mon[33633]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-03-21T13:42:50.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:49 smithi097 ceph-mon[39416]: pgmap v1145: 300 pgs: 300 active+clean; 4.4 MiB data, 773 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:42:50.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:49 smithi097 ceph-mon[30040]: pgmap v1145: 300 pgs: 300 active+clean; 4.4 MiB data, 773 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:42:50.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:49 smithi144 ceph-mon[33633]: pgmap v1145: 300 pgs: 300 active+clean; 4.4 MiB data, 773 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:42:52.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:51 smithi097 ceph-mon[39416]: pgmap v1146: 300 pgs: 300 active+clean; 4.4 MiB data, 773 MiB used, 714 GiB / 715 GiB avail; 887 B/s rd, 0 op/s 2024-03-21T13:42:52.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:51 smithi097 ceph-mon[30040]: pgmap v1146: 300 pgs: 300 active+clean; 4.4 MiB data, 773 MiB used, 714 GiB / 715 GiB avail; 887 B/s rd, 0 op/s 2024-03-21T13:42:52.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:42:51 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:42:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:42:52.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:51 smithi144 ceph-mon[33633]: pgmap v1146: 300 pgs: 300 active+clean; 4.4 MiB data, 773 MiB used, 714 GiB / 715 GiB avail; 887 B/s rd, 0 op/s 2024-03-21T13:42:54.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:53 smithi097 ceph-mon[39416]: pgmap v1147: 300 pgs: 300 active+clean; 4.4 MiB data, 773 MiB used, 714 GiB / 715 GiB avail; 757 B/s rd, 0 op/s 2024-03-21T13:42:54.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:53 smithi097 ceph-mon[30040]: pgmap v1147: 300 pgs: 300 active+clean; 4.4 MiB data, 773 MiB used, 714 GiB / 715 GiB avail; 757 B/s rd, 0 op/s 2024-03-21T13:42:54.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:53 smithi144 ceph-mon[33633]: pgmap v1147: 300 pgs: 300 active+clean; 4.4 MiB data, 773 MiB used, 714 GiB / 715 GiB avail; 757 B/s rd, 0 op/s 2024-03-21T13:42:54.371 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:42:54 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:42:54.371 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:42:54 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=sqlstore.transactions t=2024-03-21T13:42:54.047689159Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-03-21T13:42:55.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:55 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:42:55.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:55 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:42:55.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:55 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:42:56.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:56 smithi097 ceph-mon[39416]: pgmap v1148: 300 pgs: 300 active+clean; 4.4 MiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:42:56.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:56 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:42:56.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:56 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:42:56.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:56 smithi097 ceph-mon[30040]: pgmap v1148: 300 pgs: 300 active+clean; 4.4 MiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:42:56.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:56 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:42:56.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:56 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:42:56.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:56 smithi144 ceph-mon[33633]: pgmap v1148: 300 pgs: 300 active+clean; 4.4 MiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:42:56.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:56 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:42:56.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:56 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:42:58.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:58 smithi097 ceph-mon[39416]: pgmap v1149: 300 pgs: 300 active+clean; 4.4 MiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:42:58.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:58 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:42:58.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:58 smithi097 ceph-mon[30040]: pgmap v1149: 300 pgs: 300 active+clean; 4.4 MiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:42:58.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:58 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:42:58.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:58 smithi144 ceph-mon[33633]: pgmap v1149: 300 pgs: 300 active+clean; 4.4 MiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:42:58.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:58 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:42:59.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:59 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:42:59.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:59 smithi097 ceph-mon[39416]: osdmap e735: 8 total, 8 up, 8 in 2024-03-21T13:42:59.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:42:59 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-146"}]: dispatch 2024-03-21T13:42:59.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:59 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:42:59.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:59 smithi097 ceph-mon[30040]: osdmap e735: 8 total, 8 up, 8 in 2024-03-21T13:42:59.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:42:59 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-146"}]: dispatch 2024-03-21T13:42:59.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:59 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:42:59.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:59 smithi144 ceph-mon[33633]: osdmap e735: 8 total, 8 up, 8 in 2024-03-21T13:42:59.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:42:59 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-146"}]: dispatch 2024-03-21T13:43:00.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:00 smithi097 ceph-mon[30040]: pgmap v1151: 300 pgs: 300 active+clean; 4.4 MiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:43:00.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:00 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-146"}]': finished 2024-03-21T13:43:00.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:00 smithi097 ceph-mon[30040]: osdmap e736: 8 total, 8 up, 8 in 2024-03-21T13:43:00.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:00 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:43:00.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:00 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-146"}]: dispatch 2024-03-21T13:43:00.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:00 smithi097 ceph-mon[39416]: pgmap v1151: 300 pgs: 300 active+clean; 4.4 MiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:43:00.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:00 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-146"}]': finished 2024-03-21T13:43:00.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:00 smithi097 ceph-mon[39416]: osdmap e736: 8 total, 8 up, 8 in 2024-03-21T13:43:00.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:00 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:43:00.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:00 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-146"}]: dispatch 2024-03-21T13:43:00.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:00 smithi144 ceph-mon[33633]: pgmap v1151: 300 pgs: 300 active+clean; 4.4 MiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:43:00.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:00 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-146"}]': finished 2024-03-21T13:43:00.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:00 smithi144 ceph-mon[33633]: osdmap e736: 8 total, 8 up, 8 in 2024-03-21T13:43:00.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:00 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:43:00.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:00 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-146"}]: dispatch 2024-03-21T13:43:01.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:01 smithi097 ceph-mon[39416]: osdmap e737: 8 total, 8 up, 8 in 2024-03-21T13:43:01.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:01 smithi097 ceph-mon[30040]: osdmap e737: 8 total, 8 up, 8 in 2024-03-21T13:43:01.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:01 smithi144 ceph-mon[33633]: osdmap e737: 8 total, 8 up, 8 in 2024-03-21T13:43:02.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:43:01 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:43:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:43:02.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:02 smithi097 ceph-mon[39416]: pgmap v1154: 268 pgs: 268 active+clean; 4.4 MiB data, 790 MiB used, 714 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-03-21T13:43:02.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:02 smithi097 ceph-mon[39416]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-03-21T13:43:02.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:02 smithi097 ceph-mon[39416]: osdmap e738: 8 total, 8 up, 8 in 2024-03-21T13:43:02.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:02 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:43:02.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:02 smithi097 ceph-mon[30040]: pgmap v1154: 268 pgs: 268 active+clean; 4.4 MiB data, 790 MiB used, 714 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-03-21T13:43:02.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:02 smithi097 ceph-mon[30040]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-03-21T13:43:02.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:02 smithi097 ceph-mon[30040]: osdmap e738: 8 total, 8 up, 8 in 2024-03-21T13:43:02.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:02 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:43:02.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:02 smithi144 ceph-mon[33633]: pgmap v1154: 268 pgs: 268 active+clean; 4.4 MiB data, 790 MiB used, 714 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-03-21T13:43:02.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:02 smithi144 ceph-mon[33633]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-03-21T13:43:02.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:02 smithi144 ceph-mon[33633]: osdmap e738: 8 total, 8 up, 8 in 2024-03-21T13:43:02.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:02 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:43:03.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:03 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:43:03.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:03 smithi097 ceph-mon[39416]: osdmap e739: 8 total, 8 up, 8 in 2024-03-21T13:43:03.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:03 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:43:03.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:03 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:43:03.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:03 smithi097 ceph-mon[30040]: osdmap e739: 8 total, 8 up, 8 in 2024-03-21T13:43:03.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:03 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:43:03.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:03 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:43:03.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:03 smithi144 ceph-mon[33633]: osdmap e739: 8 total, 8 up, 8 in 2024-03-21T13:43:03.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:03 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:43:04.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:04 smithi097 ceph-mon[39416]: pgmap v1157: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 790 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:43:04.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:04 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-148", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:43:04.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:04 smithi097 ceph-mon[39416]: osdmap e740: 8 total, 8 up, 8 in 2024-03-21T13:43:04.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:04 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-148"}]: dispatch 2024-03-21T13:43:04.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:04 smithi097 ceph-mon[30040]: pgmap v1157: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 790 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:43:04.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:04 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-148", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:43:04.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:04 smithi097 ceph-mon[30040]: osdmap e740: 8 total, 8 up, 8 in 2024-03-21T13:43:04.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:04 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-148"}]: dispatch 2024-03-21T13:43:04.371 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:43:04 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:43:04.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:04 smithi144 ceph-mon[33633]: pgmap v1157: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 790 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:43:04.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:04 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-148", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:43:04.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:04 smithi144 ceph-mon[33633]: osdmap e740: 8 total, 8 up, 8 in 2024-03-21T13:43:04.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:04 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-148"}]: dispatch 2024-03-21T13:43:05.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:05 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-148"}]': finished 2024-03-21T13:43:05.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:05 smithi144 ceph-mon[33633]: osdmap e741: 8 total, 8 up, 8 in 2024-03-21T13:43:05.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:05 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:43:05.373 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:05 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-148"}]: dispatch 2024-03-21T13:43:05.373 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:05 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:43:05.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:05 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-148"}]': finished 2024-03-21T13:43:05.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:05 smithi097 ceph-mon[30040]: osdmap e741: 8 total, 8 up, 8 in 2024-03-21T13:43:05.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:05 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:43:05.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:05 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-148"}]: dispatch 2024-03-21T13:43:05.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:05 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:43:05.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:05 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-148"}]': finished 2024-03-21T13:43:05.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:05 smithi097 ceph-mon[39416]: osdmap e741: 8 total, 8 up, 8 in 2024-03-21T13:43:05.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:05 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:43:05.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:05 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-148"}]: dispatch 2024-03-21T13:43:05.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:05 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:43:06.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:06 smithi144 ceph-mon[33633]: pgmap v1160: 300 pgs: 300 active+clean; 4.4 MiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-21T13:43:06.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:06 smithi144 ceph-mon[33633]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:43:06.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:06 smithi144 ceph-mon[33633]: osdmap e742: 8 total, 8 up, 8 in 2024-03-21T13:43:06.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:06 smithi097 ceph-mon[39416]: pgmap v1160: 300 pgs: 300 active+clean; 4.4 MiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-21T13:43:06.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:06 smithi097 ceph-mon[39416]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:43:06.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:06 smithi097 ceph-mon[39416]: osdmap e742: 8 total, 8 up, 8 in 2024-03-21T13:43:06.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:06 smithi097 ceph-mon[30040]: pgmap v1160: 300 pgs: 300 active+clean; 4.4 MiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-21T13:43:06.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:06 smithi097 ceph-mon[30040]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:43:06.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:06 smithi097 ceph-mon[30040]: osdmap e742: 8 total, 8 up, 8 in 2024-03-21T13:43:07.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:07 smithi144 ceph-mon[33633]: osdmap e743: 8 total, 8 up, 8 in 2024-03-21T13:43:07.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:07 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:43:07.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:07 smithi097 ceph-mon[39416]: osdmap e743: 8 total, 8 up, 8 in 2024-03-21T13:43:07.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:07 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:43:07.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:07 smithi097 ceph-mon[30040]: osdmap e743: 8 total, 8 up, 8 in 2024-03-21T13:43:07.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:07 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:43:08.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:08 smithi144 ceph-mon[33633]: pgmap v1163: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T13:43:08.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:08 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:43:08.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:08 smithi144 ceph-mon[33633]: osdmap e744: 8 total, 8 up, 8 in 2024-03-21T13:43:08.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:08 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:43:08.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:08 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-150"}]: dispatch 2024-03-21T13:43:08.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:08 smithi097 ceph-mon[39416]: pgmap v1163: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T13:43:08.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:08 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:43:08.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:08 smithi097 ceph-mon[39416]: osdmap e744: 8 total, 8 up, 8 in 2024-03-21T13:43:08.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:08 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:43:08.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:08 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-150"}]: dispatch 2024-03-21T13:43:08.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:08 smithi097 ceph-mon[30040]: pgmap v1163: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T13:43:08.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:08 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:43:08.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:08 smithi097 ceph-mon[30040]: osdmap e744: 8 total, 8 up, 8 in 2024-03-21T13:43:08.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:08 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:43:08.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:08 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-150"}]: dispatch 2024-03-21T13:43:09.374 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:09 smithi144 ceph-mon[33633]: osdmap e745: 8 total, 8 up, 8 in 2024-03-21T13:43:09.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:09 smithi097 ceph-mon[39416]: osdmap e745: 8 total, 8 up, 8 in 2024-03-21T13:43:09.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:09 smithi097 ceph-mon[30040]: osdmap e745: 8 total, 8 up, 8 in 2024-03-21T13:43:10.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:10 smithi144 ceph-mon[33633]: pgmap v1166: 268 pgs: 268 active+clean; 4.4 MiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-21T13:43:10.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:10 smithi144 ceph-mon[33633]: osdmap e746: 8 total, 8 up, 8 in 2024-03-21T13:43:10.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:10 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:43:10.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:10 smithi097 ceph-mon[39416]: pgmap v1166: 268 pgs: 268 active+clean; 4.4 MiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-21T13:43:10.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:10 smithi097 ceph-mon[39416]: osdmap e746: 8 total, 8 up, 8 in 2024-03-21T13:43:10.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:10 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:43:10.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:10 smithi097 ceph-mon[30040]: pgmap v1166: 268 pgs: 268 active+clean; 4.4 MiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-21T13:43:10.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:10 smithi097 ceph-mon[30040]: osdmap e746: 8 total, 8 up, 8 in 2024-03-21T13:43:10.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:10 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:43:11.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:11 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:43:11.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:11 smithi144 ceph-mon[33633]: osdmap e747: 8 total, 8 up, 8 in 2024-03-21T13:43:11.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:11 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:43:11.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:11 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-152"}]: dispatch 2024-03-21T13:43:11.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:11 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:43:11.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:11 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:43:11.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:11 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:43:11.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:11 smithi097 ceph-mon[39416]: osdmap e747: 8 total, 8 up, 8 in 2024-03-21T13:43:11.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:11 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:43:11.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:11 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-152"}]: dispatch 2024-03-21T13:43:11.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:11 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:43:11.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:11 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:43:11.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:11 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:43:11.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:11 smithi097 ceph-mon[30040]: osdmap e747: 8 total, 8 up, 8 in 2024-03-21T13:43:11.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:11 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:43:11.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:11 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-152"}]: dispatch 2024-03-21T13:43:11.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:11 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:43:11.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:11 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:43:12.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:12 smithi097 ceph-mon[39416]: pgmap v1169: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-21T13:43:12.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:12 smithi097 ceph-mon[39416]: osdmap e748: 8 total, 8 up, 8 in 2024-03-21T13:43:12.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:12 smithi097 ceph-mon[30040]: pgmap v1169: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-21T13:43:12.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:12 smithi097 ceph-mon[30040]: osdmap e748: 8 total, 8 up, 8 in 2024-03-21T13:43:12.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:43:11 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:43:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:43:12.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:12 smithi144 ceph-mon[33633]: pgmap v1169: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-21T13:43:12.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:12 smithi144 ceph-mon[33633]: osdmap e748: 8 total, 8 up, 8 in 2024-03-21T13:43:13.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:13 smithi097 ceph-mon[39416]: osdmap e749: 8 total, 8 up, 8 in 2024-03-21T13:43:13.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:13 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:43:13.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:13 smithi097 ceph-mon[39416]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:43:13.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:13 smithi097 ceph-mon[30040]: osdmap e749: 8 total, 8 up, 8 in 2024-03-21T13:43:13.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:13 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:43:13.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:13 smithi097 ceph-mon[30040]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:43:13.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:13 smithi144 ceph-mon[33633]: osdmap e749: 8 total, 8 up, 8 in 2024-03-21T13:43:13.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:13 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:43:13.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:13 smithi144 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:43:14.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:14 smithi097 ceph-mon[30040]: pgmap v1172: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 809 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:43:14.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:14 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:43:14.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:14 smithi097 ceph-mon[30040]: osdmap e750: 8 total, 8 up, 8 in 2024-03-21T13:43:14.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:14 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-111","var": "dedup_tier","val": "test-rados-api-smithi097-79906-154"}]: dispatch 2024-03-21T13:43:14.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:14 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:43:14.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:14 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-154"}]: dispatch 2024-03-21T13:43:14.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:14 smithi097 ceph-mon[39416]: pgmap v1172: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 809 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:43:14.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:14 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:43:14.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:14 smithi097 ceph-mon[39416]: osdmap e750: 8 total, 8 up, 8 in 2024-03-21T13:43:14.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:14 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-111","var": "dedup_tier","val": "test-rados-api-smithi097-79906-154"}]: dispatch 2024-03-21T13:43:14.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:14 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:43:14.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:14 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-154"}]: dispatch 2024-03-21T13:43:14.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:43:14 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:43:14.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:14 smithi144 ceph-mon[33633]: pgmap v1172: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 809 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:43:14.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:14 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi097-79906-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:43:14.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:14 smithi144 ceph-mon[33633]: osdmap e750: 8 total, 8 up, 8 in 2024-03-21T13:43:14.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:14 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi097-79906-111","var": "dedup_tier","val": "test-rados-api-smithi097-79906-154"}]: dispatch 2024-03-21T13:43:14.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:14 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:43:14.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:14 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi097-79906-111", "tierpool": "test-rados-api-smithi097-79906-154"}]: dispatch 2024-03-21T13:43:15.522 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:15 smithi144 ceph-mon[33633]: osdmap e751: 8 total, 8 up, 8 in 2024-03-21T13:43:15.523 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:15 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:43:15.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:15 smithi097 ceph-mon[39416]: osdmap e751: 8 total, 8 up, 8 in 2024-03-21T13:43:15.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:15 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:43:15.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:15 smithi097 ceph-mon[30040]: osdmap e751: 8 total, 8 up, 8 in 2024-03-21T13:43:15.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:15 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:43:16.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:16 smithi097 ceph-mon[39416]: pgmap v1175: 268 pgs: 268 active+clean; 4.4 MiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-03-21T13:43:16.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:16 smithi097 ceph-mon[39416]: osdmap e752: 8 total, 8 up, 8 in 2024-03-21T13:43:16.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:16 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:43:16.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:16 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:43:16.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:16 smithi097 ceph-mon[39416]: osdmap e753: 8 total, 8 up, 8 in 2024-03-21T13:43:16.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:16 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:43:16.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:16 smithi097 ceph-mon[30040]: pgmap v1175: 268 pgs: 268 active+clean; 4.4 MiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-03-21T13:43:16.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:16 smithi097 ceph-mon[30040]: osdmap e752: 8 total, 8 up, 8 in 2024-03-21T13:43:16.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:16 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:43:16.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:16 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:43:16.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:16 smithi097 ceph-mon[30040]: osdmap e753: 8 total, 8 up, 8 in 2024-03-21T13:43:16.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:16 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:43:16.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:16 smithi144 ceph-mon[33633]: pgmap v1175: 268 pgs: 268 active+clean; 4.4 MiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-03-21T13:43:16.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:16 smithi144 ceph-mon[33633]: osdmap e752: 8 total, 8 up, 8 in 2024-03-21T13:43:16.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:16 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:43:16.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:16 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:43:16.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:16 smithi144 ceph-mon[33633]: osdmap e753: 8 total, 8 up, 8 in 2024-03-21T13:43:16.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:16 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi097-79906-111"}]: dispatch 2024-03-21T13:43:17.153 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting LibRadosTwoPoolsECPP.TryFlush (7901 ms) 2024-03-21T13:43:17.153 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FailedFlush 2024-03-21T13:43:17.154 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FailedFlush (13072 ms) 2024-03-21T13:43:17.154 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Flush 2024-03-21T13:43:17.154 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Flush (8050 ms) 2024-03-21T13:43:17.154 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushSnap 2024-03-21T13:43:17.154 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushSnap (13077 ms) 2024-03-21T13:43:17.154 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushTryFlushRaces 2024-03-21T13:43:17.154 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushTryFlushRaces (7818 ms) 2024-03-21T13:43:17.154 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlushReadRace 2024-03-21T13:43:17.154 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlushReadRace (8060 ms) 2024-03-21T13:43:17.154 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetRead 2024-03-21T13:43:17.154 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: hmm, no HitSet yet 2024-03-21T13:43:17.154 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: ok, hit_set contains 330:602f83fe:::foo:head 2024-03-21T13:43:17.154 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetRead (9031 ms) 2024-03-21T13:43:17.155 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetTrim 2024-03-21T13:43:17.155 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: got ls 1711028512,0 2024-03-21T13:43:17.155 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: first is 1711028512 2024-03-21T13:43:17.155 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: got ls 1711028512,0 2024-03-21T13:43:17.155 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: got ls 1711028512,0 2024-03-21T13:43:17.155 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: got ls 1711028512,1711028514,1711028515,0 2024-03-21T13:43:17.155 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: got ls 1711028512,1711028514,1711028515,0 2024-03-21T13:43:17.155 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: got ls 1711028512,1711028514,1711028515,0 2024-03-21T13:43:17.155 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: got ls 1711028512,1711028514,1711028515,1711028517,1711028518,0 2024-03-21T13:43:17.155 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: got ls 1711028512,1711028514,1711028515,1711028517,1711028518,0 2024-03-21T13:43:17.155 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: got ls 1711028512,1711028514,1711028515,1711028517,1711028518,0 2024-03-21T13:43:17.155 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: got ls 1711028512,1711028514,1711028515,1711028517,1711028518,1711028520,1711028521,0 2024-03-21T13:43:17.155 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: got ls 1711028512,1711028514,1711028515,1711028517,1711028518,1711028520,1711028521,0 2024-03-21T13:43:17.155 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: got ls 1711028512,1711028514,1711028515,1711028517,1711028518,1711028520,1711028521,0 2024-03-21T13:43:17.155 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: got ls 1711028515,1711028517,1711028518,1711028520,1711028521,1711028523,1711028524,0 2024-03-21T13:43:17.155 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: first now 1711028515, trimmed 2024-03-21T13:43:17.156 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetTrim (20098 ms) 2024-03-21T13:43:17.156 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteOn2ndRead 2024-03-21T13:43:17.156 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: foo0 2024-03-21T13:43:17.156 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-03-21T13:43:17.156 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteOn2ndRead (15092 ms) 2024-03-21T13:43:17.156 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ProxyRead 2024-03-21T13:43:17.156 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ProxyRead (18068 ms) 2024-03-21T13:43:17.156 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.CachePin 2024-03-21T13:43:17.156 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.CachePin (22144 ms) 2024-03-21T13:43:17.156 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetRedirectRead 2024-03-21T13:43:17.156 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetRedirectRead (5036 ms) 2024-03-21T13:43:17.156 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetChunkRead 2024-03-21T13:43:17.156 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetChunkRead (3021 ms) 2024-03-21T13:43:17.156 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ManifestPromoteRead 2024-03-21T13:43:17.157 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ManifestPromoteRead (3032 ms) 2024-03-21T13:43:17.157 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TrySetDedupTier 2024-03-21T13:43:17.157 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TrySetDedupTier (3019 ms) 2024-03-21T13:43:17.157 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP (232013 ms total) 2024-03-21T13:43:17.157 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: 2024-03-21T13:43:17.157 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [----------] Global test environment tear-down 2024-03-21T13:43:17.157 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [==========] 77 tests from 4 test suites ran. (857619 ms total) 2024-03-21T13:43:17.157 INFO:tasks.workunit.client.0.smithi097.stdout: api_tier_pp: [ PASSED ] 77 tests. 2024-03-21T13:43:17.158 INFO:tasks.workunit.client.0.smithi097.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:43:17.158 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=79594 2024-03-21T13:43:17.158 INFO:tasks.workunit.client.0.smithi097.stderr:+ wait 79594 2024-03-21T13:43:17.158 INFO:tasks.workunit.client.0.smithi097.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:43:17.158 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=79746 2024-03-21T13:43:17.158 INFO:tasks.workunit.client.0.smithi097.stderr:+ wait 79746 2024-03-21T13:43:17.159 INFO:tasks.workunit.client.0.smithi097.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:43:17.162 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=80337 2024-03-21T13:43:17.163 INFO:tasks.workunit.client.0.smithi097.stderr:+ wait 80337 2024-03-21T13:43:17.163 INFO:tasks.workunit.client.0.smithi097.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:43:17.163 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=80043 2024-03-21T13:43:17.163 INFO:tasks.workunit.client.0.smithi097.stderr:+ wait 80043 2024-03-21T13:43:17.164 INFO:tasks.workunit.client.0.smithi097.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:43:17.164 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=79856 2024-03-21T13:43:17.164 INFO:tasks.workunit.client.0.smithi097.stderr:+ wait 79856 2024-03-21T13:43:17.164 INFO:tasks.workunit.client.0.smithi097.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:43:17.164 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=79672 2024-03-21T13:43:17.164 INFO:tasks.workunit.client.0.smithi097.stderr:+ wait 79672 2024-03-21T13:43:18.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:18 smithi097 ceph-mon[39416]: pgmap v1178: 260 pgs: 260 active+clean; 582 KiB data, 827 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:43:18.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:18 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:43:18.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:18 smithi097 ceph-mon[39416]: osdmap e754: 8 total, 8 up, 8 in 2024-03-21T13:43:18.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:18 smithi097 ceph-mon[39416]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:43:18.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:18 smithi097 ceph-mon[30040]: pgmap v1178: 260 pgs: 260 active+clean; 582 KiB data, 827 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:43:18.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:18 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:43:18.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:18 smithi097 ceph-mon[30040]: osdmap e754: 8 total, 8 up, 8 in 2024-03-21T13:43:18.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:18 smithi097 ceph-mon[30040]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:43:18.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:18 smithi144 ceph-mon[33633]: pgmap v1178: 260 pgs: 260 active+clean; 582 KiB data, 827 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:43:18.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:18 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1921688807' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi097-79906-111"}]': finished 2024-03-21T13:43:18.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:18 smithi144 ceph-mon[33633]: osdmap e754: 8 total, 8 up, 8 in 2024-03-21T13:43:18.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:18 smithi144 ceph-mon[33633]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:43:18.984 INFO:tasks.workunit.client.0.smithi097.stdout:1 sec 2024-03-21T13:43:18.984 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.984 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.984 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.985 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.985 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.985 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.985 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.985 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.985 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.985 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.985 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.986 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.986 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.986 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.986 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.986 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.986 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.986 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.986 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.986 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.986 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.986 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.987 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.987 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.987 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.987 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.987 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.987 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.987 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.987 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.987 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.987 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.987 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.987 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.987 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.987 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.988 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.988 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.988 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.988 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.988 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.988 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.988 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.988 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.988 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.988 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.988 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.988 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.988 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.988 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.988 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.988 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.989 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.989 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.989 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.989 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.989 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.989 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.989 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.989 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.989 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.989 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.989 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.989 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.989 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.989 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.990 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.990 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.990 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.990 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.990 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.990 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.990 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.990 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.990 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.990 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.990 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.990 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.990 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.990 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.991 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.991 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.991 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.991 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.991 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.991 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.991 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.991 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.991 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.991 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.991 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.991 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.991 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.991 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.992 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.992 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.992 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.992 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:18.992 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:43:20.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:20 smithi097 ceph-mon[39416]: pgmap v1180: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:43:20.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:20 smithi097 ceph-mon[30040]: pgmap v1180: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:43:20.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:20 smithi144 ceph-mon[33633]: pgmap v1180: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:43:22.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:22 smithi097 ceph-mon[39416]: pgmap v1181: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:43:22.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:22 smithi097 ceph-mon[30040]: pgmap v1181: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:43:22.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:43:21 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:43:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:43:22.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:22 smithi144 ceph-mon[33633]: pgmap v1181: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:43:24.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:24 smithi097 ceph-mon[30040]: pgmap v1182: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 678 B/s rd, 0 op/s 2024-03-21T13:43:24.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:24 smithi097 ceph-mon[39416]: pgmap v1182: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 678 B/s rd, 0 op/s 2024-03-21T13:43:24.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:24 smithi144 ceph-mon[33633]: pgmap v1182: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 678 B/s rd, 0 op/s 2024-03-21T13:43:24.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:43:24 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:43:25.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:25 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:43:25.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:25 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:43:25.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:25 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:43:26.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:26 smithi097 ceph-mon[39416]: pgmap v1183: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:43:26.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:26 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:43:26.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:26 smithi097 ceph-mon[30040]: pgmap v1183: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:43:26.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:26 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:43:26.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:26 smithi144 ceph-mon[33633]: pgmap v1183: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:43:26.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:26 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:43:28.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:28 smithi097 ceph-mon[39416]: pgmap v1184: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:43:28.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:28 smithi097 ceph-mon[30040]: pgmap v1184: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:43:28.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:28 smithi144 ceph-mon[33633]: pgmap v1184: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:43:30.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:30 smithi097 ceph-mon[39416]: pgmap v1185: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-21T13:43:30.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:30 smithi097 ceph-mon[30040]: pgmap v1185: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-21T13:43:30.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:30 smithi144 ceph-mon[33633]: pgmap v1185: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-21T13:43:32.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:32 smithi097 ceph-mon[39416]: pgmap v1186: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:43:32.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:32 smithi097 ceph-mon[30040]: pgmap v1186: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:43:32.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:43:31 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:43:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:43:32.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:32 smithi144 ceph-mon[33633]: pgmap v1186: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:43:34.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:34 smithi097 ceph-mon[39416]: pgmap v1187: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:43:34.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:34 smithi097 ceph-mon[30040]: pgmap v1187: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:43:34.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:34 smithi144 ceph-mon[33633]: pgmap v1187: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:43:34.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:43:34 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:43:35.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:35 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:43:35.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:35 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:43:35.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:35 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:43:36.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:36 smithi097 ceph-mon[39416]: pgmap v1188: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:43:36.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:36 smithi097 ceph-mon[30040]: pgmap v1188: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:43:36.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:36 smithi144 ceph-mon[33633]: pgmap v1188: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:43:38.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:38 smithi097 ceph-mon[39416]: pgmap v1189: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:43:38.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:38 smithi097 ceph-mon[30040]: pgmap v1189: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:43:38.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:38 smithi144 ceph-mon[33633]: pgmap v1189: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:43:40.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:40 smithi097 ceph-mon[39416]: pgmap v1190: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:43:40.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:40 smithi097 ceph-mon[30040]: pgmap v1190: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:43:40.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:40 smithi144 ceph-mon[33633]: pgmap v1190: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:43:41.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:41 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:43:41.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:41 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:43:41.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:41 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:43:42.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:42 smithi097 ceph-mon[30040]: pgmap v1191: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:43:42.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:42 smithi097 ceph-mon[39416]: pgmap v1191: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:43:42.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:43:41 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:43:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:43:42.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:42 smithi144 ceph-mon[33633]: pgmap v1191: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:43:44.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:44 smithi097 ceph-mon[30040]: pgmap v1192: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:43:44.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:44 smithi097 ceph-mon[39416]: pgmap v1192: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:43:44.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:44 smithi144 ceph-mon[33633]: pgmap v1192: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:43:44.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:43:44 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:43:45.522 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:45 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:43:45.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:45 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:43:45.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:45 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:43:46.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:46 smithi097 ceph-mon[39416]: pgmap v1193: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:43:46.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:46 smithi097 ceph-mon[30040]: pgmap v1193: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:43:46.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:46 smithi144 ceph-mon[33633]: pgmap v1193: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:43:48.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:48 smithi097 ceph-mon[39416]: pgmap v1194: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:43:48.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:48 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:43:48.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:48 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:43:48.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:48 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:43:48.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:48 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:43:48.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:48 smithi097 ceph-mon[30040]: pgmap v1194: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:43:48.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:48 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:43:48.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:48 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:43:48.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:48 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:43:48.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:48 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:43:48.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:48 smithi144 ceph-mon[33633]: pgmap v1194: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:43:48.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:48 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:43:48.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:48 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:43:48.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:48 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:43:48.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:48 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:43:50.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:50 smithi097 ceph-mon[39416]: pgmap v1195: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:43:50.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:50 smithi097 ceph-mon[30040]: pgmap v1195: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:43:50.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:50 smithi144 ceph-mon[33633]: pgmap v1195: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:43:52.212 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:43:51 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:43:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:43:52.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:52 smithi097 ceph-mon[39416]: pgmap v1196: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:43:52.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:52 smithi097 ceph-mon[30040]: pgmap v1196: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:43:52.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:52 smithi144 ceph-mon[33633]: pgmap v1196: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:43:54.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:54 smithi097 ceph-mon[39416]: pgmap v1197: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:43:54.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:54 smithi097 ceph-mon[30040]: pgmap v1197: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:43:54.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:54 smithi144 ceph-mon[33633]: pgmap v1197: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:43:54.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:43:54 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:43:55.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:55 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:43:55.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:55 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:43:55.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:55 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:43:56.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:56 smithi097 ceph-mon[39416]: pgmap v1198: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:43:56.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:56 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:43:56.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:56 smithi097 ceph-mon[30040]: pgmap v1198: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:43:56.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:56 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:43:56.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:56 smithi144 ceph-mon[33633]: pgmap v1198: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:43:56.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:56 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:43:58.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:43:58 smithi097 ceph-mon[39416]: pgmap v1199: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:43:58.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:43:58 smithi097 ceph-mon[30040]: pgmap v1199: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:43:58.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:43:58 smithi144 ceph-mon[33633]: pgmap v1199: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:44:00.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:44:00 smithi097 ceph-mon[39416]: pgmap v1200: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:44:00.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:44:00 smithi097 ceph-mon[30040]: pgmap v1200: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:44:00.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:44:00 smithi144 ceph-mon[33633]: pgmap v1200: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:44:02.231 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:44:01 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:44:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:44:02.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:44:02 smithi097 ceph-mon[30040]: pgmap v1201: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:44:02.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:44:02 smithi097 ceph-mon[39416]: pgmap v1201: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:44:02.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:44:02 smithi144 ceph-mon[33633]: pgmap v1201: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:44:04.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:44:04 smithi097 ceph-mon[39416]: pgmap v1202: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:44:04.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:44:04 smithi097 ceph-mon[30040]: pgmap v1202: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:44:04.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:44:04 smithi144 ceph-mon[33633]: pgmap v1202: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:44:04.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:44:04 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:44:05.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:44:05 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:44:05.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:44:05 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:44:05.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:44:05 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:44:06.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:44:06 smithi097 ceph-mon[39416]: pgmap v1203: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:44:06.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:44:06 smithi097 ceph-mon[30040]: pgmap v1203: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:44:06.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:44:06 smithi144 ceph-mon[33633]: pgmap v1203: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:44:08.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:44:08 smithi097 ceph-mon[39416]: pgmap v1204: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:44:08.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:44:08 smithi097 ceph-mon[30040]: pgmap v1204: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:44:08.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:44:08 smithi144 ceph-mon[33633]: pgmap v1204: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:44:10.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:44:10 smithi097 ceph-mon[39416]: pgmap v1205: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:44:10.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:44:10 smithi097 ceph-mon[30040]: pgmap v1205: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:44:10.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:44:10 smithi144 ceph-mon[33633]: pgmap v1205: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:44:11.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:44:11 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:44:11.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:44:11 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:44:11.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:44:11 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:44:12.256 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:44:11 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:44:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:44:12.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:44:12 smithi097 ceph-mon[39416]: pgmap v1206: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:44:12.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:44:12 smithi097 ceph-mon[30040]: pgmap v1206: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:44:12.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:44:12 smithi144 ceph-mon[33633]: pgmap v1206: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:44:14.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:44:14 smithi097 ceph-mon[30040]: pgmap v1207: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:44:14.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:44:14 smithi097 ceph-mon[39416]: pgmap v1207: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:44:14.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:44:14 smithi144 ceph-mon[33633]: pgmap v1207: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:44:14.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:44:14 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:44:15.522 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:44:15 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:44:15.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:44:15 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:44:15.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:44:15 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:44:16.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:44:16 smithi097 ceph-mon[39416]: pgmap v1208: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:44:16.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:44:16 smithi097 ceph-mon[30040]: pgmap v1208: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:44:16.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:44:16 smithi144 ceph-mon[33633]: pgmap v1208: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:44:18.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:44:18 smithi097 ceph-mon[39416]: pgmap v1209: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:44:18.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:44:18 smithi097 ceph-mon[30040]: pgmap v1209: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:44:18.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:44:18 smithi144 ceph-mon[33633]: pgmap v1209: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:44:20.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:44:20 smithi097 ceph-mon[39416]: pgmap v1210: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:44:20.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:44:20 smithi097 ceph-mon[30040]: pgmap v1210: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:44:20.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:44:20 smithi144 ceph-mon[33633]: pgmap v1210: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:44:22.278 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:44:21 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:44:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:44:22.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:44:22 smithi097 ceph-mon[30040]: pgmap v1211: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:44:22.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:44:22 smithi097 ceph-mon[39416]: pgmap v1211: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:44:22.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:44:22 smithi144 ceph-mon[33633]: pgmap v1211: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:44:24.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:44:24 smithi097 ceph-mon[39416]: pgmap v1212: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:44:24.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:44:24 smithi097 ceph-mon[30040]: pgmap v1212: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:44:24.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:44:24 smithi144 ceph-mon[33633]: pgmap v1212: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:44:24.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:44:24 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:44:25.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:44:25 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:44:25.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:44:25 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:44:25.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:44:25 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:44:26.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:44:26 smithi097 ceph-mon[39416]: pgmap v1213: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:44:26.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:44:26 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:44:26.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:44:26 smithi097 ceph-mon[30040]: pgmap v1213: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:44:26.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:44:26 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:44:26.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:44:26 smithi144 ceph-mon[33633]: pgmap v1213: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:44:26.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:44:26 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:44:28.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:44:28 smithi097 ceph-mon[39416]: pgmap v1214: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:44:28.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:44:28 smithi097 ceph-mon[30040]: pgmap v1214: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:44:28.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:44:28 smithi144 ceph-mon[33633]: pgmap v1214: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:44:30.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:44:30 smithi097 ceph-mon[39416]: pgmap v1215: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:44:30.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:44:30 smithi097 ceph-mon[30040]: pgmap v1215: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:44:30.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:44:30 smithi144 ceph-mon[33633]: pgmap v1215: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:44:32.296 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:44:31 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:44:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:44:32.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:44:32 smithi097 ceph-mon[30040]: pgmap v1216: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:44:32.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:44:32 smithi097 ceph-mon[39416]: pgmap v1216: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:44:32.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:44:32 smithi144 ceph-mon[33633]: pgmap v1216: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:44:34.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:44:34 smithi097 ceph-mon[30040]: pgmap v1217: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:44:34.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:44:34 smithi097 ceph-mon[39416]: pgmap v1217: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:44:34.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:44:34 smithi144 ceph-mon[33633]: pgmap v1217: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:44:34.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:44:34 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:44:35.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:44:35 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:44:35.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:44:35 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:44:35.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:44:35 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:44:36.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:44:36 smithi144 ceph-mon[33633]: pgmap v1218: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:44:36.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:44:36 smithi097 ceph-mon[30040]: pgmap v1218: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:44:36.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:44:36 smithi097 ceph-mon[39416]: pgmap v1218: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:44:38.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:44:38 smithi144 ceph-mon[33633]: pgmap v1219: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:44:38.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:44:38 smithi097 ceph-mon[30040]: pgmap v1219: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:44:38.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:44:38 smithi097 ceph-mon[39416]: pgmap v1219: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:44:40.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:44:40 smithi144 ceph-mon[33633]: pgmap v1220: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:44:40.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:44:40 smithi097 ceph-mon[30040]: pgmap v1220: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:44:40.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:44:40 smithi097 ceph-mon[39416]: pgmap v1220: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:44:41.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:44:41 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:44:41.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:44:41 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:44:41.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:44:41 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:44:42.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:44:41 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:44:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:44:42.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:44:42 smithi144 ceph-mon[33633]: pgmap v1221: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:44:42.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:44:42 smithi097 ceph-mon[39416]: pgmap v1221: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:44:42.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:44:42 smithi097 ceph-mon[30040]: pgmap v1221: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:44:44.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:44:44 smithi144 ceph-mon[33633]: pgmap v1222: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:44:44.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:44:44 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:44:44.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:44:44 smithi097 ceph-mon[39416]: pgmap v1222: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:44:44.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:44:44 smithi097 ceph-mon[30040]: pgmap v1222: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:44:45.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:44:45 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:44:45.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:44:45 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:44:45.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:44:45 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:44:46.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:44:46 smithi144 ceph-mon[33633]: pgmap v1223: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:44:46.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:44:46 smithi097 ceph-mon[39416]: pgmap v1223: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:44:46.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:44:46 smithi097 ceph-mon[30040]: pgmap v1223: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:44:48.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:44:48 smithi144 ceph-mon[33633]: pgmap v1224: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:44:48.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:44:48 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:44:48.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:44:48 smithi097 ceph-mon[39416]: pgmap v1224: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:44:48.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:44:48 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:44:48.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:44:48 smithi097 ceph-mon[30040]: pgmap v1224: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:44:48.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:44:48 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:44:49.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:44:49 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:44:49.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:44:49 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:44:49.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:44:49 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:44:49.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:44:49 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:44:49.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:44:49 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:44:49.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:44:49 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:44:49.807 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:44:49 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:44:49.807 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:44:49 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:44:49.807 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:44:49 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:44:50.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:44:50 smithi144 ceph-mon[33633]: pgmap v1225: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:44:50.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:44:50 smithi097 ceph-mon[39416]: pgmap v1225: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:44:50.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:44:50 smithi097 ceph-mon[30040]: pgmap v1225: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:44:52.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:44:51 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:44:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:44:52.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:44:52 smithi144 ceph-mon[33633]: pgmap v1226: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:44:52.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:44:52 smithi097 ceph-mon[30040]: pgmap v1226: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:44:52.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:44:52 smithi097 ceph-mon[39416]: pgmap v1226: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:44:54.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:44:54 smithi144 ceph-mon[33633]: pgmap v1227: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:44:54.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:44:54 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:44:54.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:44:54 smithi097 ceph-mon[39416]: pgmap v1227: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:44:54.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:44:54 smithi097 ceph-mon[30040]: pgmap v1227: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:44:55.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:44:55 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:44:55.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:44:55 smithi144 ceph-mon[33633]: pgmap v1228: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:44:55.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:44:55 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:44:55.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:44:55 smithi097 ceph-mon[39416]: pgmap v1228: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:44:55.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:44:55 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:44:55.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:44:55 smithi097 ceph-mon[30040]: pgmap v1228: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:44:56.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:44:56 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:44:56.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:44:56 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:44:56.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:44:56 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:44:57.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:44:57 smithi144 ceph-mon[33633]: pgmap v1229: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:44:57.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:44:57 smithi097 ceph-mon[30040]: pgmap v1229: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:44:57.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:44:57 smithi097 ceph-mon[39416]: pgmap v1229: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:44:58.995 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:58.996 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:58.996 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:58.996 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:58.996 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:58.996 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:58.997 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:58.997 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:58.997 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:58.997 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:58.997 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:58.997 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:58.998 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:58.998 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:58.998 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:58.998 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:58.998 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:58.999 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:58.999 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:58.999 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:58.999 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:58.999 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:58.999 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.000 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.000 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.000 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.000 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.000 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.000 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.000 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.001 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.001 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.001 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.002 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.002 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.002 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.002 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.002 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.002 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.002 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.003 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.003 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.003 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.003 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.003 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.003 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.003 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.004 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.004 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.004 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.004 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.004 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.004 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.004 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.004 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.005 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.005 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.005 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.005 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.005 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.005 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.005 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.006 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.006 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.006 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.006 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.006 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.006 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.006 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.007 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.007 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.007 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.007 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.007 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.007 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.007 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.008 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.008 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.008 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.008 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.008 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.008 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.009 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.009 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.009 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.009 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.009 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.009 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.009 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.010 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.010 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.010 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.010 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.010 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.010 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.010 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.010 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.011 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.011 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:44:59.011 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:45:00.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:44:59 smithi097 ceph-mon[39416]: pgmap v1230: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:45:00.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:44:59 smithi097 ceph-mon[30040]: pgmap v1230: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:45:00.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:44:59 smithi144 ceph-mon[33633]: pgmap v1230: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:45:02.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:45:01 smithi097 ceph-mon[30040]: pgmap v1231: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:45:02.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:45:01 smithi097 ceph-mon[39416]: pgmap v1231: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:45:02.054 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:45:01 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:45:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:45:02.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:45:01 smithi144 ceph-mon[33633]: pgmap v1231: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:45:04.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:45:03 smithi097 ceph-mon[39416]: pgmap v1232: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:45:04.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:45:03 smithi097 ceph-mon[30040]: pgmap v1232: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:45:04.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:45:03 smithi144 ceph-mon[33633]: pgmap v1232: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:45:04.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:45:04 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:45:05.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:45:04 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:45:05.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:45:04 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:45:05.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:45:04 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:45:06.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:45:05 smithi097 ceph-mon[39416]: pgmap v1233: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:45:06.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:45:05 smithi097 ceph-mon[30040]: pgmap v1233: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:45:06.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:45:05 smithi144 ceph-mon[33633]: pgmap v1233: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:45:08.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:45:07 smithi097 ceph-mon[39416]: pgmap v1234: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:45:08.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:45:07 smithi097 ceph-mon[30040]: pgmap v1234: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:45:08.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:45:07 smithi144 ceph-mon[33633]: pgmap v1234: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:45:10.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:45:09 smithi097 ceph-mon[39416]: pgmap v1235: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:45:10.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:45:09 smithi097 ceph-mon[30040]: pgmap v1235: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:45:10.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:45:09 smithi144 ceph-mon[33633]: pgmap v1235: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:45:12.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:45:11 smithi097 ceph-mon[39416]: pgmap v1236: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:45:12.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:45:11 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:45:12.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:45:11 smithi097 ceph-mon[30040]: pgmap v1236: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:45:12.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:45:11 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:45:12.054 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:45:11 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:45:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:45:12.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:45:11 smithi144 ceph-mon[33633]: pgmap v1236: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:45:12.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:45:11 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:45:14.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:45:13 smithi097 ceph-mon[39416]: pgmap v1237: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:45:14.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:45:13 smithi097 ceph-mon[30040]: pgmap v1237: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:45:14.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:45:13 smithi144 ceph-mon[33633]: pgmap v1237: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:45:14.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:45:14 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:45:15.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:45:14 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:45:15.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:45:14 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:45:15.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:45:14 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:45:16.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:45:15 smithi144 ceph-mon[33633]: pgmap v1238: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:45:16.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:45:15 smithi097 ceph-mon[30040]: pgmap v1238: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:45:16.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:45:15 smithi097 ceph-mon[39416]: pgmap v1238: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:45:18.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:45:17 smithi144 ceph-mon[33633]: pgmap v1239: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:45:18.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:45:17 smithi097 ceph-mon[30040]: pgmap v1239: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:45:18.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:45:17 smithi097 ceph-mon[39416]: pgmap v1239: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:45:20.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:45:19 smithi144 ceph-mon[33633]: pgmap v1240: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:45:20.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:45:19 smithi097 ceph-mon[30040]: pgmap v1240: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:45:20.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:45:19 smithi097 ceph-mon[39416]: pgmap v1240: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:45:22.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:45:21 smithi144 ceph-mon[33633]: pgmap v1241: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:45:22.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:45:21 smithi097 ceph-mon[39416]: pgmap v1241: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:45:22.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:45:21 smithi097 ceph-mon[30040]: pgmap v1241: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:45:22.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:45:21 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:45:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:45:24.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:45:23 smithi144 ceph-mon[33633]: pgmap v1242: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:45:24.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:45:23 smithi097 ceph-mon[39416]: pgmap v1242: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:45:24.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:45:23 smithi097 ceph-mon[30040]: pgmap v1242: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:45:24.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:45:24 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:45:25.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:45:24 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:45:25.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:45:24 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:45:25.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:45:24 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:45:26.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:45:25 smithi144 ceph-mon[33633]: pgmap v1243: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:45:26.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:45:25 smithi097 ceph-mon[30040]: pgmap v1243: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:45:26.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:45:25 smithi097 ceph-mon[39416]: pgmap v1243: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:45:27.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:45:26 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:45:27.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:45:26 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:45:27.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:45:26 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:45:28.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:45:27 smithi144 ceph-mon[33633]: pgmap v1244: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:45:28.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:45:27 smithi097 ceph-mon[30040]: pgmap v1244: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:45:28.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:45:27 smithi097 ceph-mon[39416]: pgmap v1244: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:45:30.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:45:29 smithi144 ceph-mon[33633]: pgmap v1245: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:45:30.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:45:29 smithi097 ceph-mon[30040]: pgmap v1245: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:45:30.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:45:29 smithi097 ceph-mon[39416]: pgmap v1245: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:45:32.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:45:31 smithi144 ceph-mon[33633]: pgmap v1246: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:45:32.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:45:31 smithi097 ceph-mon[39416]: pgmap v1246: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:45:32.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:45:31 smithi097 ceph-mon[30040]: pgmap v1246: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:45:32.306 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:45:31 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:45:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:45:34.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:45:33 smithi144 ceph-mon[33633]: pgmap v1247: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:45:34.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:45:33 smithi097 ceph-mon[39416]: pgmap v1247: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:45:34.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:45:33 smithi097 ceph-mon[30040]: pgmap v1247: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:45:34.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:45:34 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:45:35.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:45:34 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:45:35.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:45:34 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:45:35.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:45:34 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:45:36.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:45:35 smithi144 ceph-mon[33633]: pgmap v1248: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:45:36.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:45:35 smithi097 ceph-mon[39416]: pgmap v1248: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:45:36.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:45:35 smithi097 ceph-mon[30040]: pgmap v1248: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:45:38.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:45:37 smithi144 ceph-mon[33633]: pgmap v1249: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:45:38.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:45:37 smithi097 ceph-mon[39416]: pgmap v1249: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:45:38.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:45:37 smithi097 ceph-mon[30040]: pgmap v1249: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:45:40.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:45:39 smithi144 ceph-mon[33633]: pgmap v1250: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:45:40.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:45:39 smithi097 ceph-mon[30040]: pgmap v1250: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:45:40.306 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:45:39 smithi097 ceph-mon[39416]: pgmap v1250: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:45:42.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:45:41 smithi144 ceph-mon[33633]: pgmap v1251: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:45:42.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:45:41 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:45:42.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:45:41 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:45:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:45:42.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:45:41 smithi097 ceph-mon[39416]: pgmap v1251: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:45:42.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:45:41 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:45:42.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:45:41 smithi097 ceph-mon[30040]: pgmap v1251: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:45:42.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:45:41 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:45:44.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:45:43 smithi097 ceph-mon[30040]: pgmap v1252: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:45:44.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:45:43 smithi097 ceph-mon[39416]: pgmap v1252: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:45:44.354 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:45:43 smithi144 ceph-mon[33633]: pgmap v1252: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:45:44.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:45:44 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:45:45.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:45:44 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:45:45.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:45:44 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:45:45.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:45:44 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:45:46.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:45:45 smithi097 ceph-mon[39416]: pgmap v1253: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:45:46.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:45:45 smithi097 ceph-mon[30040]: pgmap v1253: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:45:46.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:45:45 smithi144 ceph-mon[33633]: pgmap v1253: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:45:48.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:45:47 smithi097 ceph-mon[39416]: pgmap v1254: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:45:48.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:45:47 smithi097 ceph-mon[30040]: pgmap v1254: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:45:48.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:45:47 smithi144 ceph-mon[33633]: pgmap v1254: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:45:50.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:45:49 smithi097 ceph-mon[39416]: pgmap v1255: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:45:50.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:45:49 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:45:50.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:45:49 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:45:50.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:45:49 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:45:50.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:45:49 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:45:50.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:45:49 smithi097 ceph-mon[30040]: pgmap v1255: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:45:50.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:45:49 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:45:50.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:45:49 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:45:50.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:45:49 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:45:50.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:45:49 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:45:50.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:45:49 smithi144 ceph-mon[33633]: pgmap v1255: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:45:50.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:45:49 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:45:50.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:45:49 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:45:50.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:45:49 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:45:50.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:45:49 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:45:52.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:45:51 smithi097 ceph-mon[39416]: pgmap v1256: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:45:52.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:45:51 smithi097 ceph-mon[30040]: pgmap v1256: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:45:52.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:45:51 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:45:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:45:52.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:45:51 smithi144 ceph-mon[33633]: pgmap v1256: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:45:54.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:45:53 smithi097 ceph-mon[39416]: pgmap v1257: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:45:54.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:45:53 smithi097 ceph-mon[30040]: pgmap v1257: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:45:54.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:45:53 smithi144 ceph-mon[33633]: pgmap v1257: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:45:54.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:45:54 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:45:56.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:45:55 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:45:56.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:45:55 smithi097 ceph-mon[39416]: pgmap v1258: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:45:56.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:45:55 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:45:56.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:45:55 smithi097 ceph-mon[30040]: pgmap v1258: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:45:56.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:45:55 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:45:56.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:45:55 smithi144 ceph-mon[33633]: pgmap v1258: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:45:57.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:45:56 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:45:57.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:45:56 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:45:57.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:45:56 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:45:58.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:45:57 smithi097 ceph-mon[39416]: pgmap v1259: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:45:58.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:45:57 smithi097 ceph-mon[30040]: pgmap v1259: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:45:58.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:45:57 smithi144 ceph-mon[33633]: pgmap v1259: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:46:00.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:45:59 smithi097 ceph-mon[39416]: pgmap v1260: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:46:00.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:45:59 smithi097 ceph-mon[30040]: pgmap v1260: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:46:00.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:45:59 smithi144 ceph-mon[33633]: pgmap v1260: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:46:02.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:46:01 smithi097 ceph-mon[30040]: pgmap v1261: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:46:02.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:46:01 smithi097 ceph-mon[39416]: pgmap v1261: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:46:02.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:46:01 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:46:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:46:02.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:46:01 smithi144 ceph-mon[33633]: pgmap v1261: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:46:04.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:46:03 smithi097 ceph-mon[30040]: pgmap v1262: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:46:04.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:46:03 smithi097 ceph-mon[39416]: pgmap v1262: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:46:04.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:46:03 smithi144 ceph-mon[33633]: pgmap v1262: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:46:04.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:46:04 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:46:06.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:46:05 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:46:06.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:46:05 smithi097 ceph-mon[30040]: pgmap v1263: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:46:06.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:46:05 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:46:06.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:46:05 smithi097 ceph-mon[39416]: pgmap v1263: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:46:06.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:46:05 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:46:06.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:46:05 smithi144 ceph-mon[33633]: pgmap v1263: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:46:08.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:46:07 smithi097 ceph-mon[30040]: pgmap v1264: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:46:08.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:46:07 smithi097 ceph-mon[39416]: pgmap v1264: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:46:08.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:46:07 smithi144 ceph-mon[33633]: pgmap v1264: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:46:10.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:46:09 smithi097 ceph-mon[30040]: pgmap v1265: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:46:10.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:46:09 smithi097 ceph-mon[39416]: pgmap v1265: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:46:10.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:46:09 smithi144 ceph-mon[33633]: pgmap v1265: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:46:12.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:46:11 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:46:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:46:12.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:46:11 smithi097 ceph-mon[30040]: pgmap v1266: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:46:12.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:46:11 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:46:12.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:46:11 smithi097 ceph-mon[39416]: pgmap v1266: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:46:12.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:46:11 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:46:12.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:46:11 smithi144 ceph-mon[33633]: pgmap v1266: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:46:12.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:46:11 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:46:14.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:46:13 smithi097 ceph-mon[39416]: pgmap v1267: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:46:14.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:46:13 smithi097 ceph-mon[30040]: pgmap v1267: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:46:14.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:46:13 smithi144 ceph-mon[33633]: pgmap v1267: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:46:14.871 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:46:14 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:46:16.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:46:15 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:46:16.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:46:15 smithi097 ceph-mon[39416]: pgmap v1268: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:46:16.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:46:15 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:46:16.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:46:15 smithi097 ceph-mon[30040]: pgmap v1268: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:46:16.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:46:15 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:46:16.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:46:15 smithi144 ceph-mon[33633]: pgmap v1268: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:46:18.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:46:17 smithi097 ceph-mon[30040]: pgmap v1269: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:46:18.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:46:17 smithi097 ceph-mon[39416]: pgmap v1269: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:46:18.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:46:17 smithi144 ceph-mon[33633]: pgmap v1269: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:46:20.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:46:19 smithi097 ceph-mon[39416]: pgmap v1270: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-21T13:46:20.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:46:19 smithi097 ceph-mon[30040]: pgmap v1270: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-21T13:46:20.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:46:19 smithi144 ceph-mon[33633]: pgmap v1270: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-21T13:46:22.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:46:21 smithi097 ceph-mon[30040]: pgmap v1271: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-21T13:46:22.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:46:21 smithi097 ceph-mon[39416]: pgmap v1271: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-21T13:46:22.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:46:21 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:46:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:46:22.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:46:21 smithi144 ceph-mon[33633]: pgmap v1271: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-21T13:46:24.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:46:23 smithi097 ceph-mon[30040]: pgmap v1272: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-21T13:46:24.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:46:23 smithi097 ceph-mon[39416]: pgmap v1272: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-21T13:46:24.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:46:23 smithi144 ceph-mon[33633]: pgmap v1272: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-21T13:46:24.871 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:46:24 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:46:26.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:46:25 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:46:26.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:46:25 smithi097 ceph-mon[39416]: pgmap v1273: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-21T13:46:26.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:46:25 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:46:26.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:46:25 smithi097 ceph-mon[30040]: pgmap v1273: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-21T13:46:26.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:46:25 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:46:26.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:46:25 smithi144 ceph-mon[33633]: pgmap v1273: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-21T13:46:27.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:46:26 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:46:27.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:46:26 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:46:27.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:46:26 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:46:28.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:46:27 smithi097 ceph-mon[39416]: pgmap v1274: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-21T13:46:28.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:46:27 smithi097 ceph-mon[30040]: pgmap v1274: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-21T13:46:28.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:46:27 smithi144 ceph-mon[33633]: pgmap v1274: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-21T13:46:30.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:46:29 smithi097 ceph-mon[39416]: pgmap v1275: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-21T13:46:30.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:46:29 smithi097 ceph-mon[30040]: pgmap v1275: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-21T13:46:30.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:46:29 smithi144 ceph-mon[33633]: pgmap v1275: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-21T13:46:32.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:46:31 smithi097 ceph-mon[39416]: pgmap v1276: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 53 op/s 2024-03-21T13:46:32.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:46:31 smithi097 ceph-mon[30040]: pgmap v1276: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 53 op/s 2024-03-21T13:46:32.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:46:31 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:46:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:46:32.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:46:31 smithi144 ceph-mon[33633]: pgmap v1276: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 53 op/s 2024-03-21T13:46:34.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:46:33 smithi097 ceph-mon[39416]: pgmap v1277: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 53 op/s 2024-03-21T13:46:34.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:46:33 smithi097 ceph-mon[30040]: pgmap v1277: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 53 op/s 2024-03-21T13:46:34.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:46:33 smithi144 ceph-mon[33633]: pgmap v1277: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 53 op/s 2024-03-21T13:46:34.871 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:46:34 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:46:36.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:46:35 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:46:36.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:46:35 smithi097 ceph-mon[39416]: pgmap v1278: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 54 op/s 2024-03-21T13:46:36.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:46:35 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:46:36.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:46:35 smithi097 ceph-mon[30040]: pgmap v1278: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 54 op/s 2024-03-21T13:46:36.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:46:35 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:46:36.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:46:35 smithi144 ceph-mon[33633]: pgmap v1278: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 54 op/s 2024-03-21T13:46:38.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:46:37 smithi097 ceph-mon[39416]: pgmap v1279: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:46:38.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:46:37 smithi097 ceph-mon[30040]: pgmap v1279: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:46:38.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:46:37 smithi144 ceph-mon[33633]: pgmap v1279: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:46:39.007 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.007 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.007 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.007 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.007 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.007 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.007 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.007 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.007 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.007 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.007 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.007 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.008 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.008 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.008 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.008 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.008 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.008 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.008 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.008 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.008 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.008 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.008 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.008 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.008 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.008 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.008 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.009 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.009 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.009 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.009 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.009 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.009 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.009 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.009 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.009 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.009 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.009 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.009 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.009 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.009 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.009 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.009 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.009 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.010 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.010 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.010 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.010 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.010 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.010 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.010 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.010 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.010 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.010 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.010 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.010 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.010 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.010 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.010 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.011 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.011 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.011 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.011 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.011 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.011 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.011 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.011 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.011 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.011 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.011 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.011 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.011 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.011 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.011 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.011 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.012 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.012 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.012 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.012 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.012 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.012 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.012 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.012 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.012 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.012 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.012 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.012 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.012 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.012 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.012 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.013 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.013 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.013 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.013 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.013 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.013 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.013 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.013 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.013 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:39.013 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:46:40.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:46:39 smithi097 ceph-mon[39416]: pgmap v1280: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:46:40.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:46:39 smithi097 ceph-mon[30040]: pgmap v1280: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:46:40.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:46:39 smithi144 ceph-mon[33633]: pgmap v1280: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:46:42.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:46:41 smithi097 ceph-mon[39416]: pgmap v1281: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:46:42.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:46:41 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:46:42.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:46:41 smithi097 ceph-mon[30040]: pgmap v1281: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:46:42.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:46:41 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:46:42.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:46:41 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:46:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:46:42.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:46:41 smithi144 ceph-mon[33633]: pgmap v1281: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:46:42.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:46:41 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:46:44.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:46:43 smithi097 ceph-mon[39416]: pgmap v1282: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:46:44.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:46:43 smithi097 ceph-mon[30040]: pgmap v1282: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:46:44.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:46:43 smithi144 ceph-mon[33633]: pgmap v1282: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:46:44.871 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:46:44 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:46:46.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:46:45 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:46:46.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:46:45 smithi097 ceph-mon[30040]: pgmap v1283: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:46:46.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:46:45 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:46:46.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:46:45 smithi097 ceph-mon[39416]: pgmap v1283: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:46:46.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:46:45 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:46:46.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:46:45 smithi144 ceph-mon[33633]: pgmap v1283: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:46:48.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:46:47 smithi097 ceph-mon[30040]: pgmap v1284: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:46:48.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:46:47 smithi097 ceph-mon[39416]: pgmap v1284: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:46:48.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:46:47 smithi144 ceph-mon[33633]: pgmap v1284: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:46:50.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:46:49 smithi097 ceph-mon[39416]: pgmap v1285: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:46:50.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:46:49 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:46:50.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:46:49 smithi097 ceph-mon[30040]: pgmap v1285: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:46:50.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:46:49 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:46:50.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:46:49 smithi144 ceph-mon[33633]: pgmap v1285: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:46:50.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:46:49 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:46:51.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:46:50 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:46:51.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:46:50 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:46:51.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:46:50 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:46:51.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:46:50 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:46:51.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:46:50 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:46:51.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:46:50 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:46:51.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:46:50 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:46:51.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:46:50 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:46:51.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:46:50 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:46:52.267 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:46:51 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:46:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:46:52.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:46:52 smithi097 ceph-mon[39416]: pgmap v1286: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:46:52.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:46:52 smithi097 ceph-mon[30040]: pgmap v1286: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:46:52.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:46:52 smithi144 ceph-mon[33633]: pgmap v1286: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:46:54.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:46:54 smithi097 ceph-mon[39416]: pgmap v1287: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:46:54.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:46:54 smithi097 ceph-mon[30040]: pgmap v1287: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:46:54.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:46:54 smithi144 ceph-mon[33633]: pgmap v1287: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:46:54.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:46:54 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:46:56.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:46:56 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:46:56.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:46:56 smithi097 ceph-mon[39416]: pgmap v1288: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:46:56.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:46:56 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:46:56.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:46:56 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:46:56.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:46:56 smithi097 ceph-mon[30040]: pgmap v1288: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:46:56.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:46:56 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:46:56.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:46:56 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:46:56.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:46:56 smithi144 ceph-mon[33633]: pgmap v1288: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:46:56.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:46:56 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:46:58.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:46:58 smithi097 ceph-mon[39416]: pgmap v1289: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:46:58.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:46:58 smithi097 ceph-mon[30040]: pgmap v1289: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:46:58.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:46:58 smithi144 ceph-mon[33633]: pgmap v1289: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:47:00.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:47:00 smithi097 ceph-mon[39416]: pgmap v1290: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:47:00.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:47:00 smithi097 ceph-mon[30040]: pgmap v1290: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:47:00.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:47:00 smithi144 ceph-mon[33633]: pgmap v1290: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:47:02.282 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:47:01 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:47:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:47:02.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:47:02 smithi097 ceph-mon[30040]: pgmap v1291: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:47:02.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:47:02 smithi097 ceph-mon[39416]: pgmap v1291: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:47:02.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:47:02 smithi144 ceph-mon[33633]: pgmap v1291: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:47:04.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:47:04 smithi097 ceph-mon[39416]: pgmap v1292: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:47:04.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:47:04 smithi097 ceph-mon[30040]: pgmap v1292: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:47:04.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:47:04 smithi144 ceph-mon[33633]: pgmap v1292: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:47:04.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:47:04 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:47:06.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:47:06 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:47:06.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:47:06 smithi097 ceph-mon[39416]: pgmap v1293: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:47:06.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:47:06 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:47:06.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:47:06 smithi097 ceph-mon[30040]: pgmap v1293: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:47:06.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:47:06 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:47:06.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:47:06 smithi144 ceph-mon[33633]: pgmap v1293: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:47:08.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:47:08 smithi097 ceph-mon[39416]: pgmap v1294: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:47:08.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:47:08 smithi097 ceph-mon[30040]: pgmap v1294: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:47:08.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:47:08 smithi144 ceph-mon[33633]: pgmap v1294: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:47:10.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:47:10 smithi097 ceph-mon[39416]: pgmap v1295: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:47:10.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:47:10 smithi097 ceph-mon[30040]: pgmap v1295: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:47:10.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:47:10 smithi144 ceph-mon[33633]: pgmap v1295: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:47:11.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:47:11 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:47:11.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:47:11 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:47:11.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:47:11 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:47:12.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:47:11 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:47:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:47:12.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:47:12 smithi144 ceph-mon[33633]: pgmap v1296: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:47:12.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:47:12 smithi097 ceph-mon[39416]: pgmap v1296: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:47:12.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:47:12 smithi097 ceph-mon[30040]: pgmap v1296: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:47:14.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:47:14 smithi144 ceph-mon[33633]: pgmap v1297: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:47:14.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:47:14 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:47:14.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:47:14 smithi097 ceph-mon[39416]: pgmap v1297: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:47:14.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:47:14 smithi097 ceph-mon[30040]: pgmap v1297: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:47:16.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:47:16 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:47:16.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:47:16 smithi144 ceph-mon[33633]: pgmap v1298: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:47:16.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:47:16 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:47:16.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:47:16 smithi097 ceph-mon[39416]: pgmap v1298: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:47:16.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:47:16 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:47:16.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:47:16 smithi097 ceph-mon[30040]: pgmap v1298: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:47:18.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:47:18 smithi144 ceph-mon[33633]: pgmap v1299: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:47:18.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:47:18 smithi097 ceph-mon[39416]: pgmap v1299: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:47:18.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:47:18 smithi097 ceph-mon[30040]: pgmap v1299: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:47:20.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:47:20 smithi144 ceph-mon[33633]: pgmap v1300: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:47:20.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:47:20 smithi097 ceph-mon[39416]: pgmap v1300: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:47:20.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:47:20 smithi097 ceph-mon[30040]: pgmap v1300: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:47:22.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:47:21 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:47:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:47:22.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:47:22 smithi144 ceph-mon[33633]: pgmap v1301: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:47:22.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:47:22 smithi097 ceph-mon[39416]: pgmap v1301: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:47:22.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:47:22 smithi097 ceph-mon[30040]: pgmap v1301: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:47:23.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:47:23 smithi144 ceph-mon[33633]: pgmap v1302: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:47:24.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:47:23 smithi097 ceph-mon[39416]: pgmap v1302: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:47:24.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:47:23 smithi097 ceph-mon[30040]: pgmap v1302: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:47:24.871 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:47:24 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:47:26.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:47:25 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:47:26.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:47:25 smithi144 ceph-mon[33633]: pgmap v1303: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:47:26.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:47:25 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:47:26.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:47:25 smithi097 ceph-mon[39416]: pgmap v1303: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:47:26.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:47:25 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:47:26.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:47:25 smithi097 ceph-mon[30040]: pgmap v1303: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:47:27.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:47:26 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:47:27.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:47:26 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:47:27.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:47:26 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:47:28.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:47:27 smithi144 ceph-mon[33633]: pgmap v1304: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:47:28.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:47:27 smithi097 ceph-mon[39416]: pgmap v1304: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:47:28.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:47:27 smithi097 ceph-mon[30040]: pgmap v1304: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:47:30.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:47:29 smithi144 ceph-mon[33633]: pgmap v1305: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:47:30.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:47:29 smithi097 ceph-mon[39416]: pgmap v1305: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:47:30.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:47:29 smithi097 ceph-mon[30040]: pgmap v1305: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:47:32.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:47:31 smithi144 ceph-mon[33633]: pgmap v1306: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:47:32.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:47:31 smithi097 ceph-mon[39416]: pgmap v1306: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:47:32.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:47:31 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:47:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:47:32.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:47:31 smithi097 ceph-mon[30040]: pgmap v1306: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:47:34.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:47:33 smithi144 ceph-mon[33633]: pgmap v1307: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:47:34.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:47:33 smithi097 ceph-mon[39416]: pgmap v1307: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:47:34.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:47:33 smithi097 ceph-mon[30040]: pgmap v1307: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:47:34.871 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:47:34 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:47:36.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:47:35 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:47:36.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:47:35 smithi144 ceph-mon[33633]: pgmap v1308: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:47:36.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:47:35 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:47:36.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:47:35 smithi097 ceph-mon[30040]: pgmap v1308: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:47:36.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:47:35 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:47:36.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:47:35 smithi097 ceph-mon[39416]: pgmap v1308: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:47:38.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:47:37 smithi144 ceph-mon[33633]: pgmap v1309: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:47:38.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:47:37 smithi097 ceph-mon[39416]: pgmap v1309: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:47:38.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:47:37 smithi097 ceph-mon[30040]: pgmap v1309: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:47:40.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:47:39 smithi144 ceph-mon[33633]: pgmap v1310: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:47:40.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:47:39 smithi097 ceph-mon[39416]: pgmap v1310: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:47:40.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:47:39 smithi097 ceph-mon[30040]: pgmap v1310: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:47:42.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:47:41 smithi097 ceph-mon[30040]: pgmap v1311: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:47:42.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:47:41 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:47:42.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:47:41 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:47:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:47:42.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:47:41 smithi097 ceph-mon[39416]: pgmap v1311: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:47:42.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:47:41 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:47:42.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:47:41 smithi144 ceph-mon[33633]: pgmap v1311: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:47:42.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:47:41 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:47:44.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:47:43 smithi097 ceph-mon[30040]: pgmap v1312: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:47:44.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:47:43 smithi097 ceph-mon[39416]: pgmap v1312: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:47:44.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:47:43 smithi144 ceph-mon[33633]: pgmap v1312: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:47:44.871 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:47:44 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:47:46.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:47:45 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:47:46.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:47:45 smithi097 ceph-mon[39416]: pgmap v1313: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:47:46.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:47:45 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:47:46.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:47:45 smithi097 ceph-mon[30040]: pgmap v1313: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:47:46.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:47:45 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:47:46.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:47:45 smithi144 ceph-mon[33633]: pgmap v1313: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:47:48.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:47:47 smithi097 ceph-mon[39416]: pgmap v1314: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:47:48.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:47:47 smithi097 ceph-mon[30040]: pgmap v1314: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:47:48.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:47:47 smithi144 ceph-mon[33633]: pgmap v1314: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:47:50.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:47:49 smithi097 ceph-mon[39416]: pgmap v1315: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:47:50.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:47:49 smithi097 ceph-mon[30040]: pgmap v1315: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:47:50.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:47:49 smithi144 ceph-mon[33633]: pgmap v1315: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:47:51.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:47:50 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:47:51.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:47:50 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:47:51.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:47:50 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:47:52.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:47:51 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:47:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:47:52.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:47:51 smithi097 ceph-mon[30040]: pgmap v1316: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:47:52.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:47:51 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:47:52.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:47:51 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:47:52.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:47:51 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:47:52.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:47:51 smithi097 ceph-mon[39416]: pgmap v1316: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:47:52.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:47:51 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:47:52.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:47:51 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:47:52.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:47:51 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:47:52.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:47:51 smithi144 ceph-mon[33633]: pgmap v1316: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:47:52.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:47:51 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:47:52.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:47:51 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:47:52.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:47:51 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:47:54.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:47:53 smithi097 ceph-mon[39416]: pgmap v1317: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:47:54.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:47:53 smithi097 ceph-mon[30040]: pgmap v1317: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:47:54.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:47:53 smithi144 ceph-mon[33633]: pgmap v1317: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:47:54.871 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:47:54 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:47:54.871 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:47:54 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=cleanup t=2024-03-21T13:47:54.663298333Z level=info msg="Completed cleanup jobs" duration=6.815407ms 2024-03-21T13:47:56.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:47:55 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:47:56.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:47:55 smithi097 ceph-mon[39416]: pgmap v1318: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:47:56.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:47:55 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:47:56.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:47:55 smithi097 ceph-mon[30040]: pgmap v1318: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:47:56.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:47:55 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:47:56.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:47:55 smithi144 ceph-mon[33633]: pgmap v1318: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:47:57.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:47:56 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:47:57.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:47:56 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:47:57.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:47:56 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:47:58.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:47:57 smithi097 ceph-mon[39416]: pgmap v1319: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:47:58.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:47:57 smithi097 ceph-mon[30040]: pgmap v1319: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:47:58.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:47:57 smithi144 ceph-mon[33633]: pgmap v1319: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:48:00.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:47:59 smithi097 ceph-mon[39416]: pgmap v1320: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:48:00.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:47:59 smithi097 ceph-mon[30040]: pgmap v1320: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:48:00.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:47:59 smithi144 ceph-mon[33633]: pgmap v1320: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:48:02.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:48:01 smithi097 ceph-mon[30040]: pgmap v1321: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:48:02.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:48:01 smithi097 ceph-mon[39416]: pgmap v1321: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:48:02.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:48:01 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:48:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:48:02.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:48:01 smithi144 ceph-mon[33633]: pgmap v1321: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:48:04.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:48:03 smithi097 ceph-mon[30040]: pgmap v1322: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:48:04.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:48:03 smithi097 ceph-mon[39416]: pgmap v1322: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:48:04.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:48:03 smithi144 ceph-mon[33633]: pgmap v1322: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:48:04.871 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:48:04 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:48:06.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:48:05 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:48:06.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:48:05 smithi097 ceph-mon[39416]: pgmap v1323: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:48:06.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:48:05 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:48:06.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:48:05 smithi097 ceph-mon[30040]: pgmap v1323: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:48:06.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:48:05 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:48:06.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:48:05 smithi144 ceph-mon[33633]: pgmap v1323: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:48:08.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:48:07 smithi097 ceph-mon[39416]: pgmap v1324: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:48:08.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:48:07 smithi097 ceph-mon[30040]: pgmap v1324: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:48:08.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:48:07 smithi144 ceph-mon[33633]: pgmap v1324: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:48:10.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:48:09 smithi097 ceph-mon[39416]: pgmap v1325: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:48:10.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:48:09 smithi097 ceph-mon[30040]: pgmap v1325: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:48:10.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:48:09 smithi144 ceph-mon[33633]: pgmap v1325: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:48:12.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:48:11 smithi097 ceph-mon[39416]: pgmap v1326: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:48:12.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:48:11 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:48:12.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:48:11 smithi097 ceph-mon[30040]: pgmap v1326: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:48:12.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:48:11 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:48:12.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:48:11 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:48:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:48:12.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:48:11 smithi144 ceph-mon[33633]: pgmap v1326: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:48:12.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:48:11 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:48:14.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:48:13 smithi097 ceph-mon[39416]: pgmap v1327: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:48:14.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:48:13 smithi097 ceph-mon[30040]: pgmap v1327: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:48:14.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:48:13 smithi144 ceph-mon[33633]: pgmap v1327: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:48:14.871 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:48:14 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:48:16.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:48:15 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:48:16.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:48:15 smithi097 ceph-mon[39416]: pgmap v1328: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:48:16.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:48:15 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:48:16.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:48:15 smithi097 ceph-mon[30040]: pgmap v1328: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:48:16.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:48:15 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:48:16.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:48:15 smithi144 ceph-mon[33633]: pgmap v1328: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:48:18.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:48:17 smithi097 ceph-mon[39416]: pgmap v1329: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:48:18.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:48:17 smithi097 ceph-mon[30040]: pgmap v1329: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:48:18.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:48:17 smithi144 ceph-mon[33633]: pgmap v1329: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:48:19.018 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.019 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.019 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.019 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.019 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.019 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.019 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.019 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.019 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.019 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.019 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.019 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.019 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.019 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.019 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.020 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.020 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.020 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.020 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.020 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.020 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.020 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.020 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.020 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.020 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.020 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.020 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.020 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.020 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.021 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.021 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.021 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.021 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.021 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.021 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.021 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.021 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.021 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.021 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.021 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.021 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.021 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.021 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.021 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.021 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.021 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.022 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.022 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.022 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.022 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.022 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.022 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.022 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.022 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.022 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.022 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.022 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.022 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.022 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.022 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.022 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.022 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.023 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.023 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.023 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.023 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.023 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.023 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.023 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.023 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.023 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.023 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.023 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.023 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.023 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.023 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.023 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.023 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.024 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.024 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.024 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.024 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.024 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.024 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.024 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.024 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.024 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.024 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.024 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.024 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.024 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.024 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.024 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.024 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.024 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.025 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.025 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.025 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.025 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:19.025 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:48:20.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:48:19 smithi097 ceph-mon[39416]: pgmap v1330: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:48:20.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:48:19 smithi097 ceph-mon[30040]: pgmap v1330: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:48:20.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:48:19 smithi144 ceph-mon[33633]: pgmap v1330: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:48:22.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:48:21 smithi097 ceph-mon[39416]: pgmap v1331: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:48:22.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:48:21 smithi097 ceph-mon[30040]: pgmap v1331: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:48:22.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:48:21 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:48:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:48:22.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:48:21 smithi144 ceph-mon[33633]: pgmap v1331: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:48:24.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:48:23 smithi097 ceph-mon[39416]: pgmap v1332: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:48:24.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:48:23 smithi097 ceph-mon[30040]: pgmap v1332: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:48:24.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:48:23 smithi144 ceph-mon[33633]: pgmap v1332: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:48:24.871 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:48:24 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:48:26.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:48:25 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:48:26.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:48:25 smithi097 ceph-mon[39416]: pgmap v1333: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:48:26.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:48:25 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:48:26.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:48:25 smithi097 ceph-mon[30040]: pgmap v1333: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:48:26.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:48:25 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:48:26.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:48:25 smithi144 ceph-mon[33633]: pgmap v1333: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:48:27.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:48:26 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:48:27.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:48:26 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:48:27.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:48:26 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:48:28.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:48:27 smithi097 ceph-mon[30040]: pgmap v1334: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:48:28.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:48:27 smithi097 ceph-mon[39416]: pgmap v1334: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:48:28.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:48:27 smithi144 ceph-mon[33633]: pgmap v1334: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:48:30.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:48:29 smithi097 ceph-mon[30040]: pgmap v1335: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:48:30.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:48:29 smithi097 ceph-mon[39416]: pgmap v1335: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:48:30.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:48:29 smithi144 ceph-mon[33633]: pgmap v1335: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:48:32.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:48:31 smithi097 ceph-mon[30040]: pgmap v1336: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:48:32.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:48:31 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:48:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:48:32.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:48:31 smithi097 ceph-mon[39416]: pgmap v1336: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:48:32.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:48:31 smithi144 ceph-mon[33633]: pgmap v1336: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:48:34.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:48:33 smithi097 ceph-mon[30040]: pgmap v1337: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:48:34.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:48:33 smithi097 ceph-mon[39416]: pgmap v1337: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:48:34.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:48:33 smithi144 ceph-mon[33633]: pgmap v1337: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:48:34.871 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:48:34 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:48:36.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:48:35 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:48:36.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:48:35 smithi097 ceph-mon[39416]: pgmap v1338: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:48:36.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:48:35 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:48:36.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:48:35 smithi097 ceph-mon[30040]: pgmap v1338: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:48:36.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:48:35 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:48:36.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:48:35 smithi144 ceph-mon[33633]: pgmap v1338: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:48:38.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:48:37 smithi097 ceph-mon[39416]: pgmap v1339: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:48:38.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:48:37 smithi097 ceph-mon[30040]: pgmap v1339: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:48:38.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:48:37 smithi144 ceph-mon[33633]: pgmap v1339: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:48:40.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:48:39 smithi097 ceph-mon[39416]: pgmap v1340: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:48:40.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:48:39 smithi097 ceph-mon[30040]: pgmap v1340: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:48:40.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:48:39 smithi144 ceph-mon[33633]: pgmap v1340: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:48:42.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:48:41 smithi097 ceph-mon[39416]: pgmap v1341: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:48:42.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:48:41 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:48:42.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:48:41 smithi097 ceph-mon[30040]: pgmap v1341: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:48:42.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:48:41 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:48:42.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:48:41 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:48:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:48:42.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:48:41 smithi144 ceph-mon[33633]: pgmap v1341: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:48:42.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:48:41 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:48:44.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:48:43 smithi097 ceph-mon[39416]: pgmap v1342: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:48:44.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:48:43 smithi097 ceph-mon[30040]: pgmap v1342: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:48:44.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:48:43 smithi144 ceph-mon[33633]: pgmap v1342: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:48:44.871 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:48:44 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:48:46.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:48:46 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:48:46.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:48:46 smithi097 ceph-mon[39416]: pgmap v1343: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:48:46.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:48:45 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:48:46.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:48:45 smithi097 ceph-mon[30040]: pgmap v1343: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:48:46.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:48:46 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:48:46.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:48:46 smithi144 ceph-mon[33633]: pgmap v1343: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:48:48.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:48:48 smithi097 ceph-mon[39416]: pgmap v1344: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:48:48.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:48:48 smithi097 ceph-mon[30040]: pgmap v1344: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:48:48.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:48:48 smithi144 ceph-mon[33633]: pgmap v1344: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:48:50.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:48:50 smithi097 ceph-mon[39416]: pgmap v1345: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:48:50.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:48:50 smithi097 ceph-mon[30040]: pgmap v1345: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:48:50.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:48:50 smithi144 ceph-mon[33633]: pgmap v1345: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:48:52.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:48:51 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:48:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:48:52.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:48:52 smithi097 ceph-mon[39416]: pgmap v1346: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:48:52.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:48:52 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:48:52.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:48:52 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:48:52.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:48:52 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:48:52.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:48:52 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:48:52.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:48:52 smithi097 ceph-mon[30040]: pgmap v1346: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:48:52.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:48:52 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:48:52.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:48:52 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:48:52.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:48:52 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:48:52.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:48:52 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:48:52.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:48:52 smithi144 ceph-mon[33633]: pgmap v1346: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:48:52.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:48:52 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:48:52.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:48:52 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:48:52.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:48:52 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:48:52.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:48:52 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:48:54.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:48:54 smithi097 ceph-mon[39416]: pgmap v1347: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:48:54.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:48:54 smithi097 ceph-mon[30040]: pgmap v1347: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:48:54.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:48:54 smithi144 ceph-mon[33633]: pgmap v1347: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:48:54.871 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:48:54 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:48:56.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:48:56 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:48:56.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:48:56 smithi097 ceph-mon[39416]: pgmap v1348: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:48:56.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:48:56 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:48:56.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:48:56 smithi097 ceph-mon[30040]: pgmap v1348: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:48:56.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:48:56 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:48:56.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:48:56 smithi144 ceph-mon[33633]: pgmap v1348: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:48:57.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:48:57 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:48:57.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:48:57 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:48:57.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:48:57 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:48:58.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:48:58 smithi097 ceph-mon[39416]: pgmap v1349: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:48:58.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:48:58 smithi097 ceph-mon[30040]: pgmap v1349: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:48:58.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:48:58 smithi144 ceph-mon[33633]: pgmap v1349: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:49:00.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:49:00 smithi097 ceph-mon[39416]: pgmap v1350: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:49:00.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:49:00 smithi097 ceph-mon[30040]: pgmap v1350: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:49:00.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:49:00 smithi144 ceph-mon[33633]: pgmap v1350: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:49:02.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:49:02 smithi097 ceph-mon[39416]: pgmap v1351: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:49:02.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:49:02 smithi097 ceph-mon[30040]: pgmap v1351: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:49:02.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:49:01 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:49:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:49:02.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:49:02 smithi144 ceph-mon[33633]: pgmap v1351: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:49:04.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:49:04 smithi097 ceph-mon[39416]: pgmap v1352: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:49:04.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:49:04 smithi097 ceph-mon[30040]: pgmap v1352: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:49:04.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:49:04 smithi144 ceph-mon[33633]: pgmap v1352: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:49:04.871 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:49:04 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:49:06.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:49:06 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:49:06.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:49:06 smithi097 ceph-mon[39416]: pgmap v1353: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:49:06.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:49:06 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:49:06.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:49:06 smithi097 ceph-mon[30040]: pgmap v1353: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:49:06.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:49:06 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:49:06.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:49:06 smithi144 ceph-mon[33633]: pgmap v1353: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:49:08.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:49:08 smithi097 ceph-mon[39416]: pgmap v1354: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:49:08.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:49:08 smithi097 ceph-mon[30040]: pgmap v1354: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:49:08.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:49:08 smithi144 ceph-mon[33633]: pgmap v1354: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:49:10.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:49:10 smithi097 ceph-mon[30040]: pgmap v1355: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:49:10.306 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:49:10 smithi097 ceph-mon[39416]: pgmap v1355: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:49:10.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:49:10 smithi144 ceph-mon[33633]: pgmap v1355: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:49:12.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:49:11 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:49:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:49:12.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:49:12 smithi097 ceph-mon[39416]: pgmap v1356: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:49:12.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:49:12 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:49:12.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:49:12 smithi097 ceph-mon[30040]: pgmap v1356: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:49:12.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:49:12 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:49:12.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:49:12 smithi144 ceph-mon[33633]: pgmap v1356: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:49:12.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:49:12 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:49:14.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:49:14 smithi097 ceph-mon[39416]: pgmap v1357: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:49:14.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:49:14 smithi097 ceph-mon[30040]: pgmap v1357: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:49:14.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:49:14 smithi144 ceph-mon[33633]: pgmap v1357: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:49:14.871 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:49:14 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:49:16.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:49:16 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:49:16.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:49:16 smithi097 ceph-mon[39416]: pgmap v1358: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:49:16.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:49:16 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:49:16.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:49:16 smithi097 ceph-mon[30040]: pgmap v1358: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:49:16.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:49:16 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:49:16.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:49:16 smithi144 ceph-mon[33633]: pgmap v1358: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:49:18.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:49:18 smithi097 ceph-mon[30040]: pgmap v1359: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:49:18.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:49:18 smithi097 ceph-mon[39416]: pgmap v1359: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:49:18.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:49:18 smithi144 ceph-mon[33633]: pgmap v1359: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:49:20.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:49:20 smithi144 ceph-mon[33633]: pgmap v1360: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:49:20.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:49:20 smithi097 ceph-mon[39416]: pgmap v1360: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:49:20.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:49:20 smithi097 ceph-mon[30040]: pgmap v1360: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:49:22.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:49:22 smithi097 ceph-mon[30040]: pgmap v1361: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:49:22.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:49:22 smithi097 ceph-mon[39416]: pgmap v1361: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:49:22.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:49:21 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:49:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:49:22.374 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:49:22 smithi144 ceph-mon[33633]: pgmap v1361: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:49:24.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:49:24 smithi144 ceph-mon[33633]: pgmap v1362: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:49:24.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:49:24 smithi097 ceph-mon[30040]: pgmap v1362: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:49:24.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:49:24 smithi097 ceph-mon[39416]: pgmap v1362: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:49:24.871 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:49:24 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:49:26.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:49:26 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:49:26.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:49:26 smithi144 ceph-mon[33633]: pgmap v1363: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:49:26.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:49:26 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:49:26.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:49:26 smithi097 ceph-mon[39416]: pgmap v1363: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:49:26.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:49:26 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:49:26.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:49:26 smithi097 ceph-mon[30040]: pgmap v1363: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:49:27.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:49:27 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:49:27.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:49:27 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:49:27.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:49:27 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:49:28.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:49:28 smithi144 ceph-mon[33633]: pgmap v1364: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-21T13:49:28.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:49:28 smithi097 ceph-mon[39416]: pgmap v1364: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-21T13:49:28.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:49:28 smithi097 ceph-mon[30040]: pgmap v1364: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-21T13:49:30.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:49:30 smithi144 ceph-mon[33633]: pgmap v1365: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:49:30.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:49:30 smithi097 ceph-mon[39416]: pgmap v1365: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:49:30.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:49:30 smithi097 ceph-mon[30040]: pgmap v1365: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:49:32.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:49:32 smithi097 ceph-mon[39416]: pgmap v1366: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-21T13:49:32.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:49:32 smithi097 ceph-mon[30040]: pgmap v1366: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-21T13:49:32.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:49:31 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:49:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:49:32.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:49:32 smithi144 ceph-mon[33633]: pgmap v1366: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-21T13:49:34.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:49:34 smithi144 ceph-mon[33633]: pgmap v1367: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-21T13:49:34.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:49:34 smithi097 ceph-mon[39416]: pgmap v1367: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-21T13:49:34.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:49:34 smithi097 ceph-mon[30040]: pgmap v1367: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-21T13:49:34.871 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:49:34 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:49:36.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:49:36 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:49:36.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:49:36 smithi144 ceph-mon[33633]: pgmap v1368: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:49:36.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:49:36 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:49:36.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:49:36 smithi097 ceph-mon[39416]: pgmap v1368: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:49:36.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:49:36 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:49:36.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:49:36 smithi097 ceph-mon[30040]: pgmap v1368: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:49:38.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:49:38 smithi144 ceph-mon[33633]: pgmap v1369: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:49:38.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:49:38 smithi097 ceph-mon[39416]: pgmap v1369: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:49:38.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:49:38 smithi097 ceph-mon[30040]: pgmap v1369: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:49:40.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:49:40 smithi144 ceph-mon[33633]: pgmap v1370: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:49:40.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:49:40 smithi097 ceph-mon[39416]: pgmap v1370: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:49:40.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:49:40 smithi097 ceph-mon[30040]: pgmap v1370: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:49:42.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:49:42 smithi097 ceph-mon[30040]: pgmap v1371: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:49:42.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:49:42 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:49:42.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:49:41 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:49:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:49:42.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:49:42 smithi097 ceph-mon[39416]: pgmap v1371: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:49:42.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:49:42 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:49:42.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:49:42 smithi144 ceph-mon[33633]: pgmap v1371: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:49:42.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:49:42 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:49:44.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:49:44 smithi144 ceph-mon[33633]: pgmap v1372: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:49:44.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:49:44 smithi097 ceph-mon[39416]: pgmap v1372: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:49:44.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:49:44 smithi097 ceph-mon[30040]: pgmap v1372: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:49:44.871 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:49:44 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:49:46.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:49:46 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:49:46.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:49:46 smithi144 ceph-mon[33633]: pgmap v1373: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:49:46.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:49:46 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:49:46.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:49:46 smithi097 ceph-mon[39416]: pgmap v1373: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:49:46.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:49:46 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:49:46.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:49:46 smithi097 ceph-mon[30040]: pgmap v1373: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:49:48.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:49:48 smithi144 ceph-mon[33633]: pgmap v1374: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:49:48.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:49:48 smithi097 ceph-mon[30040]: pgmap v1374: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:49:48.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:49:48 smithi097 ceph-mon[39416]: pgmap v1374: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:49:50.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:49:50 smithi144 ceph-mon[33633]: pgmap v1375: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:49:50.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:49:50 smithi097 ceph-mon[39416]: pgmap v1375: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:49:50.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:49:50 smithi097 ceph-mon[30040]: pgmap v1375: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:49:52.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:49:52 smithi097 ceph-mon[39416]: pgmap v1376: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:49:52.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:49:52 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:49:52.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:49:52 smithi097 ceph-mon[30040]: pgmap v1376: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:49:52.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:49:52 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:49:52.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:49:51 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:49:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:49:52.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:49:52 smithi144 ceph-mon[33633]: pgmap v1376: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:49:52.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:49:52 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:49:54.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:49:54 smithi144 ceph-mon[33633]: pgmap v1377: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:49:54.408 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:49:54 smithi097 ceph-mon[39416]: pgmap v1377: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:49:54.408 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:49:54 smithi097 ceph-mon[30040]: pgmap v1377: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:49:54.871 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:49:54 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:49:55.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:49:55 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:49:55.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:49:55 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:49:55.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:49:55 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:49:55.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:49:55 smithi144 ceph-mon[33633]: pgmap v1378: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:49:56.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:49:55 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:49:56.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:49:55 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:49:56.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:49:55 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:49:56.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:49:55 smithi097 ceph-mon[39416]: pgmap v1378: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:49:56.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:49:55 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:49:56.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:49:55 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:49:56.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:49:55 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:49:56.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:49:55 smithi097 ceph-mon[30040]: pgmap v1378: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:49:57.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:49:56 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:49:57.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:49:56 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:49:57.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:49:56 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:49:57.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:49:56 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:49:57.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:49:56 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:49:57.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:49:56 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:49:57.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:49:56 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:49:57.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:49:56 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:49:57.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:49:56 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:49:57.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:49:56 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:49:57.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:49:56 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:49:57.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:49:56 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:49:57.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:49:56 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:49:57.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:49:56 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:49:57.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:49:56 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:49:57.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:49:56 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:49:57.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:49:56 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:49:57.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:49:56 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:49:58.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:49:57 smithi097 ceph-mon[39416]: pgmap v1379: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:49:58.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:49:57 smithi097 ceph-mon[30040]: pgmap v1379: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:49:58.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:49:57 smithi144 ceph-mon[33633]: pgmap v1379: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:49:59.030 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.030 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.031 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.031 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.031 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.031 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.031 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.031 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.031 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.031 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.031 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.031 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.032 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.032 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.032 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.032 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.032 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.032 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.032 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.032 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.032 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.032 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.032 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.032 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.032 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.033 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.033 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.033 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.033 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.033 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.033 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.033 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.033 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.033 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.033 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.033 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.034 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.034 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.034 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.034 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.034 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.034 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.034 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.034 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.034 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.034 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.034 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.034 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.035 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.035 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.035 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.035 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.035 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.035 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.035 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.035 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.035 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.035 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.035 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.035 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.035 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.036 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.036 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.036 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.036 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.036 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.036 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.036 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.036 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.036 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.036 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.036 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.036 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.036 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.037 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.037 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.037 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.037 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.037 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.037 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.037 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.037 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.037 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.037 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.037 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.037 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.038 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.038 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.038 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.038 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.038 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.038 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.038 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.038 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.038 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.038 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.038 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.039 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.039 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:49:59.039 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:50:00.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:49:59 smithi097 ceph-mon[30040]: pgmap v1380: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:50:00.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:49:59 smithi097 ceph-mon[39416]: pgmap v1380: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:50:00.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:49:59 smithi144 ceph-mon[33633]: pgmap v1380: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:50:01.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:50:00 smithi097 ceph-mon[30040]: Health detail: HEALTH_WARN 4 pool(s) do not have an application enabled 2024-03-21T13:50:01.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:50:00 smithi097 ceph-mon[30040]: [WRN] POOL_APP_NOT_ENABLED: 4 pool(s) do not have an application enabled 2024-03-21T13:50:01.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:50:00 smithi097 ceph-mon[30040]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-03-21T13:50:01.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:50:00 smithi097 ceph-mon[30040]: application not enabled on pool 'poolrace.1804289383' 2024-03-21T13:50:01.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:50:00 smithi097 ceph-mon[30040]: application not enabled on pool 'AssertExistssmithi097-80928-1' 2024-03-21T13:50:01.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:50:00 smithi097 ceph-mon[30040]: application not enabled on pool 'poolrace2.846930886' 2024-03-21T13:50:01.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:50:00 smithi097 ceph-mon[30040]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-03-21T13:50:01.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:50:00 smithi097 ceph-mon[39416]: Health detail: HEALTH_WARN 4 pool(s) do not have an application enabled 2024-03-21T13:50:01.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:50:00 smithi097 ceph-mon[39416]: [WRN] POOL_APP_NOT_ENABLED: 4 pool(s) do not have an application enabled 2024-03-21T13:50:01.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:50:00 smithi097 ceph-mon[39416]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-03-21T13:50:01.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:50:00 smithi097 ceph-mon[39416]: application not enabled on pool 'poolrace.1804289383' 2024-03-21T13:50:01.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:50:00 smithi097 ceph-mon[39416]: application not enabled on pool 'AssertExistssmithi097-80928-1' 2024-03-21T13:50:01.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:50:00 smithi097 ceph-mon[39416]: application not enabled on pool 'poolrace2.846930886' 2024-03-21T13:50:01.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:50:00 smithi097 ceph-mon[39416]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-03-21T13:50:01.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:50:00 smithi144 ceph-mon[33633]: Health detail: HEALTH_WARN 4 pool(s) do not have an application enabled 2024-03-21T13:50:01.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:50:00 smithi144 ceph-mon[33633]: [WRN] POOL_APP_NOT_ENABLED: 4 pool(s) do not have an application enabled 2024-03-21T13:50:01.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:50:00 smithi144 ceph-mon[33633]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-03-21T13:50:01.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:50:00 smithi144 ceph-mon[33633]: application not enabled on pool 'poolrace.1804289383' 2024-03-21T13:50:01.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:50:00 smithi144 ceph-mon[33633]: application not enabled on pool 'AssertExistssmithi097-80928-1' 2024-03-21T13:50:01.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:50:00 smithi144 ceph-mon[33633]: application not enabled on pool 'poolrace2.846930886' 2024-03-21T13:50:01.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:50:00 smithi144 ceph-mon[33633]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-03-21T13:50:02.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:50:01 smithi097 ceph-mon[39416]: pgmap v1381: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:50:02.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:50:01 smithi097 ceph-mon[30040]: pgmap v1381: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:50:02.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:50:01 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:50:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:50:02.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:50:01 smithi144 ceph-mon[33633]: pgmap v1381: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:50:04.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:50:03 smithi097 ceph-mon[39416]: pgmap v1382: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:50:04.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:50:03 smithi097 ceph-mon[30040]: pgmap v1382: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:50:04.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:50:03 smithi144 ceph-mon[33633]: pgmap v1382: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:50:04.871 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:50:04 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:50:06.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:50:05 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:50:06.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:50:05 smithi097 ceph-mon[39416]: pgmap v1383: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:50:06.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:50:05 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:50:06.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:50:05 smithi097 ceph-mon[30040]: pgmap v1383: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:50:06.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:50:05 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:50:06.422 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:50:05 smithi144 ceph-mon[33633]: pgmap v1383: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:50:08.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:50:07 smithi097 ceph-mon[39416]: pgmap v1384: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:50:08.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:50:07 smithi097 ceph-mon[30040]: pgmap v1384: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:50:08.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:50:07 smithi144 ceph-mon[33633]: pgmap v1384: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:50:10.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:50:09 smithi097 ceph-mon[30040]: pgmap v1385: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:50:10.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:50:09 smithi097 ceph-mon[39416]: pgmap v1385: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:50:10.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:50:09 smithi144 ceph-mon[33633]: pgmap v1385: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:50:12.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:50:11 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:50:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:50:12.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:50:11 smithi097 ceph-mon[30040]: pgmap v1386: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:50:12.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:50:11 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:50:12.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:50:11 smithi097 ceph-mon[39416]: pgmap v1386: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:50:12.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:50:11 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:50:12.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:50:11 smithi144 ceph-mon[33633]: pgmap v1386: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:50:12.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:50:11 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:50:14.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:50:13 smithi097 ceph-mon[30040]: pgmap v1387: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:50:14.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:50:13 smithi097 ceph-mon[39416]: pgmap v1387: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:50:14.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:50:13 smithi144 ceph-mon[33633]: pgmap v1387: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:50:14.871 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:50:14 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:50:16.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:50:15 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:50:16.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:50:15 smithi097 ceph-mon[39416]: pgmap v1388: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:50:16.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:50:15 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:50:16.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:50:15 smithi097 ceph-mon[30040]: pgmap v1388: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:50:16.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:50:15 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:50:16.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:50:15 smithi144 ceph-mon[33633]: pgmap v1388: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:50:18.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:50:18 smithi097 ceph-mon[30040]: pgmap v1389: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:50:18.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:50:18 smithi097 ceph-mon[39416]: pgmap v1389: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:50:18.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:50:18 smithi144 ceph-mon[33633]: pgmap v1389: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:50:20.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:50:20 smithi097 ceph-mon[39416]: pgmap v1390: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:50:20.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:50:20 smithi097 ceph-mon[30040]: pgmap v1390: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:50:20.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:50:20 smithi144 ceph-mon[33633]: pgmap v1390: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:50:22.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:50:22 smithi097 ceph-mon[39416]: pgmap v1391: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:50:22.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:50:22 smithi097 ceph-mon[30040]: pgmap v1391: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:50:22.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:50:21 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:50:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:50:22.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:50:22 smithi144 ceph-mon[33633]: pgmap v1391: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:50:24.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:50:24 smithi097 ceph-mon[39416]: pgmap v1392: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:50:24.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:50:24 smithi097 ceph-mon[30040]: pgmap v1392: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:50:24.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:50:24 smithi144 ceph-mon[33633]: pgmap v1392: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:50:24.871 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:50:24 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:50:26.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:50:26 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:50:26.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:50:26 smithi097 ceph-mon[39416]: pgmap v1393: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:50:26.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:50:26 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:50:26.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:50:26 smithi097 ceph-mon[30040]: pgmap v1393: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:50:26.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:50:26 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:50:26.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:50:26 smithi144 ceph-mon[33633]: pgmap v1393: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:50:27.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:50:27 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:50:27.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:50:27 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:50:27.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:50:27 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:50:28.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:50:28 smithi097 ceph-mon[39416]: pgmap v1394: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:50:28.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:50:28 smithi097 ceph-mon[30040]: pgmap v1394: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:50:28.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:50:28 smithi144 ceph-mon[33633]: pgmap v1394: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:50:30.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:50:30 smithi097 ceph-mon[39416]: pgmap v1395: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:50:30.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:50:30 smithi097 ceph-mon[30040]: pgmap v1395: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:50:30.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:50:30 smithi144 ceph-mon[33633]: pgmap v1395: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:50:32.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:50:32 smithi097 ceph-mon[39416]: pgmap v1396: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:50:32.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:50:32 smithi097 ceph-mon[30040]: pgmap v1396: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:50:32.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:50:31 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:50:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:50:32.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:50:32 smithi144 ceph-mon[33633]: pgmap v1396: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:50:34.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:50:34 smithi097 ceph-mon[39416]: pgmap v1397: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:50:34.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:50:34 smithi097 ceph-mon[30040]: pgmap v1397: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:50:34.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:50:34 smithi144 ceph-mon[33633]: pgmap v1397: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:50:35.121 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:50:34 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:50:36.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:50:36 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:50:36.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:50:36 smithi097 ceph-mon[39416]: pgmap v1398: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:50:36.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:50:36 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:50:36.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:50:36 smithi097 ceph-mon[30040]: pgmap v1398: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:50:36.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:50:36 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:50:36.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:50:36 smithi144 ceph-mon[33633]: pgmap v1398: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:50:38.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:50:38 smithi097 ceph-mon[39416]: pgmap v1399: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:50:38.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:50:38 smithi097 ceph-mon[30040]: pgmap v1399: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:50:38.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:50:38 smithi144 ceph-mon[33633]: pgmap v1399: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:50:40.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:50:40 smithi097 ceph-mon[39416]: pgmap v1400: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:50:40.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:50:40 smithi097 ceph-mon[30040]: pgmap v1400: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:50:40.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:50:40 smithi144 ceph-mon[33633]: pgmap v1400: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:50:42.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:50:42 smithi097 ceph-mon[39416]: pgmap v1401: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:50:42.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:50:42 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:50:42.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:50:42 smithi097 ceph-mon[30040]: pgmap v1401: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:50:42.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:50:42 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:50:42.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:50:41 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:50:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:50:42.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:50:42 smithi144 ceph-mon[33633]: pgmap v1401: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:50:42.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:50:42 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:50:44.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:50:44 smithi097 ceph-mon[39416]: pgmap v1402: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:50:44.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:50:44 smithi097 ceph-mon[30040]: pgmap v1402: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:50:44.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:50:44 smithi144 ceph-mon[33633]: pgmap v1402: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:50:45.121 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:50:44 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:50:46.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:50:46 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:50:46.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:50:46 smithi097 ceph-mon[39416]: pgmap v1403: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:50:46.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:50:46 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:50:46.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:50:46 smithi097 ceph-mon[30040]: pgmap v1403: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:50:46.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:50:46 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:50:46.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:50:46 smithi144 ceph-mon[33633]: pgmap v1403: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:50:48.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:50:48 smithi097 ceph-mon[39416]: pgmap v1404: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:50:48.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:50:48 smithi097 ceph-mon[30040]: pgmap v1404: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:50:48.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:50:48 smithi144 ceph-mon[33633]: pgmap v1404: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:50:50.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:50:50 smithi097 ceph-mon[30040]: pgmap v1405: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:50:50.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:50:50 smithi097 ceph-mon[39416]: pgmap v1405: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:50:50.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:50:50 smithi144 ceph-mon[33633]: pgmap v1405: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:50:52.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:50:52 smithi097 ceph-mon[39416]: pgmap v1406: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:50:52.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:50:52 smithi097 ceph-mon[30040]: pgmap v1406: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:50:52.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:50:51 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:50:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:50:52.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:50:52 smithi144 ceph-mon[33633]: pgmap v1406: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:50:54.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:50:54 smithi144 ceph-mon[33633]: pgmap v1407: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:50:54.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:50:54 smithi097 ceph-mon[39416]: pgmap v1407: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:50:54.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:50:54 smithi097 ceph-mon[30040]: pgmap v1407: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:50:55.121 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:50:54 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:50:56.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:50:56 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:50:56.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:50:56 smithi144 ceph-mon[33633]: pgmap v1408: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:50:56.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:50:56 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:50:56.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:50:56 smithi097 ceph-mon[39416]: pgmap v1408: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:50:56.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:50:56 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:50:56.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:50:56 smithi097 ceph-mon[30040]: pgmap v1408: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:50:57.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:50:57 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:50:57.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:50:57 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:50:57.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:50:57 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:50:57.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:50:57 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:50:57.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:50:57 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:50:57.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:50:57 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:50:58.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:50:58 smithi144 ceph-mon[33633]: pgmap v1409: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:50:58.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:50:58 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:50:58.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:50:58 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:50:58.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:50:58 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:50:58.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:50:58 smithi097 ceph-mon[39416]: pgmap v1409: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:50:58.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:50:58 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:50:58.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:50:58 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:50:58.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:50:58 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:50:58.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:50:58 smithi097 ceph-mon[30040]: pgmap v1409: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:50:58.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:50:58 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:50:58.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:50:58 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:50:58.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:50:58 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:51:00.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:51:00 smithi144 ceph-mon[33633]: pgmap v1410: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:51:00.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:51:00 smithi097 ceph-mon[39416]: pgmap v1410: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:51:00.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:51:00 smithi097 ceph-mon[30040]: pgmap v1410: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:51:02.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:51:02 smithi097 ceph-mon[30040]: pgmap v1411: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:51:02.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:51:02 smithi097 ceph-mon[39416]: pgmap v1411: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:51:02.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:51:01 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:51:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:51:02.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:51:02 smithi144 ceph-mon[33633]: pgmap v1411: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:51:04.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:51:04 smithi144 ceph-mon[33633]: pgmap v1412: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:51:04.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:51:04 smithi097 ceph-mon[39416]: pgmap v1412: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:51:04.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:51:04 smithi097 ceph-mon[30040]: pgmap v1412: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:51:05.121 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:51:04 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:51:06.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:51:06 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:51:06.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:51:06 smithi144 ceph-mon[33633]: pgmap v1413: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:51:06.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:51:06 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:51:06.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:51:06 smithi097 ceph-mon[39416]: pgmap v1413: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:51:06.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:51:06 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:51:06.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:51:06 smithi097 ceph-mon[30040]: pgmap v1413: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:51:08.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:51:08 smithi144 ceph-mon[33633]: pgmap v1414: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:51:08.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:51:08 smithi097 ceph-mon[39416]: pgmap v1414: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:51:08.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:51:08 smithi097 ceph-mon[30040]: pgmap v1414: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:51:10.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:51:10 smithi144 ceph-mon[33633]: pgmap v1415: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:51:10.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:51:10 smithi097 ceph-mon[39416]: pgmap v1415: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:51:10.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:51:10 smithi097 ceph-mon[30040]: pgmap v1415: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:51:12.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:51:11 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:51:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:51:12.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:51:12 smithi097 ceph-mon[39416]: pgmap v1416: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:51:12.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:51:12 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:51:12.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:51:12 smithi097 ceph-mon[30040]: pgmap v1416: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:51:12.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:51:12 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:51:12.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:51:12 smithi144 ceph-mon[33633]: pgmap v1416: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:51:12.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:51:12 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:51:14.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:51:14 smithi144 ceph-mon[33633]: pgmap v1417: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:51:14.507 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:51:14 smithi097 ceph-mon[39416]: pgmap v1417: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:51:14.507 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:51:14 smithi097 ceph-mon[30040]: pgmap v1417: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:51:15.121 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:51:14 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:51:16.373 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:51:16 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:51:16.374 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:51:16 smithi144 ceph-mon[33633]: pgmap v1418: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:51:16.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:51:16 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:51:16.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:51:16 smithi097 ceph-mon[39416]: pgmap v1418: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:51:16.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:51:16 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:51:16.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:51:16 smithi097 ceph-mon[30040]: pgmap v1418: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:51:18.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:51:18 smithi144 ceph-mon[33633]: pgmap v1419: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:51:18.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:51:18 smithi097 ceph-mon[39416]: pgmap v1419: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:51:18.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:51:18 smithi097 ceph-mon[30040]: pgmap v1419: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:51:20.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:51:20 smithi144 ceph-mon[33633]: pgmap v1420: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:51:20.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:51:20 smithi097 ceph-mon[39416]: pgmap v1420: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:51:20.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:51:20 smithi097 ceph-mon[30040]: pgmap v1420: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:51:22.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:51:22 smithi097 ceph-mon[39416]: pgmap v1421: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:51:22.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:51:22 smithi097 ceph-mon[30040]: pgmap v1421: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:51:22.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:51:21 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:51:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:51:22.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:51:22 smithi144 ceph-mon[33633]: pgmap v1421: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:51:24.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:51:24 smithi097 ceph-mon[30040]: pgmap v1422: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:51:24.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:51:24 smithi097 ceph-mon[39416]: pgmap v1422: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:51:24.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:51:24 smithi144 ceph-mon[33633]: pgmap v1422: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:51:25.121 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:51:24 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:51:26.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:51:26 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:51:26.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:51:26 smithi144 ceph-mon[33633]: pgmap v1423: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:51:26.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:51:26 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:51:26.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:51:26 smithi097 ceph-mon[39416]: pgmap v1423: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:51:26.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:51:26 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:51:26.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:51:26 smithi097 ceph-mon[30040]: pgmap v1423: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:51:27.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:51:27 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:51:27.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:51:27 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:51:27.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:51:27 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:51:28.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:51:28 smithi097 ceph-mon[39416]: pgmap v1424: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:51:28.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:51:28 smithi097 ceph-mon[30040]: pgmap v1424: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:51:28.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:51:28 smithi144 ceph-mon[33633]: pgmap v1424: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:51:30.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:51:30 smithi097 ceph-mon[30040]: pgmap v1425: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:51:30.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:51:30 smithi097 ceph-mon[39416]: pgmap v1425: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:51:30.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:51:30 smithi144 ceph-mon[33633]: pgmap v1425: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:51:32.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:51:32 smithi097 ceph-mon[39416]: pgmap v1426: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:51:32.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:51:32 smithi097 ceph-mon[30040]: pgmap v1426: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:51:32.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:51:31 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:51:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:51:32.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:51:32 smithi144 ceph-mon[33633]: pgmap v1426: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:51:34.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:51:34 smithi097 ceph-mon[39416]: pgmap v1427: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:51:34.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:51:34 smithi097 ceph-mon[30040]: pgmap v1427: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:51:34.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:51:34 smithi144 ceph-mon[33633]: pgmap v1427: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:51:35.121 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:51:34 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:51:36.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:51:36 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:51:36.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:51:36 smithi097 ceph-mon[30040]: pgmap v1428: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:51:36.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:51:36 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:51:36.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:51:36 smithi097 ceph-mon[39416]: pgmap v1428: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:51:36.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:51:36 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:51:36.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:51:36 smithi144 ceph-mon[33633]: pgmap v1428: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:51:38.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:51:38 smithi097 ceph-mon[30040]: pgmap v1429: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:51:38.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:51:38 smithi097 ceph-mon[39416]: pgmap v1429: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:51:38.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:51:38 smithi144 ceph-mon[33633]: pgmap v1429: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:51:39.042 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.042 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.042 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.042 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.042 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.042 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.042 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.042 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.042 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.043 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.043 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.043 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.043 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.043 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.043 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.043 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.043 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.043 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.043 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.043 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.043 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.043 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.044 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.044 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.044 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.044 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.044 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.044 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.044 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.044 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.044 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.044 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.044 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.044 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.044 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.044 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.044 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.045 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.045 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.045 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.045 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.045 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.045 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.045 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.045 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.045 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.045 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.045 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.045 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.045 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.045 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.045 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.046 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.046 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.046 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.046 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.046 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.046 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.046 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.046 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.046 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.046 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.046 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.046 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.046 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.046 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.046 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.047 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.047 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.047 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.047 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.047 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.047 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.047 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.047 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.047 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.047 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.047 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.047 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.047 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.047 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.047 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.048 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.048 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.048 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.048 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.048 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.048 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.048 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.048 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.048 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.048 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.048 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.048 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.048 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.048 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.048 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.049 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.049 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:39.049 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:51:40.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:51:40 smithi097 ceph-mon[30040]: pgmap v1430: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:51:40.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:51:40 smithi097 ceph-mon[39416]: pgmap v1430: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:51:40.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:51:40 smithi144 ceph-mon[33633]: pgmap v1430: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:51:42.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:51:42 smithi097 ceph-mon[30040]: pgmap v1431: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:51:42.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:51:42 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:51:42.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:51:41 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:51:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:51:42.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:51:42 smithi097 ceph-mon[39416]: pgmap v1431: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:51:42.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:51:42 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:51:42.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:51:42 smithi144 ceph-mon[33633]: pgmap v1431: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:51:42.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:51:42 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:51:44.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:51:44 smithi097 ceph-mon[30040]: pgmap v1432: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:51:44.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:51:44 smithi097 ceph-mon[39416]: pgmap v1432: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:51:44.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:51:44 smithi144 ceph-mon[33633]: pgmap v1432: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:51:45.121 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:51:44 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:51:46.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:51:46 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:51:46.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:51:46 smithi097 ceph-mon[39416]: pgmap v1433: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:51:46.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:51:46 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:51:46.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:51:46 smithi097 ceph-mon[30040]: pgmap v1433: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:51:46.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:51:46 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:51:46.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:51:46 smithi144 ceph-mon[33633]: pgmap v1433: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:51:48.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:51:48 smithi097 ceph-mon[39416]: pgmap v1434: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:51:48.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:51:48 smithi097 ceph-mon[30040]: pgmap v1434: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:51:48.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:51:48 smithi144 ceph-mon[33633]: pgmap v1434: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:51:50.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:51:50 smithi097 ceph-mon[39416]: pgmap v1435: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:51:50.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:51:50 smithi097 ceph-mon[30040]: pgmap v1435: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:51:50.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:51:50 smithi144 ceph-mon[33633]: pgmap v1435: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:51:52.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:51:52 smithi097 ceph-mon[30040]: pgmap v1436: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:51:52.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:51:52 smithi097 ceph-mon[39416]: pgmap v1436: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:51:52.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:51:51 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:51:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:51:52.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:51:52 smithi144 ceph-mon[33633]: pgmap v1436: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:51:54.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:51:54 smithi097 ceph-mon[30040]: pgmap v1437: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:51:54.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:51:54 smithi097 ceph-mon[39416]: pgmap v1437: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:51:54.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:51:54 smithi144 ceph-mon[33633]: pgmap v1437: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:51:55.121 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:51:54 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:51:56.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:51:56 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:51:56.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:51:56 smithi097 ceph-mon[39416]: pgmap v1438: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:51:56.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:51:56 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:51:56.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:51:56 smithi097 ceph-mon[30040]: pgmap v1438: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:51:56.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:51:56 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:51:56.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:51:56 smithi144 ceph-mon[33633]: pgmap v1438: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:51:57.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:51:57 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:51:57.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:51:57 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:51:57.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:51:57 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:51:58.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:51:58 smithi097 ceph-mon[39416]: pgmap v1439: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:51:58.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:51:58 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:51:58.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:51:58 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:51:58.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:51:58 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:51:58.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:51:58 smithi097 ceph-mon[30040]: pgmap v1439: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:51:58.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:51:58 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:51:58.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:51:58 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:51:58.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:51:58 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:51:58.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:51:58 smithi144 ceph-mon[33633]: pgmap v1439: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:51:58.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:51:58 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:51:58.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:51:58 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:51:58.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:51:58 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:51:59.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:51:59 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:51:59.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:51:59 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:51:59.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:51:59 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:51:59.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:51:59 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:51:59.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:51:59 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:51:59.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:51:59 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:51:59.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:51:59 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:51:59.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:51:59 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:51:59.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:51:59 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:52:00.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:52:00 smithi097 ceph-mon[39416]: pgmap v1440: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:52:00.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:52:00 smithi097 ceph-mon[30040]: pgmap v1440: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:52:00.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:52:00 smithi144 ceph-mon[33633]: pgmap v1440: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:52:02.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:52:02 smithi097 ceph-mon[39416]: pgmap v1441: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:52:02.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:52:02 smithi097 ceph-mon[30040]: pgmap v1441: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:52:02.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:52:01 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:52:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:52:02.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:52:02 smithi144 ceph-mon[33633]: pgmap v1441: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:52:04.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:52:04 smithi097 ceph-mon[39416]: pgmap v1442: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:52:04.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:52:04 smithi097 ceph-mon[30040]: pgmap v1442: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:52:04.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:52:04 smithi144 ceph-mon[33633]: pgmap v1442: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:52:05.121 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:52:04 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:52:06.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:52:06 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:52:06.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:52:06 smithi097 ceph-mon[39416]: pgmap v1443: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:52:06.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:52:06 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:52:06.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:52:06 smithi097 ceph-mon[30040]: pgmap v1443: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:52:06.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:52:06 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:52:06.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:52:06 smithi144 ceph-mon[33633]: pgmap v1443: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:52:08.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:52:08 smithi097 ceph-mon[39416]: pgmap v1444: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:52:08.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:52:08 smithi097 ceph-mon[30040]: pgmap v1444: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:52:08.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:52:08 smithi144 ceph-mon[33633]: pgmap v1444: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:52:10.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:52:10 smithi097 ceph-mon[39416]: pgmap v1445: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:52:10.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:52:10 smithi097 ceph-mon[30040]: pgmap v1445: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:52:10.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:52:10 smithi144 ceph-mon[33633]: pgmap v1445: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:52:12.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:52:12 smithi097 ceph-mon[30040]: pgmap v1446: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:52:12.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:52:12 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:52:12.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:52:11 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:52:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:52:12.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:52:12 smithi097 ceph-mon[39416]: pgmap v1446: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:52:12.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:52:12 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:52:12.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:52:12 smithi144 ceph-mon[33633]: pgmap v1446: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:52:12.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:52:12 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:52:14.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:52:14 smithi097 ceph-mon[30040]: pgmap v1447: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:52:14.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:52:14 smithi097 ceph-mon[39416]: pgmap v1447: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:52:14.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:52:14 smithi144 ceph-mon[33633]: pgmap v1447: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:52:15.121 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:52:14 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:52:16.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:52:16 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:52:16.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:52:16 smithi097 ceph-mon[39416]: pgmap v1448: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:52:16.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:52:16 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:52:16.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:52:16 smithi097 ceph-mon[30040]: pgmap v1448: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:52:16.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:52:16 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:52:16.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:52:16 smithi144 ceph-mon[33633]: pgmap v1448: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:52:18.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:52:18 smithi097 ceph-mon[39416]: pgmap v1449: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:52:18.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:52:18 smithi097 ceph-mon[30040]: pgmap v1449: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:52:18.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:52:18 smithi144 ceph-mon[33633]: pgmap v1449: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:52:20.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:52:20 smithi097 ceph-mon[39416]: pgmap v1450: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:52:20.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:52:20 smithi097 ceph-mon[30040]: pgmap v1450: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:52:20.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:52:20 smithi144 ceph-mon[33633]: pgmap v1450: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:52:22.215 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:52:21 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:52:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:52:22.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:52:22 smithi097 ceph-mon[30040]: pgmap v1451: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:52:22.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:52:22 smithi097 ceph-mon[39416]: pgmap v1451: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:52:22.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:52:22 smithi144 ceph-mon[33633]: pgmap v1451: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:52:24.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:52:24 smithi097 ceph-mon[39416]: pgmap v1452: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:52:24.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:52:24 smithi097 ceph-mon[30040]: pgmap v1452: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:52:24.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:52:24 smithi144 ceph-mon[33633]: pgmap v1452: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:52:25.121 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:52:24 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:52:26.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:52:26 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:52:26.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:52:26 smithi097 ceph-mon[39416]: pgmap v1453: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:52:26.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:52:26 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:52:26.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:52:26 smithi097 ceph-mon[30040]: pgmap v1453: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:52:26.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:52:26 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:52:26.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:52:26 smithi144 ceph-mon[33633]: pgmap v1453: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:52:27.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:52:27 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:52:27.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:52:27 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:52:27.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:52:27 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:52:28.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:52:28 smithi097 ceph-mon[39416]: pgmap v1454: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:52:28.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:52:28 smithi097 ceph-mon[30040]: pgmap v1454: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:52:28.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:52:28 smithi144 ceph-mon[33633]: pgmap v1454: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:52:30.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:52:30 smithi097 ceph-mon[30040]: pgmap v1455: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:52:30.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:52:30 smithi097 ceph-mon[39416]: pgmap v1455: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:52:30.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:52:30 smithi144 ceph-mon[33633]: pgmap v1455: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:52:32.233 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:52:31 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:52:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:52:32.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:52:32 smithi097 ceph-mon[30040]: pgmap v1456: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:52:32.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:52:32 smithi097 ceph-mon[39416]: pgmap v1456: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:52:32.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:52:32 smithi144 ceph-mon[33633]: pgmap v1456: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:52:34.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:52:34 smithi097 ceph-mon[39416]: pgmap v1457: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:52:34.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:52:34 smithi097 ceph-mon[30040]: pgmap v1457: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:52:34.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:52:34 smithi144 ceph-mon[33633]: pgmap v1457: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:52:35.121 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:52:34 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:52:36.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:52:36 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:52:36.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:52:36 smithi097 ceph-mon[39416]: pgmap v1458: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:52:36.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:52:36 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:52:36.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:52:36 smithi097 ceph-mon[30040]: pgmap v1458: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:52:36.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:52:36 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:52:36.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:52:36 smithi144 ceph-mon[33633]: pgmap v1458: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:52:38.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:52:38 smithi097 ceph-mon[39416]: pgmap v1459: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:52:38.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:52:38 smithi097 ceph-mon[30040]: pgmap v1459: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:52:38.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:52:38 smithi144 ceph-mon[33633]: pgmap v1459: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:52:40.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:52:40 smithi097 ceph-mon[39416]: pgmap v1460: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:52:40.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:52:40 smithi097 ceph-mon[30040]: pgmap v1460: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:52:40.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:52:40 smithi144 ceph-mon[33633]: pgmap v1460: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:52:42.247 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:52:41 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:52:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:52:42.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:52:42 smithi097 ceph-mon[39416]: pgmap v1461: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:52:42.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:52:42 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:52:42.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:52:42 smithi097 ceph-mon[30040]: pgmap v1461: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:52:42.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:52:42 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:52:42.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:52:42 smithi144 ceph-mon[33633]: pgmap v1461: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:52:42.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:52:42 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:52:44.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:52:44 smithi097 ceph-mon[39416]: pgmap v1462: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:52:44.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:52:44 smithi097 ceph-mon[30040]: pgmap v1462: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:52:44.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:52:44 smithi144 ceph-mon[33633]: pgmap v1462: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:52:45.121 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:52:44 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:52:46.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:52:46 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:52:46.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:52:46 smithi097 ceph-mon[39416]: pgmap v1463: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:52:46.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:52:46 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:52:46.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:52:46 smithi097 ceph-mon[30040]: pgmap v1463: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:52:46.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:52:46 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:52:46.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:52:46 smithi144 ceph-mon[33633]: pgmap v1463: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:52:48.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:52:48 smithi097 ceph-mon[30040]: pgmap v1464: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:52:48.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:52:48 smithi097 ceph-mon[39416]: pgmap v1464: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:52:48.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:52:48 smithi144 ceph-mon[33633]: pgmap v1464: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:52:50.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:52:50 smithi097 ceph-mon[39416]: pgmap v1465: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:52:50.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:52:50 smithi097 ceph-mon[30040]: pgmap v1465: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:52:50.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:52:50 smithi144 ceph-mon[33633]: pgmap v1465: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:52:52.261 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:52:51 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:52:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:52:52.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:52:52 smithi097 ceph-mon[30040]: pgmap v1466: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:52:52.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:52:52 smithi097 ceph-mon[39416]: pgmap v1466: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:52:52.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:52:52 smithi144 ceph-mon[33633]: pgmap v1466: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:52:54.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:52:54 smithi097 ceph-mon[30040]: pgmap v1467: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:52:54.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:52:54 smithi097 ceph-mon[39416]: pgmap v1467: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:52:54.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:52:54 smithi144 ceph-mon[33633]: pgmap v1467: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:52:55.121 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:52:54 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:52:56.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:52:56 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:52:56.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:52:56 smithi097 ceph-mon[39416]: pgmap v1468: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:52:56.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:52:56 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:52:56.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:52:56 smithi097 ceph-mon[30040]: pgmap v1468: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:52:56.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:52:56 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:52:56.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:52:56 smithi144 ceph-mon[33633]: pgmap v1468: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:52:57.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:52:57 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:52:57.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:52:57 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:52:57.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:52:57 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:52:58.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:52:58 smithi097 ceph-mon[39416]: pgmap v1469: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:52:58.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:52:58 smithi097 ceph-mon[30040]: pgmap v1469: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:52:58.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:52:58 smithi144 ceph-mon[33633]: pgmap v1469: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:52:59.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:52:59 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:52:59.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:52:59 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:52:59.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:52:59 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:52:59.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:52:59 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:52:59.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:52:59 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:52:59.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:52:59 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:52:59.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:52:59 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:52:59.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:52:59 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:52:59.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:52:59 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:53:00.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:53:00 smithi097 ceph-mon[30040]: pgmap v1470: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:53:00.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:53:00 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:53:00.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:53:00 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:53:00.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:53:00 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:53:00.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:53:00 smithi097 ceph-mon[39416]: pgmap v1470: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:53:00.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:53:00 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:53:00.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:53:00 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:53:00.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:53:00 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:53:00.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:53:00 smithi144 ceph-mon[33633]: pgmap v1470: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:53:00.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:53:00 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:53:00.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:53:00 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:53:00.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:53:00 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:53:02.284 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:53:01 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:53:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:53:02.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:53:02 smithi097 ceph-mon[39416]: pgmap v1471: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:53:02.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:53:02 smithi097 ceph-mon[30040]: pgmap v1471: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:53:02.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:53:02 smithi144 ceph-mon[33633]: pgmap v1471: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:53:04.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:53:04 smithi097 ceph-mon[39416]: pgmap v1472: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:53:04.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:53:04 smithi097 ceph-mon[30040]: pgmap v1472: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:53:04.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:53:04 smithi144 ceph-mon[33633]: pgmap v1472: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:53:05.121 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:53:04 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:53:06.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:53:06 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:53:06.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:53:06 smithi097 ceph-mon[39416]: pgmap v1473: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:53:06.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:53:06 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:53:06.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:53:06 smithi097 ceph-mon[30040]: pgmap v1473: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:53:06.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:53:06 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:53:06.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:53:06 smithi144 ceph-mon[33633]: pgmap v1473: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:53:08.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:53:08 smithi097 ceph-mon[39416]: pgmap v1474: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:53:08.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:53:08 smithi097 ceph-mon[30040]: pgmap v1474: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:53:08.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:53:08 smithi144 ceph-mon[33633]: pgmap v1474: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:53:10.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:53:10 smithi097 ceph-mon[39416]: pgmap v1475: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:53:10.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:53:10 smithi097 ceph-mon[30040]: pgmap v1475: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:53:10.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:53:10 smithi144 ceph-mon[33633]: pgmap v1475: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:53:12.299 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:53:11 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:53:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:53:12.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:53:12 smithi097 ceph-mon[30040]: pgmap v1476: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:53:12.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:53:12 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:53:12.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:53:12 smithi097 ceph-mon[39416]: pgmap v1476: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:53:12.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:53:12 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:53:12.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:53:12 smithi144 ceph-mon[33633]: pgmap v1476: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:53:12.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:53:12 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:53:14.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:53:14 smithi097 ceph-mon[30040]: pgmap v1477: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:53:14.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:53:14 smithi097 ceph-mon[39416]: pgmap v1477: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:53:14.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:53:14 smithi144 ceph-mon[33633]: pgmap v1477: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:53:15.121 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:53:14 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:53:16.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:53:16 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:53:16.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:53:16 smithi144 ceph-mon[33633]: pgmap v1478: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:53:16.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:53:16 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:53:16.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:53:16 smithi097 ceph-mon[39416]: pgmap v1478: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:53:16.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:53:16 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:53:16.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:53:16 smithi097 ceph-mon[30040]: pgmap v1478: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:53:18.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:53:18 smithi144 ceph-mon[33633]: pgmap v1479: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:53:18.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:53:18 smithi097 ceph-mon[39416]: pgmap v1479: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:53:18.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:53:18 smithi097 ceph-mon[30040]: pgmap v1479: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:53:19.054 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.054 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.054 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.054 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.054 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.054 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.054 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.055 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.055 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.055 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.055 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.055 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.055 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.055 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.055 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.055 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.055 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.055 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.055 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.055 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.055 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.055 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.056 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.056 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.056 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.056 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.056 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.056 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.056 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.056 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.056 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.056 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.056 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.056 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.056 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.056 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.056 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.056 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.057 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.057 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.057 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.057 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.057 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.057 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.057 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.057 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.057 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.057 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.057 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.057 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.057 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.057 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.057 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.058 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.058 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.058 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.058 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.058 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.058 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.058 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.058 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.058 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.058 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.058 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.058 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.058 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.059 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.059 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.059 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.059 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.059 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.059 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.059 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.059 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.059 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.059 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.059 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.059 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.059 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.059 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.059 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.059 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.060 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.060 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.060 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.060 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.060 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.060 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.060 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.060 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.060 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.060 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.060 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.060 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.060 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.060 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.060 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.060 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.061 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:19.061 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:53:20.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:53:20 smithi144 ceph-mon[33633]: pgmap v1480: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:53:20.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:53:20 smithi097 ceph-mon[39416]: pgmap v1480: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:53:20.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:53:20 smithi097 ceph-mon[30040]: pgmap v1480: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:53:22.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:53:21 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:53:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:53:22.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:53:22 smithi144 ceph-mon[33633]: pgmap v1481: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:53:22.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:53:22 smithi097 ceph-mon[30040]: pgmap v1481: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:53:22.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:53:22 smithi097 ceph-mon[39416]: pgmap v1481: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:53:24.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:53:24 smithi144 ceph-mon[33633]: pgmap v1482: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:53:24.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:53:24 smithi097 ceph-mon[39416]: pgmap v1482: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:53:24.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:53:24 smithi097 ceph-mon[30040]: pgmap v1482: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:53:25.121 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:53:24 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:53:26.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:53:26 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:53:26.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:53:26 smithi144 ceph-mon[33633]: pgmap v1483: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:53:26.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:53:26 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:53:26.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:53:26 smithi097 ceph-mon[39416]: pgmap v1483: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:53:26.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:53:26 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:53:26.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:53:26 smithi097 ceph-mon[30040]: pgmap v1483: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:53:27.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:53:27 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:53:27.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:53:27 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:53:27.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:53:27 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:53:28.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:53:28 smithi144 ceph-mon[33633]: pgmap v1484: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:53:28.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:53:28 smithi097 ceph-mon[39416]: pgmap v1484: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:53:28.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:53:28 smithi097 ceph-mon[30040]: pgmap v1484: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:53:30.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:53:30 smithi144 ceph-mon[33633]: pgmap v1485: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:53:30.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:53:30 smithi097 ceph-mon[39416]: pgmap v1485: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:53:30.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:53:30 smithi097 ceph-mon[30040]: pgmap v1485: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:53:32.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:53:31 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:53:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:53:32.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:53:32 smithi144 ceph-mon[33633]: pgmap v1486: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:53:32.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:53:32 smithi097 ceph-mon[39416]: pgmap v1486: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:53:32.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:53:32 smithi097 ceph-mon[30040]: pgmap v1486: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:53:34.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:53:34 smithi144 ceph-mon[33633]: pgmap v1487: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:53:34.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:53:34 smithi097 ceph-mon[39416]: pgmap v1487: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:53:34.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:53:34 smithi097 ceph-mon[30040]: pgmap v1487: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:53:35.121 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:53:34 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:53:36.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:53:36 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:53:36.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:53:36 smithi144 ceph-mon[33633]: pgmap v1488: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:53:36.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:53:36 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:53:36.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:53:36 smithi097 ceph-mon[39416]: pgmap v1488: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:53:36.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:53:36 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:53:36.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:53:36 smithi097 ceph-mon[30040]: pgmap v1488: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:53:38.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:53:38 smithi144 ceph-mon[33633]: pgmap v1489: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:53:38.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:53:38 smithi097 ceph-mon[30040]: pgmap v1489: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:53:38.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:53:38 smithi097 ceph-mon[39416]: pgmap v1489: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:53:40.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:53:40 smithi144 ceph-mon[33633]: pgmap v1490: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:53:40.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:53:40 smithi097 ceph-mon[30040]: pgmap v1490: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:53:40.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:53:40 smithi097 ceph-mon[39416]: pgmap v1490: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:53:42.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:53:41 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:53:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:53:42.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:53:42 smithi144 ceph-mon[33633]: pgmap v1491: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:53:42.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:53:42 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:53:42.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:53:42 smithi097 ceph-mon[39416]: pgmap v1491: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:53:42.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:53:42 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:53:42.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:53:42 smithi097 ceph-mon[30040]: pgmap v1491: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:53:42.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:53:42 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:53:44.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:53:44 smithi144 ceph-mon[33633]: pgmap v1492: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:53:44.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:53:44 smithi097 ceph-mon[39416]: pgmap v1492: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:53:44.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:53:44 smithi097 ceph-mon[30040]: pgmap v1492: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:53:45.121 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:53:44 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:53:46.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:53:46 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:53:46.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:53:46 smithi144 ceph-mon[33633]: pgmap v1493: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:53:46.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:53:46 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:53:46.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:53:46 smithi097 ceph-mon[39416]: pgmap v1493: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:53:46.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:53:46 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:53:46.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:53:46 smithi097 ceph-mon[30040]: pgmap v1493: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:53:48.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:53:48 smithi144 ceph-mon[33633]: pgmap v1494: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:53:48.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:53:48 smithi097 ceph-mon[39416]: pgmap v1494: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:53:48.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:53:48 smithi097 ceph-mon[30040]: pgmap v1494: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:53:50.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:53:50 smithi144 ceph-mon[33633]: pgmap v1495: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:53:50.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:53:50 smithi097 ceph-mon[39416]: pgmap v1495: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:53:50.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:53:50 smithi097 ceph-mon[30040]: pgmap v1495: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:53:52.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:53:51 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:53:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:53:52.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:53:52 smithi144 ceph-mon[33633]: pgmap v1496: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:53:52.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:53:52 smithi097 ceph-mon[39416]: pgmap v1496: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:53:52.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:53:52 smithi097 ceph-mon[30040]: pgmap v1496: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:53:54.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:53:54 smithi144 ceph-mon[33633]: pgmap v1497: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:53:54.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:53:54 smithi097 ceph-mon[39416]: pgmap v1497: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:53:54.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:53:54 smithi097 ceph-mon[30040]: pgmap v1497: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:53:55.121 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:53:54 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:53:56.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:53:56 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:53:56.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:53:56 smithi144 ceph-mon[33633]: pgmap v1498: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:53:56.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:53:56 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:53:56.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:53:56 smithi097 ceph-mon[39416]: pgmap v1498: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:53:56.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:53:56 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:53:56.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:53:56 smithi097 ceph-mon[30040]: pgmap v1498: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:53:57.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:53:57 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:53:57.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:53:57 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:53:57.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:53:57 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:53:58.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:53:58 smithi097 ceph-mon[39416]: pgmap v1499: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:53:58.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:53:58 smithi097 ceph-mon[30040]: pgmap v1499: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:53:58.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:53:58 smithi144 ceph-mon[33633]: pgmap v1499: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:54:00.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:54:00 smithi097 ceph-mon[30040]: pgmap v1500: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:54:00.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:54:00 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:54:00.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:54:00 smithi097 ceph-mon[39416]: pgmap v1500: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:54:00.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:54:00 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:54:00.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:54:00 smithi144 ceph-mon[33633]: pgmap v1500: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:54:00.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:54:00 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:54:01.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:54:01 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:54:01.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:54:01 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:54:01.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:54:01 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:54:01.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:54:01 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:54:01.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:54:01 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:54:01.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:54:01 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:54:01.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:54:01 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:54:01.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:54:01 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:54:01.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:54:01 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:54:02.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:54:01 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:54:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:54:02.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:54:02 smithi097 ceph-mon[39416]: pgmap v1501: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:54:02.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:54:02 smithi097 ceph-mon[30040]: pgmap v1501: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:54:02.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:54:02 smithi144 ceph-mon[33633]: pgmap v1501: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:54:04.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:54:04 smithi097 ceph-mon[39416]: pgmap v1502: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:54:04.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:54:04 smithi097 ceph-mon[30040]: pgmap v1502: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:54:04.808 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:54:04 smithi144 ceph-mon[33633]: pgmap v1502: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:54:05.121 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:54:04 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:54:06.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:54:06 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:54:06.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:54:06 smithi097 ceph-mon[39416]: pgmap v1503: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:54:06.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:54:06 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:54:06.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:54:06 smithi097 ceph-mon[30040]: pgmap v1503: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:54:06.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:54:06 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:54:06.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:54:06 smithi144 ceph-mon[33633]: pgmap v1503: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:54:08.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:54:08 smithi097 ceph-mon[39416]: pgmap v1504: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:54:08.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:54:08 smithi097 ceph-mon[30040]: pgmap v1504: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:54:08.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:54:08 smithi144 ceph-mon[33633]: pgmap v1504: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:54:10.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:54:10 smithi097 ceph-mon[39416]: pgmap v1505: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:54:10.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:54:10 smithi097 ceph-mon[30040]: pgmap v1505: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:54:10.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:54:10 smithi144 ceph-mon[33633]: pgmap v1505: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:54:12.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:54:11 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:54:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:54:12.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:54:12 smithi097 ceph-mon[39416]: pgmap v1506: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:54:12.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:54:12 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:54:12.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:54:12 smithi097 ceph-mon[30040]: pgmap v1506: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:54:12.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:54:12 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:54:12.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:54:12 smithi144 ceph-mon[33633]: pgmap v1506: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:54:12.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:54:12 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:54:14.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:54:14 smithi097 ceph-mon[39416]: pgmap v1507: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:54:14.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:54:14 smithi097 ceph-mon[30040]: pgmap v1507: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:54:14.819 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:54:14 smithi144 ceph-mon[33633]: pgmap v1507: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:54:15.121 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:54:14 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:54:16.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:54:16 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:54:16.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:54:16 smithi097 ceph-mon[39416]: pgmap v1508: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:54:16.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:54:16 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:54:16.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:54:16 smithi097 ceph-mon[30040]: pgmap v1508: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:54:16.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:54:16 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:54:16.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:54:16 smithi144 ceph-mon[33633]: pgmap v1508: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:54:18.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:54:18 smithi097 ceph-mon[39416]: pgmap v1509: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:54:18.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:54:18 smithi097 ceph-mon[30040]: pgmap v1509: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:54:18.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:54:18 smithi144 ceph-mon[33633]: pgmap v1509: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:54:20.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:54:20 smithi097 ceph-mon[39416]: pgmap v1510: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:54:20.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:54:20 smithi097 ceph-mon[30040]: pgmap v1510: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:54:20.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:54:20 smithi144 ceph-mon[33633]: pgmap v1510: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:54:22.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:54:21 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:54:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:54:22.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:54:22 smithi097 ceph-mon[30040]: pgmap v1511: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:54:22.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:54:22 smithi097 ceph-mon[39416]: pgmap v1511: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:54:22.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:54:22 smithi144 ceph-mon[33633]: pgmap v1511: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:54:24.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:54:24 smithi097 ceph-mon[39416]: pgmap v1512: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:54:24.806 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:54:24 smithi097 ceph-mon[30040]: pgmap v1512: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:54:24.824 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:54:24 smithi144 ceph-mon[33633]: pgmap v1512: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:54:25.121 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:54:24 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:54:26.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:54:26 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:54:26.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:54:26 smithi097 ceph-mon[39416]: pgmap v1513: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:54:26.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:54:26 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:54:26.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:54:26 smithi097 ceph-mon[30040]: pgmap v1513: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:54:26.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:54:26 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:54:26.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:54:26 smithi144 ceph-mon[33633]: pgmap v1513: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:54:27.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:54:27 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:54:27.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:54:27 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:54:27.874 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:54:27 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:54:28.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:54:28 smithi097 ceph-mon[39416]: pgmap v1514: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:54:28.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:54:28 smithi097 ceph-mon[30040]: pgmap v1514: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:54:28.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:54:28 smithi144 ceph-mon[33633]: pgmap v1514: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:54:30.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:54:30 smithi097 ceph-mon[39416]: pgmap v1515: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:54:30.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:54:30 smithi097 ceph-mon[30040]: pgmap v1515: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:54:30.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:54:30 smithi144 ceph-mon[33633]: pgmap v1515: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:54:32.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:54:31 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:54:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:54:32.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:54:32 smithi097 ceph-mon[39416]: pgmap v1516: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:54:32.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:54:32 smithi097 ceph-mon[30040]: pgmap v1516: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:54:32.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:54:32 smithi144 ceph-mon[33633]: pgmap v1516: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:54:34.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:54:34 smithi097 ceph-mon[39416]: pgmap v1517: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:54:34.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:54:34 smithi097 ceph-mon[30040]: pgmap v1517: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:54:34.833 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:54:34 smithi144 ceph-mon[33633]: pgmap v1517: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:54:35.121 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:54:34 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:54:36.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:54:36 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:54:36.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:54:36 smithi097 ceph-mon[30040]: pgmap v1518: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:54:36.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:54:36 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:54:36.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:54:36 smithi097 ceph-mon[39416]: pgmap v1518: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:54:36.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:54:36 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:54:36.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:54:36 smithi144 ceph-mon[33633]: pgmap v1518: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:54:38.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:54:38 smithi097 ceph-mon[39416]: pgmap v1519: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:54:38.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:54:38 smithi097 ceph-mon[30040]: pgmap v1519: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:54:38.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:54:38 smithi144 ceph-mon[33633]: pgmap v1519: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:54:40.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:54:40 smithi097 ceph-mon[39416]: pgmap v1520: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:54:40.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:54:40 smithi097 ceph-mon[30040]: pgmap v1520: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:54:40.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:54:40 smithi144 ceph-mon[33633]: pgmap v1520: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:54:42.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:54:41 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:54:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:54:42.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:54:42 smithi097 ceph-mon[39416]: pgmap v1521: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:54:42.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:54:42 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:54:42.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:54:42 smithi097 ceph-mon[30040]: pgmap v1521: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:54:42.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:54:42 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:54:42.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:54:42 smithi144 ceph-mon[33633]: pgmap v1521: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:54:42.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:54:42 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:54:43.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:54:43 smithi097 ceph-mon[39416]: pgmap v1522: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:54:43.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:54:43 smithi097 ceph-mon[30040]: pgmap v1522: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:54:43.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:54:43 smithi144 ceph-mon[33633]: pgmap v1522: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:54:45.121 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:54:44 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:54:46.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:54:46 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:54:46.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:54:46 smithi097 ceph-mon[39416]: pgmap v1523: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:54:46.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:54:46 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:54:46.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:54:46 smithi097 ceph-mon[30040]: pgmap v1523: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:54:46.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:54:46 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:54:46.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:54:46 smithi144 ceph-mon[33633]: pgmap v1523: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:54:48.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:54:48 smithi097 ceph-mon[39416]: pgmap v1524: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:54:48.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:54:48 smithi097 ceph-mon[30040]: pgmap v1524: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:54:48.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:54:48 smithi144 ceph-mon[33633]: pgmap v1524: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:54:50.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:54:50 smithi097 ceph-mon[39416]: pgmap v1525: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:54:50.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:54:50 smithi097 ceph-mon[30040]: pgmap v1525: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:54:50.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:54:50 smithi144 ceph-mon[33633]: pgmap v1525: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:54:52.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:54:52 smithi097 ceph-mon[39416]: pgmap v1526: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:54:52.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:54:52 smithi097 ceph-mon[30040]: pgmap v1526: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:54:52.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:54:51 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:54:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:54:52.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:54:52 smithi144 ceph-mon[33633]: pgmap v1526: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:54:54.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:54:54 smithi097 ceph-mon[39416]: pgmap v1527: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:54:54.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:54:54 smithi097 ceph-mon[30040]: pgmap v1527: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:54:54.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:54:54 smithi144 ceph-mon[33633]: pgmap v1527: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:54:55.121 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:54:54 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:54:56.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:54:56 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:54:56.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:54:56 smithi097 ceph-mon[39416]: pgmap v1528: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:54:56.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:54:56 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:54:56.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:54:56 smithi097 ceph-mon[30040]: pgmap v1528: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:54:56.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:54:56 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:54:56.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:54:56 smithi144 ceph-mon[33633]: pgmap v1528: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:54:57.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:54:57 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:54:57.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:54:57 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:54:57.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:54:57 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:54:58.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:54:58 smithi097 ceph-mon[39416]: pgmap v1529: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:54:58.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:54:58 smithi097 ceph-mon[30040]: pgmap v1529: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:54:58.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:54:58 smithi144 ceph-mon[33633]: pgmap v1529: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:54:59.065 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.066 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.066 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.066 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.066 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.066 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.066 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.066 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.066 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.066 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.066 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.066 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.066 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.066 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.067 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.067 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.067 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.067 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.067 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.067 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.067 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.067 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.067 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.067 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.067 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.067 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.067 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.067 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.068 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.068 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.068 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.068 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.068 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.068 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.068 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.068 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.068 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.068 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.068 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.068 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.068 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.068 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.068 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.069 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.069 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.069 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.069 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.069 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.069 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.069 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.069 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.069 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.069 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.069 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.069 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.069 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.069 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.069 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.070 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.070 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.070 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.070 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.070 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.070 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.070 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.070 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.070 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.070 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.071 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.071 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.071 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.071 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.071 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.071 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.071 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.071 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.071 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.071 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.071 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.071 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.071 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.071 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.071 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.072 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.072 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.072 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.072 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.072 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.072 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.072 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.072 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.072 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.072 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.072 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.072 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.072 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.072 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.072 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.072 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:54:59.072 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:55:00.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:55:00 smithi097 ceph-mon[39416]: pgmap v1530: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:55:00.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:55:00 smithi097 ceph-mon[30040]: pgmap v1530: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:55:00.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:55:00 smithi144 ceph-mon[33633]: pgmap v1530: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:55:01.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:55:01 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:55:01.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:55:01 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:55:01.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:55:01 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:55:02.043 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:55:01 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:55:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:55:02.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:55:02 smithi097 ceph-mon[30040]: pgmap v1531: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:55:02.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:55:02 smithi097 ceph-mon[39416]: pgmap v1531: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:55:02.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:55:02 smithi144 ceph-mon[33633]: pgmap v1531: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:55:04.291 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:55:04 smithi097 ceph-mon[39416]: pgmap v1532: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:55:04.292 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:55:04 smithi097 ceph-mon[30040]: pgmap v1532: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:55:04.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:55:04 smithi144 ceph-mon[33633]: pgmap v1532: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:55:05.121 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:55:04 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:55:06.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:55:05 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:55:06.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:55:05 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:55:06.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:55:05 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:55:06.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:55:05 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:55:06.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:55:05 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:55:06.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:55:05 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:55:06.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:55:05 smithi097 ceph-mon[30040]: pgmap v1533: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:55:06.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:55:05 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:55:06.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:55:05 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:55:06.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:55:05 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:55:06.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:55:05 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:55:06.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:55:05 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:55:06.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:55:05 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:55:06.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:55:05 smithi097 ceph-mon[39416]: pgmap v1533: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:55:06.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:55:05 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:55:06.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:55:05 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:55:06.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:55:05 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:55:06.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:55:05 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:55:06.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:55:05 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:55:06.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:55:05 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:55:06.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:55:05 smithi144 ceph-mon[33633]: pgmap v1533: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:55:08.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:55:08 smithi097 ceph-mon[30040]: pgmap v1534: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:55:08.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:55:08 smithi097 ceph-mon[39416]: pgmap v1534: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:55:08.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:55:08 smithi144 ceph-mon[33633]: pgmap v1534: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:55:10.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:55:10 smithi097 ceph-mon[30040]: pgmap v1535: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:55:10.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:55:10 smithi097 ceph-mon[39416]: pgmap v1535: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:55:10.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:55:10 smithi144 ceph-mon[33633]: pgmap v1535: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:55:12.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:55:12 smithi097 ceph-mon[39416]: pgmap v1536: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:55:12.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:55:12 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:55:12.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:55:12 smithi097 ceph-mon[30040]: pgmap v1536: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:55:12.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:55:12 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:55:12.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:55:11 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:55:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:55:12.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:55:12 smithi144 ceph-mon[33633]: pgmap v1536: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:55:12.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:55:12 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:55:14.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:55:14 smithi097 ceph-mon[30040]: pgmap v1537: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:55:14.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:55:14 smithi097 ceph-mon[39416]: pgmap v1537: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:55:14.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:55:14 smithi144 ceph-mon[33633]: pgmap v1537: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:55:15.121 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:55:14 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:55:16.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:55:16 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:55:16.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:55:16 smithi097 ceph-mon[30040]: pgmap v1538: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:55:16.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:55:16 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:55:16.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:55:16 smithi097 ceph-mon[39416]: pgmap v1538: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:55:16.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:55:16 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:55:16.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:55:16 smithi144 ceph-mon[33633]: pgmap v1538: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:55:18.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:55:18 smithi097 ceph-mon[30040]: pgmap v1539: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:55:18.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:55:18 smithi097 ceph-mon[39416]: pgmap v1539: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:55:18.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:55:18 smithi144 ceph-mon[33633]: pgmap v1539: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:55:20.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:55:20 smithi097 ceph-mon[30040]: pgmap v1540: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:55:20.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:55:20 smithi097 ceph-mon[39416]: pgmap v1540: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:55:20.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:55:20 smithi144 ceph-mon[33633]: pgmap v1540: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:55:22.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:55:22 smithi097 ceph-mon[39416]: pgmap v1541: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:55:22.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:55:22 smithi097 ceph-mon[30040]: pgmap v1541: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:55:22.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:55:21 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:55:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:55:22.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:55:22 smithi144 ceph-mon[33633]: pgmap v1541: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:55:24.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:55:24 smithi097 ceph-mon[39416]: pgmap v1542: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:55:24.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:55:24 smithi097 ceph-mon[30040]: pgmap v1542: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:55:24.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:55:24 smithi144 ceph-mon[33633]: pgmap v1542: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:55:25.371 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:55:24 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:55:26.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:55:26 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:55:26.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:55:26 smithi097 ceph-mon[39416]: pgmap v1543: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:55:26.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:55:26 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:55:26.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:55:26 smithi097 ceph-mon[30040]: pgmap v1543: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:55:26.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:55:26 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:55:26.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:55:26 smithi144 ceph-mon[33633]: pgmap v1543: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:55:27.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:55:27 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:55:27.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:55:27 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:55:27.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:55:27 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:55:28.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:55:28 smithi144 ceph-mon[33633]: pgmap v1544: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:55:28.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:55:28 smithi097 ceph-mon[30040]: pgmap v1544: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:55:28.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:55:28 smithi097 ceph-mon[39416]: pgmap v1544: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:55:30.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:55:30 smithi144 ceph-mon[33633]: pgmap v1545: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:55:30.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:55:30 smithi097 ceph-mon[39416]: pgmap v1545: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:55:30.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:55:30 smithi097 ceph-mon[30040]: pgmap v1545: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:55:32.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:55:32 smithi097 ceph-mon[39416]: pgmap v1546: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:55:32.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:55:31 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:55:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:55:32.305 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:55:32 smithi097 ceph-mon[30040]: pgmap v1546: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:55:32.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:55:32 smithi144 ceph-mon[33633]: pgmap v1546: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:55:34.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:55:34 smithi144 ceph-mon[33633]: pgmap v1547: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:55:34.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:55:34 smithi097 ceph-mon[39416]: pgmap v1547: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:55:34.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:55:34 smithi097 ceph-mon[30040]: pgmap v1547: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:55:35.371 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:55:34 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:55:36.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:55:36 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:55:36.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:55:36 smithi144 ceph-mon[33633]: pgmap v1548: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:55:36.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:55:36 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:55:36.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:55:36 smithi097 ceph-mon[30040]: pgmap v1548: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:55:36.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:55:36 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:55:36.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:55:36 smithi097 ceph-mon[39416]: pgmap v1548: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:55:38.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:55:38 smithi144 ceph-mon[33633]: pgmap v1549: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:55:38.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:55:38 smithi097 ceph-mon[39416]: pgmap v1549: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:55:38.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:55:38 smithi097 ceph-mon[30040]: pgmap v1549: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:55:40.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:55:40 smithi144 ceph-mon[33633]: pgmap v1550: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:55:40.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:55:40 smithi097 ceph-mon[39416]: pgmap v1550: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:55:40.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:55:40 smithi097 ceph-mon[30040]: pgmap v1550: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:55:42.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:55:41 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:55:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:55:42.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:55:42 smithi097 ceph-mon[39416]: pgmap v1551: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:55:42.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:55:42 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:55:42.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:55:42 smithi097 ceph-mon[30040]: pgmap v1551: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:55:42.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:55:42 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:55:42.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:55:42 smithi144 ceph-mon[33633]: pgmap v1551: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:55:42.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:55:42 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:55:44.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:55:43 smithi097 ceph-mon[39416]: pgmap v1552: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:55:44.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:55:43 smithi097 ceph-mon[30040]: pgmap v1552: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:55:44.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:55:43 smithi144 ceph-mon[33633]: pgmap v1552: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:55:45.371 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:55:44 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:55:46.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:55:46 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:55:46.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:55:46 smithi097 ceph-mon[39416]: pgmap v1553: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:55:46.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:55:46 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:55:46.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:55:46 smithi097 ceph-mon[30040]: pgmap v1553: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:55:46.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:55:46 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:55:46.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:55:46 smithi144 ceph-mon[33633]: pgmap v1553: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:55:48.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:55:48 smithi097 ceph-mon[39416]: pgmap v1554: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:55:48.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:55:48 smithi097 ceph-mon[30040]: pgmap v1554: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:55:48.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:55:48 smithi144 ceph-mon[33633]: pgmap v1554: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:55:50.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:55:50 smithi097 ceph-mon[39416]: pgmap v1555: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:55:50.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:55:50 smithi097 ceph-mon[30040]: pgmap v1555: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:55:50.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:55:50 smithi144 ceph-mon[33633]: pgmap v1555: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:55:52.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:55:52 smithi097 ceph-mon[39416]: pgmap v1556: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:55:52.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:55:52 smithi097 ceph-mon[30040]: pgmap v1556: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:55:52.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:55:51 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:55:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:55:52.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:55:52 smithi144 ceph-mon[33633]: pgmap v1556: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:55:54.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:55:54 smithi097 ceph-mon[39416]: pgmap v1557: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:55:54.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:55:54 smithi097 ceph-mon[30040]: pgmap v1557: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:55:54.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:55:54 smithi144 ceph-mon[33633]: pgmap v1557: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:55:55.371 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:55:54 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:55:56.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:55:56 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:55:56.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:55:56 smithi097 ceph-mon[39416]: pgmap v1558: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:55:56.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:55:56 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:55:56.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:55:56 smithi097 ceph-mon[30040]: pgmap v1558: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:55:56.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:55:56 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:55:56.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:55:56 smithi144 ceph-mon[33633]: pgmap v1558: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:55:57.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:55:57 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:55:57.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:55:57 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:55:57.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:55:57 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:55:58.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:55:58 smithi144 ceph-mon[33633]: pgmap v1559: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:55:58.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:55:58 smithi097 ceph-mon[39416]: pgmap v1559: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:55:58.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:55:58 smithi097 ceph-mon[30040]: pgmap v1559: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:56:00.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:56:00 smithi144 ceph-mon[33633]: pgmap v1560: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:56:00.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:56:00 smithi097 ceph-mon[39416]: pgmap v1560: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:56:00.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:56:00 smithi097 ceph-mon[30040]: pgmap v1560: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:56:02.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:56:02 smithi097 ceph-mon[39416]: pgmap v1561: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:56:02.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:56:02 smithi097 ceph-mon[30040]: pgmap v1561: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:56:02.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:56:01 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:56:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:56:02.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:56:02 smithi144 ceph-mon[33633]: pgmap v1561: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:56:04.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:56:04 smithi144 ceph-mon[33633]: pgmap v1562: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:56:04.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:56:04 smithi097 ceph-mon[39416]: pgmap v1562: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:56:04.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:56:04 smithi097 ceph-mon[30040]: pgmap v1562: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:56:05.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:56:05 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:56:05.371 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:56:04 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:56:05.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:56:05 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:56:05.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:56:05 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:56:06.341 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:56:06 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:56:06.341 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:56:06 smithi144 ceph-mon[33633]: pgmap v1563: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:56:06.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:56:06 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:56:06.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:56:06 smithi097 ceph-mon[39416]: pgmap v1563: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:56:06.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:56:06 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:56:06.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:56:06 smithi097 ceph-mon[30040]: pgmap v1563: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:56:08.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:56:08 smithi144 ceph-mon[33633]: pgmap v1564: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:56:08.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:56:08 smithi097 ceph-mon[30040]: pgmap v1564: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:56:08.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:56:08 smithi097 ceph-mon[39416]: pgmap v1564: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:56:10.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:56:09 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:56:10.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:56:09 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:56:10.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:56:09 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:56:10.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:56:09 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:56:10.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:56:09 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:56:10.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:56:09 smithi144 ceph-mon[33633]: pgmap v1565: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:56:10.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:56:09 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:56:10.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:56:09 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:56:10.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:56:09 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:56:10.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:56:09 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:56:10.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:56:09 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:56:10.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:56:09 smithi097 ceph-mon[39416]: pgmap v1565: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:56:10.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:56:09 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:56:10.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:56:09 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:56:10.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:56:09 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:56:10.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:56:09 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:56:10.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:56:09 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:56:10.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:56:09 smithi097 ceph-mon[30040]: pgmap v1565: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:56:12.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:56:11 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:56:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:56:12.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:56:12 smithi097 ceph-mon[39416]: pgmap v1566: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:56:12.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:56:12 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:56:12.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:56:12 smithi097 ceph-mon[30040]: pgmap v1566: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:56:12.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:56:12 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:56:12.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:56:12 smithi144 ceph-mon[33633]: pgmap v1566: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:56:12.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:56:12 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:56:14.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:56:14 smithi097 ceph-mon[39416]: pgmap v1567: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:56:14.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:56:14 smithi097 ceph-mon[30040]: pgmap v1567: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:56:14.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:56:14 smithi144 ceph-mon[33633]: pgmap v1567: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:56:15.371 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:56:14 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:56:16.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:56:16 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:56:16.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:56:16 smithi097 ceph-mon[39416]: pgmap v1568: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:56:16.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:56:16 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:56:16.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:56:16 smithi097 ceph-mon[30040]: pgmap v1568: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:56:16.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:56:16 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:56:16.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:56:16 smithi144 ceph-mon[33633]: pgmap v1568: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:56:18.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:56:18 smithi097 ceph-mon[39416]: pgmap v1569: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:56:18.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:56:18 smithi097 ceph-mon[30040]: pgmap v1569: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:56:18.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:56:18 smithi144 ceph-mon[33633]: pgmap v1569: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:56:20.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:56:20 smithi144 ceph-mon[33633]: pgmap v1570: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-21T13:56:20.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:56:20 smithi097 ceph-mon[39416]: pgmap v1570: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-21T13:56:20.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:56:20 smithi097 ceph-mon[30040]: pgmap v1570: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-21T13:56:22.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:56:22 smithi097 ceph-mon[39416]: pgmap v1571: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-21T13:56:22.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:56:22 smithi097 ceph-mon[30040]: pgmap v1571: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-21T13:56:22.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:56:21 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:56:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:56:22.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:56:22 smithi144 ceph-mon[33633]: pgmap v1571: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-21T13:56:24.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:56:24 smithi144 ceph-mon[33633]: pgmap v1572: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-21T13:56:24.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:56:24 smithi097 ceph-mon[39416]: pgmap v1572: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-21T13:56:24.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:56:24 smithi097 ceph-mon[30040]: pgmap v1572: 260 pgs: 260 active+clean; 582 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-21T13:56:25.371 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:56:24 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:56:26.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:56:26 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:56:26.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:56:26 smithi144 ceph-mon[33633]: pgmap v1573: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-21T13:56:26.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:56:26 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:56:26.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:56:26 smithi097 ceph-mon[39416]: pgmap v1573: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-21T13:56:26.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:56:26 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:56:26.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:56:26 smithi097 ceph-mon[30040]: pgmap v1573: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-21T13:56:27.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:56:27 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:56:27.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:56:27 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:56:27.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:56:27 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:56:28.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:56:28 smithi144 ceph-mon[33633]: pgmap v1574: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-21T13:56:28.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:56:28 smithi097 ceph-mon[39416]: pgmap v1574: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-21T13:56:28.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:56:28 smithi097 ceph-mon[30040]: pgmap v1574: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-21T13:56:30.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:56:30 smithi144 ceph-mon[33633]: pgmap v1575: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-21T13:56:30.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:56:30 smithi097 ceph-mon[30040]: pgmap v1575: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-21T13:56:30.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:56:30 smithi097 ceph-mon[39416]: pgmap v1575: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-21T13:56:32.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:56:32 smithi097 ceph-mon[39416]: pgmap v1576: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 53 op/s 2024-03-21T13:56:32.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:56:32 smithi097 ceph-mon[30040]: pgmap v1576: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 53 op/s 2024-03-21T13:56:32.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:56:31 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:56:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:56:32.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:56:32 smithi144 ceph-mon[33633]: pgmap v1576: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 53 op/s 2024-03-21T13:56:34.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:56:34 smithi144 ceph-mon[33633]: pgmap v1577: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 53 op/s 2024-03-21T13:56:34.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:56:34 smithi097 ceph-mon[30040]: pgmap v1577: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 53 op/s 2024-03-21T13:56:34.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:56:34 smithi097 ceph-mon[39416]: pgmap v1577: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 53 op/s 2024-03-21T13:56:35.371 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:56:34 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:56:36.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:56:36 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:56:36.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:56:36 smithi144 ceph-mon[33633]: pgmap v1578: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 54 op/s 2024-03-21T13:56:36.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:56:36 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:56:36.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:56:36 smithi097 ceph-mon[39416]: pgmap v1578: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 54 op/s 2024-03-21T13:56:36.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:56:36 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:56:36.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:56:36 smithi097 ceph-mon[30040]: pgmap v1578: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 54 op/s 2024-03-21T13:56:38.077 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.077 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.077 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.077 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.077 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.077 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.078 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.078 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.078 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.078 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.078 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.078 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.078 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.078 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.078 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.078 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.078 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.079 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.079 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.079 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.079 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.079 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.079 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.079 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.079 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.080 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.080 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.080 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.080 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.080 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.080 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.080 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.080 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.080 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.081 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.081 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.081 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.081 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.081 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.081 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.081 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.081 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.081 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.081 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.081 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.081 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.081 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.081 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.081 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.082 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.082 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.082 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.082 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.082 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.082 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.082 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.082 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.082 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.082 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.082 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.082 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.082 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.082 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.083 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.083 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.083 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.083 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.083 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.083 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.083 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.083 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.083 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.083 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.083 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.083 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.083 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.083 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.084 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.084 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.084 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.084 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.084 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.084 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.084 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.084 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.084 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.085 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.085 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.085 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.085 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.085 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.085 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.085 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.085 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.085 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.085 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.085 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.085 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.085 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:56:38.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:56:38 smithi144 ceph-mon[33633]: pgmap v1579: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:56:38.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:56:38 smithi097 ceph-mon[39416]: pgmap v1579: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:56:38.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:56:38 smithi097 ceph-mon[30040]: pgmap v1579: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:56:40.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:56:40 smithi144 ceph-mon[33633]: pgmap v1580: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:56:40.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:56:40 smithi097 ceph-mon[39416]: pgmap v1580: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:56:40.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:56:40 smithi097 ceph-mon[30040]: pgmap v1580: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:56:42.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:56:41 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:56:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:56:42.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:56:42 smithi097 ceph-mon[39416]: pgmap v1581: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:56:42.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:56:42 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:56:42.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:56:42 smithi097 ceph-mon[30040]: pgmap v1581: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:56:42.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:56:42 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:56:42.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:56:42 smithi144 ceph-mon[33633]: pgmap v1581: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:56:42.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:56:42 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:56:44.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:56:44 smithi144 ceph-mon[33633]: pgmap v1582: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:56:44.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:56:44 smithi097 ceph-mon[39416]: pgmap v1582: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:56:44.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:56:44 smithi097 ceph-mon[30040]: pgmap v1582: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:56:45.371 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:56:44 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:56:46.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:56:46 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:56:46.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:56:46 smithi144 ceph-mon[33633]: pgmap v1583: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:56:46.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:56:46 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:56:46.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:56:46 smithi097 ceph-mon[39416]: pgmap v1583: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:56:46.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:56:46 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:56:46.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:56:46 smithi097 ceph-mon[30040]: pgmap v1583: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:56:48.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:56:48 smithi144 ceph-mon[33633]: pgmap v1584: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:56:48.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:56:48 smithi097 ceph-mon[39416]: pgmap v1584: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:56:48.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:56:48 smithi097 ceph-mon[30040]: pgmap v1584: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:56:50.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:56:50 smithi144 ceph-mon[33633]: pgmap v1585: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:56:50.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:56:50 smithi097 ceph-mon[39416]: pgmap v1585: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:56:50.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:56:50 smithi097 ceph-mon[30040]: pgmap v1585: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:56:52.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:56:52 smithi097 ceph-mon[39416]: pgmap v1586: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:56:52.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:56:52 smithi097 ceph-mon[30040]: pgmap v1586: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:56:52.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:56:51 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:56:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:56:52.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:56:52 smithi144 ceph-mon[33633]: pgmap v1586: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:56:54.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:56:54 smithi144 ceph-mon[33633]: pgmap v1587: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:56:54.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:56:54 smithi097 ceph-mon[39416]: pgmap v1587: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:56:54.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:56:54 smithi097 ceph-mon[30040]: pgmap v1587: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:56:55.371 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:56:54 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:56:56.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:56:56 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:56:56.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:56:56 smithi144 ceph-mon[33633]: pgmap v1588: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:56:56.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:56:56 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:56:56.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:56:56 smithi097 ceph-mon[39416]: pgmap v1588: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:56:56.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:56:56 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:56:56.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:56:56 smithi097 ceph-mon[30040]: pgmap v1588: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:56:57.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:56:57 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:56:57.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:56:57 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:56:57.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:56:57 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:56:58.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:56:58 smithi097 ceph-mon[39416]: pgmap v1589: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:56:58.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:56:58 smithi097 ceph-mon[30040]: pgmap v1589: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:56:58.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:56:58 smithi144 ceph-mon[33633]: pgmap v1589: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:57:00.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:57:00 smithi097 ceph-mon[39416]: pgmap v1590: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:57:00.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:57:00 smithi097 ceph-mon[30040]: pgmap v1590: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:57:00.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:57:00 smithi144 ceph-mon[33633]: pgmap v1590: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:57:02.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:57:02 smithi097 ceph-mon[39416]: pgmap v1591: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:57:02.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:57:02 smithi097 ceph-mon[30040]: pgmap v1591: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:57:02.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:57:01 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:57:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:57:02.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:57:02 smithi144 ceph-mon[33633]: pgmap v1591: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:57:04.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:57:04 smithi097 ceph-mon[39416]: pgmap v1592: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:57:04.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:57:04 smithi097 ceph-mon[30040]: pgmap v1592: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:57:04.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:57:04 smithi144 ceph-mon[33633]: pgmap v1592: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:57:05.371 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:57:04 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:57:06.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:57:06 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:57:06.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:57:06 smithi097 ceph-mon[39416]: pgmap v1593: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:57:06.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:57:06 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:57:06.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:57:06 smithi097 ceph-mon[30040]: pgmap v1593: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:57:06.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:57:06 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:57:06.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:57:06 smithi144 ceph-mon[33633]: pgmap v1593: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:57:08.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:57:08 smithi097 ceph-mon[39416]: pgmap v1594: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:57:08.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:57:08 smithi097 ceph-mon[30040]: pgmap v1594: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:57:08.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:57:08 smithi144 ceph-mon[33633]: pgmap v1594: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:57:09.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:57:09 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:57:09.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:57:09 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:57:09.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:57:09 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:57:10.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:57:10 smithi097 ceph-mon[39416]: pgmap v1595: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:57:10.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:57:10 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:57:10.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:57:10 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:57:10.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:57:10 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:57:10.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:57:10 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:57:10.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:57:10 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:57:10.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:57:10 smithi097 ceph-mon[30040]: pgmap v1595: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:57:10.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:57:10 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:57:10.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:57:10 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:57:10.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:57:10 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:57:10.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:57:10 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:57:10.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:57:10 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:57:10.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:57:10 smithi144 ceph-mon[33633]: pgmap v1595: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:57:10.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:57:10 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:57:10.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:57:10 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:57:10.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:57:10 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:57:10.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:57:10 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:57:10.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:57:10 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:57:12.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:57:12 smithi097 ceph-mon[30040]: pgmap v1596: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:57:12.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:57:12 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:57:12.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:57:12 smithi097 ceph-mon[39416]: pgmap v1596: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:57:12.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:57:12 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:57:12.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:57:11 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:57:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:57:12.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:57:12 smithi144 ceph-mon[33633]: pgmap v1596: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:57:12.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:57:12 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:57:14.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:57:14 smithi097 ceph-mon[39416]: pgmap v1597: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:57:14.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:57:14 smithi097 ceph-mon[30040]: pgmap v1597: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:57:14.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:57:14 smithi144 ceph-mon[33633]: pgmap v1597: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:57:15.371 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:57:14 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:57:16.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:57:16 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:57:16.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:57:16 smithi144 ceph-mon[33633]: pgmap v1598: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:57:16.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:57:16 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:57:16.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:57:16 smithi097 ceph-mon[39416]: pgmap v1598: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:57:16.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:57:16 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:57:16.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:57:16 smithi097 ceph-mon[30040]: pgmap v1598: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:57:18.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:57:18 smithi097 ceph-mon[39416]: pgmap v1599: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:57:18.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:57:18 smithi097 ceph-mon[30040]: pgmap v1599: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:57:18.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:57:18 smithi144 ceph-mon[33633]: pgmap v1599: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:57:20.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:57:20 smithi097 ceph-mon[39416]: pgmap v1600: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:57:20.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:57:20 smithi097 ceph-mon[30040]: pgmap v1600: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:57:20.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:57:20 smithi144 ceph-mon[33633]: pgmap v1600: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:57:22.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:57:22 smithi097 ceph-mon[39416]: pgmap v1601: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:57:22.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:57:22 smithi097 ceph-mon[30040]: pgmap v1601: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:57:22.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:57:21 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:57:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:57:22.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:57:22 smithi144 ceph-mon[33633]: pgmap v1601: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:57:24.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:57:24 smithi097 ceph-mon[39416]: pgmap v1602: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:57:24.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:57:24 smithi097 ceph-mon[30040]: pgmap v1602: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:57:24.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:57:24 smithi144 ceph-mon[33633]: pgmap v1602: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:57:25.371 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:57:24 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:57:26.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:57:26 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:57:26.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:57:26 smithi097 ceph-mon[39416]: pgmap v1603: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:57:26.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:57:26 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:57:26.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:57:26 smithi097 ceph-mon[30040]: pgmap v1603: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:57:26.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:57:26 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:57:26.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:57:26 smithi144 ceph-mon[33633]: pgmap v1603: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:57:27.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:57:27 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:57:27.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:57:27 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:57:27.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:57:27 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:57:28.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:57:28 smithi097 ceph-mon[39416]: pgmap v1604: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:57:28.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:57:28 smithi097 ceph-mon[30040]: pgmap v1604: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:57:28.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:57:28 smithi144 ceph-mon[33633]: pgmap v1604: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:57:30.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:57:30 smithi097 ceph-mon[39416]: pgmap v1605: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:57:30.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:57:30 smithi097 ceph-mon[30040]: pgmap v1605: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:57:30.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:57:30 smithi144 ceph-mon[33633]: pgmap v1605: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:57:32.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:57:32 smithi097 ceph-mon[39416]: pgmap v1606: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:57:32.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:57:32 smithi097 ceph-mon[30040]: pgmap v1606: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:57:32.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:57:31 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:57:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:57:32.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:57:32 smithi144 ceph-mon[33633]: pgmap v1606: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:57:34.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:57:34 smithi097 ceph-mon[39416]: pgmap v1607: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:57:34.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:57:34 smithi097 ceph-mon[30040]: pgmap v1607: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:57:34.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:57:34 smithi144 ceph-mon[33633]: pgmap v1607: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:57:35.371 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:57:34 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:57:36.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:57:36 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:57:36.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:57:36 smithi097 ceph-mon[30040]: pgmap v1608: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:57:36.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:57:36 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:57:36.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:57:36 smithi097 ceph-mon[39416]: pgmap v1608: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:57:36.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:57:36 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:57:36.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:57:36 smithi144 ceph-mon[33633]: pgmap v1608: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:57:38.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:57:38 smithi097 ceph-mon[39416]: pgmap v1609: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:57:38.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:57:38 smithi097 ceph-mon[30040]: pgmap v1609: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:57:38.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:57:38 smithi144 ceph-mon[33633]: pgmap v1609: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:57:40.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:57:40 smithi097 ceph-mon[39416]: pgmap v1610: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:57:40.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:57:40 smithi097 ceph-mon[30040]: pgmap v1610: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:57:40.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:57:40 smithi144 ceph-mon[33633]: pgmap v1610: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:57:42.206 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:57:41 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:57:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:57:42.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:57:42 smithi097 ceph-mon[39416]: pgmap v1611: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:57:42.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:57:42 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:57:42.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:57:42 smithi097 ceph-mon[30040]: pgmap v1611: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:57:42.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:57:42 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:57:42.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:57:42 smithi144 ceph-mon[33633]: pgmap v1611: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:57:42.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:57:42 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:57:44.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:57:44 smithi097 ceph-mon[39416]: pgmap v1612: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:57:44.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:57:44 smithi097 ceph-mon[30040]: pgmap v1612: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:57:44.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:57:44 smithi144 ceph-mon[33633]: pgmap v1612: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:57:45.371 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:57:44 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:57:46.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:57:46 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:57:46.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:57:46 smithi097 ceph-mon[39416]: pgmap v1613: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:57:46.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:57:46 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:57:46.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:57:46 smithi097 ceph-mon[30040]: pgmap v1613: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:57:46.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:57:46 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:57:46.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:57:46 smithi144 ceph-mon[33633]: pgmap v1613: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:57:48.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:57:48 smithi097 ceph-mon[39416]: pgmap v1614: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-21T13:57:48.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:57:48 smithi097 ceph-mon[30040]: pgmap v1614: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-21T13:57:48.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:57:48 smithi144 ceph-mon[33633]: pgmap v1614: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-21T13:57:50.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:57:50 smithi097 ceph-mon[39416]: pgmap v1615: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:57:50.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:57:50 smithi097 ceph-mon[30040]: pgmap v1615: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:57:50.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:57:50 smithi144 ceph-mon[33633]: pgmap v1615: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:57:52.221 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:57:51 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:57:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:57:52.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:57:52 smithi097 ceph-mon[39416]: pgmap v1616: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-21T13:57:52.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:57:52 smithi097 ceph-mon[30040]: pgmap v1616: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-21T13:57:52.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:57:52 smithi144 ceph-mon[33633]: pgmap v1616: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-21T13:57:54.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:57:54 smithi097 ceph-mon[39416]: pgmap v1617: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-21T13:57:54.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:57:54 smithi097 ceph-mon[30040]: pgmap v1617: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-21T13:57:54.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:57:54 smithi144 ceph-mon[33633]: pgmap v1617: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-21T13:57:55.008 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 13:57:54 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=cleanup t=2024-03-21T13:57:54.660121471Z level=info msg="Completed cleanup jobs" duration=3.370438ms 2024-03-21T13:57:55.371 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:57:55 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:57:56.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:57:56 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:57:56.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:57:56 smithi097 ceph-mon[39416]: pgmap v1618: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:57:56.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:57:56 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:57:56.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:57:56 smithi097 ceph-mon[30040]: pgmap v1618: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:57:56.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:57:56 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:57:56.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:57:56 smithi144 ceph-mon[33633]: pgmap v1618: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:57:57.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:57:57 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:57:57.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:57:57 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:57:57.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:57:57 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:57:58.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:57:58 smithi097 ceph-mon[39416]: pgmap v1619: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:57:58.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:57:58 smithi097 ceph-mon[30040]: pgmap v1619: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:57:58.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:57:58 smithi144 ceph-mon[33633]: pgmap v1619: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:58:00.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:58:00 smithi097 ceph-mon[39416]: pgmap v1620: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:58:00.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:58:00 smithi097 ceph-mon[30040]: pgmap v1620: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:58:00.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:58:00 smithi144 ceph-mon[33633]: pgmap v1620: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:58:02.241 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:58:01 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:58:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:58:02.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:58:02 smithi097 ceph-mon[39416]: pgmap v1621: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:58:02.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:58:02 smithi097 ceph-mon[30040]: pgmap v1621: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:58:02.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:58:02 smithi144 ceph-mon[33633]: pgmap v1621: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:58:04.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:58:04 smithi097 ceph-mon[39416]: pgmap v1622: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:58:04.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:58:04 smithi097 ceph-mon[30040]: pgmap v1622: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:58:04.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:58:04 smithi144 ceph-mon[33633]: pgmap v1622: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:58:05.371 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:58:05 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:58:06.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:58:06 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:58:06.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:58:06 smithi097 ceph-mon[39416]: pgmap v1623: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:58:06.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:58:06 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:58:06.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:58:06 smithi097 ceph-mon[30040]: pgmap v1623: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:58:06.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:58:06 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:58:06.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:58:06 smithi144 ceph-mon[33633]: pgmap v1623: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:58:08.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:58:08 smithi097 ceph-mon[39416]: pgmap v1624: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:58:08.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:58:08 smithi097 ceph-mon[30040]: pgmap v1624: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:58:08.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:58:08 smithi144 ceph-mon[33633]: pgmap v1624: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:58:10.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:58:10 smithi097 ceph-mon[39416]: pgmap v1625: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:58:10.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:58:10 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:58:10.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:58:10 smithi097 ceph-mon[30040]: pgmap v1625: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:58:10.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:58:10 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:58:10.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:58:10 smithi144 ceph-mon[33633]: pgmap v1625: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:58:10.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:58:10 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:58:12.272 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:58:11 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:58:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:58:12.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:58:12 smithi097 ceph-mon[39416]: pgmap v1626: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:58:12.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:58:12 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:58:12.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:58:12 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:58:12.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:58:12 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:58:12.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:58:12 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:58:12.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:58:12 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:58:12.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:58:12 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:58:12.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:58:12 smithi097 ceph-mon[30040]: pgmap v1626: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:58:12.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:58:12 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:58:12.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:58:12 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:58:12.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:58:12 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:58:12.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:58:12 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:58:12.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:58:12 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:58:12.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:58:12 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:58:12.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:58:12 smithi144 ceph-mon[33633]: pgmap v1626: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:58:12.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:58:12 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:58:12.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:58:12 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:58:12.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:58:12 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:58:12.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:58:12 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:58:12.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:58:12 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:58:12.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:58:12 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:58:14.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:58:14 smithi097 ceph-mon[39416]: pgmap v1627: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:58:14.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:58:14 smithi097 ceph-mon[30040]: pgmap v1627: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:58:14.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:58:14 smithi144 ceph-mon[33633]: pgmap v1627: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:58:15.371 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:58:15 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:58:16.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:58:16 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:58:16.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:58:16 smithi097 ceph-mon[39416]: pgmap v1628: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:58:16.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:58:16 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:58:16.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:58:16 smithi097 ceph-mon[30040]: pgmap v1628: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:58:16.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:58:16 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:58:16.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:58:16 smithi144 ceph-mon[33633]: pgmap v1628: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:58:18.089 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.089 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.089 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.089 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.089 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.089 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.089 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.089 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.090 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.090 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.090 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.090 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.090 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.090 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.090 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.090 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.090 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.090 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.090 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.090 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.090 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.090 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.091 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.091 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.091 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.091 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.091 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.091 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.091 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.091 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.091 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.091 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.091 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.091 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.091 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.091 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.091 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.092 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.092 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.092 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.092 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.092 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.092 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.092 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.092 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.092 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.092 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.092 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.092 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.092 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.092 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.093 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.093 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.093 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.093 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.093 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.093 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.093 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.093 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.093 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.093 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.093 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.093 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.093 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.093 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.093 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.094 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.094 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.094 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.094 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.094 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.094 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.094 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.094 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.094 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.094 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.094 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.094 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.094 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.095 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.095 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.095 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.095 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.095 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.095 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.095 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.095 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.095 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.095 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.096 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.096 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.096 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.096 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.096 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.096 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.096 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.096 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.096 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.096 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.096 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:58:18.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:58:18 smithi097 ceph-mon[39416]: pgmap v1629: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:58:18.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:58:18 smithi097 ceph-mon[30040]: pgmap v1629: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:58:18.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:58:18 smithi144 ceph-mon[33633]: pgmap v1629: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:58:20.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:58:20 smithi097 ceph-mon[39416]: pgmap v1630: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:58:20.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:58:20 smithi097 ceph-mon[30040]: pgmap v1630: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:58:20.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:58:20 smithi144 ceph-mon[33633]: pgmap v1630: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:58:22.287 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:58:21 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:58:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:58:22.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:58:22 smithi097 ceph-mon[39416]: pgmap v1631: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:58:22.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:58:22 smithi097 ceph-mon[30040]: pgmap v1631: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:58:22.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:58:22 smithi144 ceph-mon[33633]: pgmap v1631: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:58:24.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:58:24 smithi097 ceph-mon[30040]: pgmap v1632: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:58:24.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:58:24 smithi097 ceph-mon[39416]: pgmap v1632: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:58:24.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:58:24 smithi144 ceph-mon[33633]: pgmap v1632: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:58:25.371 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:58:25 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:58:26.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:58:26 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:58:26.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:58:26 smithi097 ceph-mon[30040]: pgmap v1633: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:58:26.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:58:26 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:58:26.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:58:26 smithi097 ceph-mon[39416]: pgmap v1633: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:58:26.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:58:26 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:58:26.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:58:26 smithi144 ceph-mon[33633]: pgmap v1633: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:58:27.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:58:27 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:58:27.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:58:27 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:58:27.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:58:27 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:58:28.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:58:28 smithi097 ceph-mon[39416]: pgmap v1634: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:58:28.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:58:28 smithi097 ceph-mon[30040]: pgmap v1634: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:58:28.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:58:28 smithi144 ceph-mon[33633]: pgmap v1634: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:58:30.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:58:30 smithi097 ceph-mon[39416]: pgmap v1635: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:58:30.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:58:30 smithi097 ceph-mon[30040]: pgmap v1635: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:58:30.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:58:30 smithi144 ceph-mon[33633]: pgmap v1635: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:58:32.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:58:31 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:58:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:58:32.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:58:32 smithi144 ceph-mon[33633]: pgmap v1636: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:58:32.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:58:32 smithi097 ceph-mon[39416]: pgmap v1636: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:58:32.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:58:32 smithi097 ceph-mon[30040]: pgmap v1636: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:58:34.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:58:34 smithi144 ceph-mon[33633]: pgmap v1637: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:58:34.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:58:34 smithi097 ceph-mon[39416]: pgmap v1637: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:58:34.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:58:34 smithi097 ceph-mon[30040]: pgmap v1637: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:58:35.371 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:58:35 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:58:36.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:58:36 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:58:36.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:58:36 smithi144 ceph-mon[33633]: pgmap v1638: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:58:36.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:58:36 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:58:36.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:58:36 smithi097 ceph-mon[39416]: pgmap v1638: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:58:36.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:58:36 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:58:36.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:58:36 smithi097 ceph-mon[30040]: pgmap v1638: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:58:38.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:58:38 smithi144 ceph-mon[33633]: pgmap v1639: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:58:38.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:58:38 smithi097 ceph-mon[39416]: pgmap v1639: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:58:38.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:58:38 smithi097 ceph-mon[30040]: pgmap v1639: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:58:40.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:58:40 smithi144 ceph-mon[33633]: pgmap v1640: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:58:40.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:58:40 smithi097 ceph-mon[39416]: pgmap v1640: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:58:40.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:58:40 smithi097 ceph-mon[30040]: pgmap v1640: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:58:42.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:58:41 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:58:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:58:42.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:58:42 smithi144 ceph-mon[33633]: pgmap v1641: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:58:42.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:58:42 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:58:42.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:58:42 smithi097 ceph-mon[39416]: pgmap v1641: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:58:42.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:58:42 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:58:42.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:58:42 smithi097 ceph-mon[30040]: pgmap v1641: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:58:42.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:58:42 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:58:44.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:58:44 smithi144 ceph-mon[33633]: pgmap v1642: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:58:44.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:58:44 smithi097 ceph-mon[39416]: pgmap v1642: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:58:44.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:58:44 smithi097 ceph-mon[30040]: pgmap v1642: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:58:45.371 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:58:45 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:58:46.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:58:46 smithi144 ceph-mon[33633]: pgmap v1643: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:58:46.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:58:46 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:58:46.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:58:46 smithi097 ceph-mon[39416]: pgmap v1643: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:58:46.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:58:46 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:58:46.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:58:46 smithi097 ceph-mon[30040]: pgmap v1643: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:58:46.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:58:46 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:58:48.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:58:48 smithi144 ceph-mon[33633]: pgmap v1644: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:58:48.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:58:48 smithi097 ceph-mon[39416]: pgmap v1644: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:58:48.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:58:48 smithi097 ceph-mon[30040]: pgmap v1644: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:58:50.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:58:50 smithi144 ceph-mon[33633]: pgmap v1645: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:58:50.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:58:50 smithi097 ceph-mon[39416]: pgmap v1645: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:58:50.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:58:50 smithi097 ceph-mon[30040]: pgmap v1645: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:58:52.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:58:51 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:58:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:58:52.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:58:52 smithi144 ceph-mon[33633]: pgmap v1646: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:58:52.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:58:52 smithi097 ceph-mon[39416]: pgmap v1646: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:58:52.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:58:52 smithi097 ceph-mon[30040]: pgmap v1646: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:58:54.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:58:54 smithi144 ceph-mon[33633]: pgmap v1647: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:58:54.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:58:54 smithi097 ceph-mon[39416]: pgmap v1647: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:58:54.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:58:54 smithi097 ceph-mon[30040]: pgmap v1647: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:58:55.371 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:58:55 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:58:56.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:58:56 smithi144 ceph-mon[33633]: pgmap v1648: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:58:56.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:58:56 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:58:56.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:58:56 smithi097 ceph-mon[39416]: pgmap v1648: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:58:56.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:58:56 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:58:56.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:58:56 smithi097 ceph-mon[30040]: pgmap v1648: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:58:56.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:58:56 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:58:57.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:58:57 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:58:57.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:58:57 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:58:57.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:58:57 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:58:58.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:58:58 smithi144 ceph-mon[33633]: pgmap v1649: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:58:58.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:58:58 smithi097 ceph-mon[39416]: pgmap v1649: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:58:58.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:58:58 smithi097 ceph-mon[30040]: pgmap v1649: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:59:00.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:00 smithi144 ceph-mon[33633]: pgmap v1650: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:59:00.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:00 smithi097 ceph-mon[39416]: pgmap v1650: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:59:00.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:00 smithi097 ceph-mon[30040]: pgmap v1650: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:59:02.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:59:01 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:59:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:59:02.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:02 smithi144 ceph-mon[33633]: pgmap v1651: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:59:02.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:02 smithi097 ceph-mon[39416]: pgmap v1651: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:59:02.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:02 smithi097 ceph-mon[30040]: pgmap v1651: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:59:04.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:04 smithi144 ceph-mon[33633]: pgmap v1652: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:59:04.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:04 smithi097 ceph-mon[39416]: pgmap v1652: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:59:04.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:04 smithi097 ceph-mon[30040]: pgmap v1652: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:59:05.371 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:59:05 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:59:06.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:06 smithi144 ceph-mon[33633]: pgmap v1653: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:59:06.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:06 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:59:06.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:06 smithi097 ceph-mon[30040]: pgmap v1653: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:59:06.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:06 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:59:06.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:06 smithi097 ceph-mon[39416]: pgmap v1653: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:59:06.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:06 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:59:08.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:08 smithi097 ceph-mon[39416]: pgmap v1654: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:59:08.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:08 smithi097 ceph-mon[30040]: pgmap v1654: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:59:08.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:08 smithi144 ceph-mon[33633]: pgmap v1654: 260 pgs: 260 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:59:09.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:09 smithi097 ceph-mon[30040]: osdmap e755: 8 total, 8 up, 8 in 2024-03-21T13:59:09.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:09 smithi097 ceph-mon[39416]: osdmap e755: 8 total, 8 up, 8 in 2024-03-21T13:59:09.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:09 smithi144 ceph-mon[33633]: osdmap e755: 8 total, 8 up, 8 in 2024-03-21T13:59:10.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:10 smithi097 ceph-mon[39416]: pgmap v1656: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:59:10.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:10 smithi097 ceph-mon[39416]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:59:10.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:10 smithi097 ceph-mon[39416]: osdmap e756: 8 total, 8 up, 8 in 2024-03-21T13:59:10.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:10 smithi097 ceph-mon[30040]: pgmap v1656: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:59:10.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:10 smithi097 ceph-mon[30040]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:59:10.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:10 smithi097 ceph-mon[30040]: osdmap e756: 8 total, 8 up, 8 in 2024-03-21T13:59:10.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:10 smithi144 ceph-mon[33633]: pgmap v1656: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:59:10.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:10 smithi144 ceph-mon[33633]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:59:10.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:10 smithi144 ceph-mon[33633]: osdmap e756: 8 total, 8 up, 8 in 2024-03-21T13:59:11.096 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.096 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.096 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.096 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.096 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.097 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.097 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.097 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.097 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.097 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.097 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.097 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.097 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.097 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.097 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.097 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.097 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.098 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.098 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.098 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.098 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.098 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.098 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.098 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.098 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.098 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.098 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.099 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.099 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.099 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.099 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.099 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.099 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.099 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.099 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.099 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.099 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.100 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.100 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.100 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.100 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.100 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.100 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.100 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.100 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.100 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.100 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.100 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.100 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.101 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.101 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.101 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.101 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 1 sec 2024-03-21T13:59:11.101 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: started 512 aios 2024-03-21T13:59:11.101 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a26b600 2024-03-21T13:59:11.101 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5af790 2024-03-21T13:59:11.101 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b01c0 2024-03-21T13:59:11.101 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b07e0 2024-03-21T13:59:11.101 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b06b0 2024-03-21T13:59:11.101 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b0d30 2024-03-21T13:59:11.102 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b11e0 2024-03-21T13:59:11.102 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b1710 2024-03-21T13:59:11.102 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b1c20 2024-03-21T13:59:11.102 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b1dc0 2024-03-21T13:59:11.102 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b22e0 2024-03-21T13:59:11.102 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b2480 2024-03-21T13:59:11.102 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b1820 2024-03-21T13:59:11.102 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b1950 2024-03-21T13:59:11.102 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b1a60 2024-03-21T13:59:11.102 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b2590 2024-03-21T13:59:11.102 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b26a0 2024-03-21T13:59:11.102 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b2cd0 2024-03-21T13:59:11.103 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b2820 2024-03-21T13:59:11.103 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b2930 2024-03-21T13:59:11.103 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b2a40 2024-03-21T13:59:11.103 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b2b50 2024-03-21T13:59:11.103 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b3290 2024-03-21T13:59:11.103 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b2de0 2024-03-21T13:59:11.103 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b1ef0 2024-03-21T13:59:11.103 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b2020 2024-03-21T13:59:11.103 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b2150 2024-03-21T13:59:11.103 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b3870 2024-03-21T13:59:11.103 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b3760 2024-03-21T13:59:11.103 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b2f30 2024-03-21T13:59:11.103 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b3080 2024-03-21T13:59:11.103 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b39a0 2024-03-21T13:59:11.103 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b3af0 2024-03-21T13:59:11.104 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b40b0 2024-03-21T13:59:11.104 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b3c00 2024-03-21T13:59:11.104 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b3d30 2024-03-21T13:59:11.104 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b3e40 2024-03-21T13:59:11.104 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b3f50 2024-03-21T13:59:11.104 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b4670 2024-03-21T13:59:11.104 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b41c0 2024-03-21T13:59:11.104 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b33a0 2024-03-21T13:59:11.104 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b34d0 2024-03-21T13:59:11.104 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b3600 2024-03-21T13:59:11.104 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b4c50 2024-03-21T13:59:11.104 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b47a0 2024-03-21T13:59:11.104 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b48d0 2024-03-21T13:59:11.104 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b4a20 2024-03-21T13:59:11.105 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b4d60 2024-03-21T13:59:11.105 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b5210 2024-03-21T13:59:11.105 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b5320 2024-03-21T13:59:11.105 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b5920 2024-03-21T13:59:11.105 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b4e70 2024-03-21T13:59:11.105 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b4fa0 2024-03-21T13:59:11.105 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b50b0 2024-03-21T13:59:11.105 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b5a30 2024-03-21T13:59:11.105 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b5470 2024-03-21T13:59:11.105 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b55a0 2024-03-21T13:59:11.105 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b56d0 2024-03-21T13:59:11.105 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b5800 2024-03-21T13:59:11.105 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b6030 2024-03-21T13:59:11.105 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b5b80 2024-03-21T13:59:11.106 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b5cb0 2024-03-21T13:59:11.106 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b5dc0 2024-03-21T13:59:11.106 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b5ef0 2024-03-21T13:59:11.106 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b6630 2024-03-21T13:59:11.106 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b6180 2024-03-21T13:59:11.106 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b62d0 2024-03-21T13:59:11.106 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b6400 2024-03-21T13:59:11.106 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b6510 2024-03-21T13:59:11.106 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b6c10 2024-03-21T13:59:11.106 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b6760 2024-03-21T13:59:11.106 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b6d40 2024-03-21T13:59:11.106 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b6e90 2024-03-21T13:59:11.106 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b42d0 2024-03-21T13:59:11.107 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b4400 2024-03-21T13:59:11.107 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b4550 2024-03-21T13:59:11.107 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b6ff0 2024-03-21T13:59:11.107 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b74e0 2024-03-21T13:59:11.107 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b7610 2024-03-21T13:59:11.107 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b7bf0 2024-03-21T13:59:11.107 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b7140 2024-03-21T13:59:11.107 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b7270 2024-03-21T13:59:11.107 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b7380 2024-03-21T13:59:11.107 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b7d20 2024-03-21T13:59:11.107 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b7740 2024-03-21T13:59:11.107 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b7870 2024-03-21T13:59:11.107 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b79c0 2024-03-21T13:59:11.107 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b7e30 2024-03-21T13:59:11.107 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b7f80 2024-03-21T13:59:11.108 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b8560 2024-03-21T13:59:11.108 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b80b0 2024-03-21T13:59:11.108 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b6870 2024-03-21T13:59:11.108 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b6980 2024-03-21T13:59:11.108 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b6ad0 2024-03-21T13:59:11.108 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b8b20 2024-03-21T13:59:11.108 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b8670 2024-03-21T13:59:11.108 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b81c0 2024-03-21T13:59:11.108 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b82d0 2024-03-21T13:59:11.108 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b83e0 2024-03-21T13:59:11.108 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b9120 2024-03-21T13:59:11.108 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b8c70 2024-03-21T13:59:11.108 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b8da0 2024-03-21T13:59:11.108 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b8eb0 2024-03-21T13:59:11.109 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b9000 2024-03-21T13:59:11.109 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b7ad0 2024-03-21T13:59:11.109 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b92a0 2024-03-21T13:59:11.109 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b93f0 2024-03-21T13:59:11.109 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b99f0 2024-03-21T13:59:11.109 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b9540 2024-03-21T13:59:11.109 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b9690 2024-03-21T13:59:11.109 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b97e0 2024-03-21T13:59:11.109 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b9b00 2024-03-21T13:59:11.109 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b9ff0 2024-03-21T13:59:11.109 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5ba120 2024-03-21T13:59:11.109 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5ba700 2024-03-21T13:59:11.109 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b9c50 2024-03-21T13:59:11.110 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b9d80 2024-03-21T13:59:11.110 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b9eb0 2024-03-21T13:59:11.110 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5ba850 2024-03-21T13:59:11.110 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5ba250 2024-03-21T13:59:11.110 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5ba380 2024-03-21T13:59:11.110 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5ba490 2024-03-21T13:59:11.110 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5ba5c0 2024-03-21T13:59:11.110 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bae30 2024-03-21T13:59:11.110 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5ba980 2024-03-21T13:59:11.110 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5baab0 2024-03-21T13:59:11.110 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5babc0 2024-03-21T13:59:11.110 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bacf0 2024-03-21T13:59:11.110 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bb470 2024-03-21T13:59:11.111 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bafc0 2024-03-21T13:59:11.111 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bb110 2024-03-21T13:59:11.111 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bb220 2024-03-21T13:59:11.111 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bb350 2024-03-21T13:59:11.111 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bba70 2024-03-21T13:59:11.111 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bb5c0 2024-03-21T13:59:11.111 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bb710 2024-03-21T13:59:11.111 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bb820 2024-03-21T13:59:11.111 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bb950 2024-03-21T13:59:11.111 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bc070 2024-03-21T13:59:11.111 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bbbc0 2024-03-21T13:59:11.111 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bbd10 2024-03-21T13:59:11.111 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bbe20 2024-03-21T13:59:11.111 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bc1a0 2024-03-21T13:59:11.111 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bc670 2024-03-21T13:59:11.112 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bc7c0 2024-03-21T13:59:11.112 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bcda0 2024-03-21T13:59:11.112 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bc2b0 2024-03-21T13:59:11.112 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bc400 2024-03-21T13:59:11.112 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bc530 2024-03-21T13:59:11.112 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bbf50 2024-03-21T13:59:11.112 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bc8f0 2024-03-21T13:59:11.112 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bca60 2024-03-21T13:59:11.112 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bcb70 2024-03-21T13:59:11.112 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bced0 2024-03-21T13:59:11.112 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bd3a0 2024-03-21T13:59:11.112 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bd4f0 2024-03-21T13:59:11.112 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bcfe0 2024-03-21T13:59:11.112 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bd0f0 2024-03-21T13:59:11.113 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bd240 2024-03-21T13:59:11.113 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bd640 2024-03-21T13:59:11.113 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bdb10 2024-03-21T13:59:11.113 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bdc40 2024-03-21T13:59:11.113 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bd750 2024-03-21T13:59:11.113 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bd8a0 2024-03-21T13:59:11.113 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bd9f0 2024-03-21T13:59:11.113 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bdd50 2024-03-21T13:59:11.113 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5be220 2024-03-21T13:59:11.113 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5be370 2024-03-21T13:59:11.113 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b8780 2024-03-21T13:59:11.113 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5b88d0 2024-03-21T13:59:11.113 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5be930 2024-03-21T13:59:11.113 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5be480 2024-03-21T13:59:11.114 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5be5d0 2024-03-21T13:59:11.114 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5be720 2024-03-21T13:59:11.114 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bea60 2024-03-21T13:59:11.114 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bef30 2024-03-21T13:59:11.114 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bf040 2024-03-21T13:59:11.114 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5beb70 2024-03-21T13:59:11.114 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5beca0 2024-03-21T13:59:11.114 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bedd0 2024-03-21T13:59:11.114 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bf150 2024-03-21T13:59:11.114 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bf620 2024-03-21T13:59:11.114 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bf750 2024-03-21T13:59:11.114 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bde60 2024-03-21T13:59:11.114 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bdf90 2024-03-21T13:59:11.114 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5be0c0 2024-03-21T13:59:11.115 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bf880 2024-03-21T13:59:11.115 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bfd30 2024-03-21T13:59:11.115 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bfe80 2024-03-21T13:59:11.115 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c0480 2024-03-21T13:59:11.115 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bf990 2024-03-21T13:59:11.115 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bfae0 2024-03-21T13:59:11.115 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bfbf0 2024-03-21T13:59:11.115 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c05b0 2024-03-21T13:59:11.115 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bffd0 2024-03-21T13:59:11.115 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c0100 2024-03-21T13:59:11.115 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c0210 2024-03-21T13:59:11.115 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c0360 2024-03-21T13:59:11.115 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c0b90 2024-03-21T13:59:11.115 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c06e0 2024-03-21T13:59:11.115 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c0810 2024-03-21T13:59:11.115 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c0920 2024-03-21T13:59:11.116 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c0a70 2024-03-21T13:59:11.116 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c1190 2024-03-21T13:59:11.116 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c0ce0 2024-03-21T13:59:11.116 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c0e10 2024-03-21T13:59:11.116 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c0f20 2024-03-21T13:59:11.116 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c1070 2024-03-21T13:59:11.116 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c1790 2024-03-21T13:59:11.116 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c12e0 2024-03-21T13:59:11.116 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c1430 2024-03-21T13:59:11.116 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c1540 2024-03-21T13:59:11.116 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c1670 2024-03-21T13:59:11.116 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c1d90 2024-03-21T13:59:11.116 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c18e0 2024-03-21T13:59:11.116 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c1a10 2024-03-21T13:59:11.116 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c1b20 2024-03-21T13:59:11.117 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c1c70 2024-03-21T13:59:11.117 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c2390 2024-03-21T13:59:11.117 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c1ee0 2024-03-21T13:59:11.117 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c2030 2024-03-21T13:59:11.117 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c2180 2024-03-21T13:59:11.117 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c24e0 2024-03-21T13:59:11.117 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c29b0 2024-03-21T13:59:11.117 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c2ae0 2024-03-21T13:59:11.117 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c25f0 2024-03-21T13:59:11.117 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c2740 2024-03-21T13:59:11.117 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c2870 2024-03-21T13:59:11.117 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c2c10 2024-03-21T13:59:11.117 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c30e0 2024-03-21T13:59:11.117 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c3210 2024-03-21T13:59:11.117 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c2d20 2024-03-21T13:59:11.117 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c2e70 2024-03-21T13:59:11.118 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c2fc0 2024-03-21T13:59:11.118 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c3320 2024-03-21T13:59:11.118 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c3810 2024-03-21T13:59:11.118 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c3940 2024-03-21T13:59:11.118 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c3f00 2024-03-21T13:59:11.118 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c3470 2024-03-21T13:59:11.118 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c35c0 2024-03-21T13:59:11.118 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c3a50 2024-03-21T13:59:11.118 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c3b80 2024-03-21T13:59:11.118 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c3cb0 2024-03-21T13:59:11.118 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c3de0 2024-03-21T13:59:11.118 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c4500 2024-03-21T13:59:11.119 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c4050 2024-03-21T13:59:11.119 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c4180 2024-03-21T13:59:11.119 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c4290 2024-03-21T13:59:11.119 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c43e0 2024-03-21T13:59:11.119 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c4ae0 2024-03-21T13:59:11.119 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c4630 2024-03-21T13:59:11.119 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bf280 2024-03-21T13:59:11.119 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bf390 2024-03-21T13:59:11.119 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5bf4e0 2024-03-21T13:59:11.119 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c50c0 2024-03-21T13:59:11.119 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c4c10 2024-03-21T13:59:11.119 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c4740 2024-03-21T13:59:11.119 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c4850 2024-03-21T13:59:11.119 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c49a0 2024-03-21T13:59:11.120 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c56c0 2024-03-21T13:59:11.120 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c5210 2024-03-21T13:59:11.120 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c5340 2024-03-21T13:59:11.120 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c5450 2024-03-21T13:59:11.120 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c5580 2024-03-21T13:59:11.120 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c5cc0 2024-03-21T13:59:11.120 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c5810 2024-03-21T13:59:11.120 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c5960 2024-03-21T13:59:11.120 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c5a90 2024-03-21T13:59:11.120 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c5e10 2024-03-21T13:59:11.120 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c6300 2024-03-21T13:59:11.120 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c6450 2024-03-21T13:59:11.120 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c5f40 2024-03-21T13:59:11.121 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c6090 2024-03-21T13:59:11.121 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c61e0 2024-03-21T13:59:11.121 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c6580 2024-03-21T13:59:11.121 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c6a50 2024-03-21T13:59:11.121 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c6b80 2024-03-21T13:59:11.121 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c6690 2024-03-21T13:59:11.121 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c67e0 2024-03-21T13:59:11.121 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c6910 2024-03-21T13:59:11.121 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c6c90 2024-03-21T13:59:11.121 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c7160 2024-03-21T13:59:11.121 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c72d0 2024-03-21T13:59:11.121 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c78d0 2024-03-21T13:59:11.121 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c6dc0 2024-03-21T13:59:11.121 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c6f10 2024-03-21T13:59:11.121 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c7040 2024-03-21T13:59:11.122 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c7a20 2024-03-21T13:59:11.122 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c7420 2024-03-21T13:59:11.122 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c7570 2024-03-21T13:59:11.122 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c76a0 2024-03-21T13:59:11.122 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c77b0 2024-03-21T13:59:11.122 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c7fe0 2024-03-21T13:59:11.122 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c7b30 2024-03-21T13:59:11.122 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c7ca0 2024-03-21T13:59:11.122 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c7df0 2024-03-21T13:59:11.122 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c8130 2024-03-21T13:59:11.122 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c8600 2024-03-21T13:59:11.122 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c8730 2024-03-21T13:59:11.123 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c4d20 2024-03-21T13:59:11.123 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c4e50 2024-03-21T13:59:11.123 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c4f80 2024-03-21T13:59:11.123 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c8860 2024-03-21T13:59:11.123 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c8d30 2024-03-21T13:59:11.123 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c8e80 2024-03-21T13:59:11.123 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c9460 2024-03-21T13:59:11.123 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c8990 2024-03-21T13:59:11.123 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c8ac0 2024-03-21T13:59:11.123 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c8bf0 2024-03-21T13:59:11.123 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c9570 2024-03-21T13:59:11.123 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c8fb0 2024-03-21T13:59:11.123 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c90e0 2024-03-21T13:59:11.123 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c91f0 2024-03-21T13:59:11.124 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c9320 2024-03-21T13:59:11.124 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c9b70 2024-03-21T13:59:11.124 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c96c0 2024-03-21T13:59:11.124 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c97f0 2024-03-21T13:59:11.124 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c9900 2024-03-21T13:59:11.124 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c9a50 2024-03-21T13:59:11.124 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5ca170 2024-03-21T13:59:11.124 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c9cc0 2024-03-21T13:59:11.124 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c9df0 2024-03-21T13:59:11.124 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c9f00 2024-03-21T13:59:11.124 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5ca050 2024-03-21T13:59:11.124 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5ca750 2024-03-21T13:59:11.124 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5ca2a0 2024-03-21T13:59:11.124 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5ca3d0 2024-03-21T13:59:11.124 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5ca4e0 2024-03-21T13:59:11.125 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5ca630 2024-03-21T13:59:11.125 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cad50 2024-03-21T13:59:11.125 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5ca8a0 2024-03-21T13:59:11.125 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5ca9f0 2024-03-21T13:59:11.125 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cab40 2024-03-21T13:59:11.125 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5caea0 2024-03-21T13:59:11.125 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cb3c0 2024-03-21T13:59:11.125 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cb4f0 2024-03-21T13:59:11.125 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c8240 2024-03-21T13:59:11.125 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c8370 2024-03-21T13:59:11.125 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5c84a0 2024-03-21T13:59:11.125 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cb620 2024-03-21T13:59:11.125 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cbaf0 2024-03-21T13:59:11.125 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cbc40 2024-03-21T13:59:11.126 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cafd0 2024-03-21T13:59:11.126 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cb100 2024-03-21T13:59:11.126 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cb230 2024-03-21T13:59:11.126 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cbd70 2024-03-21T13:59:11.126 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cc240 2024-03-21T13:59:11.126 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cc3c0 2024-03-21T13:59:11.126 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cc980 2024-03-21T13:59:11.126 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cbea0 2024-03-21T13:59:11.126 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cbfd0 2024-03-21T13:59:11.126 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cc100 2024-03-21T13:59:11.126 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5ccad0 2024-03-21T13:59:11.126 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cc4d0 2024-03-21T13:59:11.126 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cc620 2024-03-21T13:59:11.126 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cc750 2024-03-21T13:59:11.127 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5ccc20 2024-03-21T13:59:11.127 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cd0f0 2024-03-21T13:59:11.127 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cd220 2024-03-21T13:59:11.127 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cb750 2024-03-21T13:59:11.127 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cb880 2024-03-21T13:59:11.127 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cb9b0 2024-03-21T13:59:11.127 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cd350 2024-03-21T13:59:11.127 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cd870 2024-03-21T13:59:11.127 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cd9c0 2024-03-21T13:59:11.127 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cdfa0 2024-03-21T13:59:11.127 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cd480 2024-03-21T13:59:11.127 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cd5d0 2024-03-21T13:59:11.127 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cd700 2024-03-21T13:59:11.128 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5ce0f0 2024-03-21T13:59:11.128 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cdaf0 2024-03-21T13:59:11.128 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cdc40 2024-03-21T13:59:11.128 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cdd70 2024-03-21T13:59:11.128 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5ce220 2024-03-21T13:59:11.128 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5ce6d0 2024-03-21T13:59:11.128 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5ce800 2024-03-21T13:59:11.128 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cde80 2024-03-21T13:59:11.128 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5ce930 2024-03-21T13:59:11.128 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cead0 2024-03-21T13:59:11.128 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cf0d0 2024-03-21T13:59:11.128 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5ccd50 2024-03-21T13:59:11.128 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cce80 2024-03-21T13:59:11.128 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5ccfb0 2024-03-21T13:59:11.129 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cf200 2024-03-21T13:59:11.129 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cec20 2024-03-21T13:59:11.129 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5ced50 2024-03-21T13:59:11.129 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cee60 2024-03-21T13:59:11.129 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cef90 2024-03-21T13:59:11.129 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cf7e0 2024-03-21T13:59:11.129 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cf330 2024-03-21T13:59:11.129 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cf480 2024-03-21T13:59:11.129 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cf5d0 2024-03-21T13:59:11.129 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cf930 2024-03-21T13:59:11.129 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cfe00 2024-03-21T13:59:11.129 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cff30 2024-03-21T13:59:11.129 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5ce330 2024-03-21T13:59:11.129 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5ce460 2024-03-21T13:59:11.129 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5ce590 2024-03-21T13:59:11.130 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d0060 2024-03-21T13:59:11.130 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d0530 2024-03-21T13:59:11.130 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d0680 2024-03-21T13:59:11.130 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cfa60 2024-03-21T13:59:11.130 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cfb90 2024-03-21T13:59:11.130 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5cfcc0 2024-03-21T13:59:11.130 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d07b0 2024-03-21T13:59:11.130 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d0ca0 2024-03-21T13:59:11.130 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d0df0 2024-03-21T13:59:11.130 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d0190 2024-03-21T13:59:11.130 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d02c0 2024-03-21T13:59:11.130 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d03f0 2024-03-21T13:59:11.130 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d0f20 2024-03-21T13:59:11.130 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d1410 2024-03-21T13:59:11.131 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d1540 2024-03-21T13:59:11.131 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d08e0 2024-03-21T13:59:11.131 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d0a30 2024-03-21T13:59:11.131 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d0b40 2024-03-21T13:59:11.131 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d1650 2024-03-21T13:59:11.131 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d1b60 2024-03-21T13:59:11.131 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d1cb0 2024-03-21T13:59:11.131 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d1070 2024-03-21T13:59:11.131 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d11a0 2024-03-21T13:59:11.131 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d12d0 2024-03-21T13:59:11.131 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d1de0 2024-03-21T13:59:11.131 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d22d0 2024-03-21T13:59:11.131 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d2420 2024-03-21T13:59:11.131 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d17a0 2024-03-21T13:59:11.131 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d18f0 2024-03-21T13:59:11.132 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d1a20 2024-03-21T13:59:11.132 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d2550 2024-03-21T13:59:11.132 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d2a20 2024-03-21T13:59:11.132 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d2b50 2024-03-21T13:59:11.132 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d2660 2024-03-21T13:59:11.132 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d27b0 2024-03-21T13:59:11.132 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d2900 2024-03-21T13:59:11.132 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d2c60 2024-03-21T13:59:11.132 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d3130 2024-03-21T13:59:11.132 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d3260 2024-03-21T13:59:11.132 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d1ef0 2024-03-21T13:59:11.132 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d2020 2024-03-21T13:59:11.132 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d2170 2024-03-21T13:59:11.133 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d3370 2024-03-21T13:59:11.133 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d34e0 2024-03-21T13:59:11.133 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d3ae0 2024-03-21T13:59:11.133 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d3630 2024-03-21T13:59:11.133 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d3760 2024-03-21T13:59:11.133 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d3890 2024-03-21T13:59:11.133 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d39c0 2024-03-21T13:59:11.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:11 smithi097 ceph-mon[39416]: osdmap e757: 8 total, 8 up, 8 in 2024-03-21T13:59:11.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:11 smithi097 ceph-mon[30040]: osdmap e757: 8 total, 8 up, 8 in 2024-03-21T13:59:11.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:11 smithi144 ceph-mon[33633]: osdmap e757: 8 total, 8 up, 8 in 2024-03-21T13:59:12.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:59:11 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:59:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:59:12.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:12 smithi097 ceph-mon[39416]: pgmap v1659: 260 pgs: 32 unknown, 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:59:12.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:12 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:59:12.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:12 smithi097 ceph-mon[39416]: osdmap e758: 8 total, 8 up, 8 in 2024-03-21T13:59:12.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:12 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:59:12.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:12 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:59:12.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:12 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:59:12.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:12 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:59:12.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:12 smithi097 ceph-mon[30040]: pgmap v1659: 260 pgs: 32 unknown, 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:59:12.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:12 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:59:12.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:12 smithi097 ceph-mon[30040]: osdmap e758: 8 total, 8 up, 8 in 2024-03-21T13:59:12.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:12 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:59:12.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:12 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:59:12.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:12 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:59:12.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:12 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:59:12.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:12 smithi144 ceph-mon[33633]: pgmap v1659: 260 pgs: 32 unknown, 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:59:12.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:12 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:59:12.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:12 smithi144 ceph-mon[33633]: osdmap e758: 8 total, 8 up, 8 in 2024-03-21T13:59:12.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:12 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:59:12.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:12 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:59:12.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:12 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:59:12.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:12 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:59:13.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:13 smithi097 ceph-mon[39416]: osdmap e759: 8 total, 8 up, 8 in 2024-03-21T13:59:13.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:13 smithi097 ceph-mon[30040]: osdmap e759: 8 total, 8 up, 8 in 2024-03-21T13:59:13.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:13 smithi144 ceph-mon[33633]: osdmap e759: 8 total, 8 up, 8 in 2024-03-21T13:59:14.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:14 smithi097 ceph-mon[30040]: pgmap v1662: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:59:14.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:14 smithi097 ceph-mon[30040]: osdmap e760: 8 total, 8 up, 8 in 2024-03-21T13:59:14.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:14 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3143203452' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi097-79867-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:59:14.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:14 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi097-79867-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:59:14.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:14 smithi097 ceph-mon[39416]: pgmap v1662: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:59:14.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:14 smithi097 ceph-mon[39416]: osdmap e760: 8 total, 8 up, 8 in 2024-03-21T13:59:14.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:14 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3143203452' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi097-79867-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:59:14.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:14 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi097-79867-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:59:14.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:14 smithi144 ceph-mon[33633]: pgmap v1662: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:59:14.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:14 smithi144 ceph-mon[33633]: osdmap e760: 8 total, 8 up, 8 in 2024-03-21T13:59:14.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:14 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3143203452' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi097-79867-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:59:14.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:14 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi097-79867-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:59:15.371 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:59:15 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:59:15.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:15 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi097-79867-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:59:15.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:15 smithi097 ceph-mon[39416]: osdmap e761: 8 total, 8 up, 8 in 2024-03-21T13:59:15.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:15 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3143203452' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-03-21T13:59:15.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:15 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3143203452' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi097-79867-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:59:15.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:15 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi097-79867-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:59:15.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:15 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi097-79867-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:59:15.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:15 smithi097 ceph-mon[30040]: osdmap e761: 8 total, 8 up, 8 in 2024-03-21T13:59:15.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:15 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3143203452' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-03-21T13:59:15.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:15 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3143203452' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi097-79867-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:59:15.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:15 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi097-79867-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:59:15.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:15 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi097-79867-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:59:15.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:15 smithi144 ceph-mon[33633]: osdmap e761: 8 total, 8 up, 8 in 2024-03-21T13:59:15.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:15 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3143203452' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-03-21T13:59:15.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:15 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3143203452' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi097-79867-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:59:15.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:15 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi097-79867-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:59:16.775 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:16 smithi144 ceph-mon[33633]: pgmap v1665: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:59:16.775 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:16 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:59:16.776 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:16 smithi144 ceph-mon[33633]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:59:16.776 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:16 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi097-79867-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-03-21T13:59:16.776 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:16 smithi144 ceph-mon[33633]: osdmap e762: 8 total, 8 up, 8 in 2024-03-21T13:59:16.776 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:16 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3143203452' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi097-79867-1","app": "app2"}]: dispatch 2024-03-21T13:59:16.776 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:16 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3143203452' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi097-79867-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:59:16.776 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:16 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi097-79867-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:59:16.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:16 smithi097 ceph-mon[39416]: pgmap v1665: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:59:16.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:16 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:59:16.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:16 smithi097 ceph-mon[39416]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:59:16.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:16 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi097-79867-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-03-21T13:59:16.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:16 smithi097 ceph-mon[39416]: osdmap e762: 8 total, 8 up, 8 in 2024-03-21T13:59:16.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:16 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3143203452' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi097-79867-1","app": "app2"}]: dispatch 2024-03-21T13:59:16.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:16 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3143203452' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi097-79867-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:59:16.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:16 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi097-79867-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:59:16.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:16 smithi097 ceph-mon[30040]: pgmap v1665: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:59:16.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:16 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:59:16.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:16 smithi097 ceph-mon[30040]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:59:16.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:16 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi097-79867-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-03-21T13:59:16.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:16 smithi097 ceph-mon[30040]: osdmap e762: 8 total, 8 up, 8 in 2024-03-21T13:59:16.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:16 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3143203452' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi097-79867-1","app": "app2"}]: dispatch 2024-03-21T13:59:16.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:16 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3143203452' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi097-79867-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:59:16.805 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:16 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi097-79867-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:59:17.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:17 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi097-79867-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-03-21T13:59:17.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:17 smithi097 ceph-mon[30040]: osdmap e763: 8 total, 8 up, 8 in 2024-03-21T13:59:17.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:17 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3143203452' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi097-79867-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-03-21T13:59:17.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:17 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3143203452' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi097-79867-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-21T13:59:17.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:17 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi097-79867-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-21T13:59:17.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:17 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi097-79867-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-03-21T13:59:17.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:17 smithi097 ceph-mon[39416]: osdmap e763: 8 total, 8 up, 8 in 2024-03-21T13:59:17.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:17 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3143203452' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi097-79867-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-03-21T13:59:17.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:17 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3143203452' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi097-79867-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-21T13:59:17.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:17 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi097-79867-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-21T13:59:17.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:17 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi097-79867-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-03-21T13:59:17.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:17 smithi144 ceph-mon[33633]: osdmap e763: 8 total, 8 up, 8 in 2024-03-21T13:59:17.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:17 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3143203452' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi097-79867-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-03-21T13:59:17.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:17 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3143203452' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi097-79867-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-21T13:59:17.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:17 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi097-79867-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-21T13:59:18.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:18 smithi097 ceph-mon[30040]: pgmap v1668: 228 pgs: 64 unknown, 164 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:59:18.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:18 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi097-79867-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-03-21T13:59:18.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:18 smithi097 ceph-mon[30040]: osdmap e764: 8 total, 8 up, 8 in 2024-03-21T13:59:18.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:18 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3143203452' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi097-79867-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-21T13:59:18.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:18 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi097-79867-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-21T13:59:18.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:18 smithi097 ceph-mon[39416]: pgmap v1668: 228 pgs: 64 unknown, 164 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:59:18.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:18 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi097-79867-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-03-21T13:59:18.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:18 smithi097 ceph-mon[39416]: osdmap e764: 8 total, 8 up, 8 in 2024-03-21T13:59:18.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:18 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3143203452' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi097-79867-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-21T13:59:18.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:18 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi097-79867-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-21T13:59:18.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:18 smithi144 ceph-mon[33633]: pgmap v1668: 228 pgs: 64 unknown, 164 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:59:18.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:18 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi097-79867-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-03-21T13:59:18.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:18 smithi144 ceph-mon[33633]: osdmap e764: 8 total, 8 up, 8 in 2024-03-21T13:59:18.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:18 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3143203452' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi097-79867-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-21T13:59:18.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:18 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi097-79867-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-21T13:59:19.456 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d40c0 2024-03-21T13:59:19.456 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d3c10 2024-03-21T13:59:19.456 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d3d60 2024-03-21T13:59:19.456 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d3e90 2024-03-21T13:59:19.456 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d41f0 2024-03-21T13:59:19.456 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d46c0 2024-03-21T13:59:19.456 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d47f0 2024-03-21T13:59:19.456 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d2d70 2024-03-21T13:59:19.456 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d2ea0 2024-03-21T13:59:19.456 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d2fd0 2024-03-21T13:59:19.457 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d4920 2024-03-21T13:59:19.457 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d4df0 2024-03-21T13:59:19.457 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d4f40 2024-03-21T13:59:19.457 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d4300 2024-03-21T13:59:19.457 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d4450 2024-03-21T13:59:19.457 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d4580 2024-03-21T13:59:19.457 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d5070 2024-03-21T13:59:19.457 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d5540 2024-03-21T13:59:19.457 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d5670 2024-03-21T13:59:19.457 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d5180 2024-03-21T13:59:19.457 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d52d0 2024-03-21T13:59:19.457 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d5400 2024-03-21T13:59:19.457 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d57c0 2024-03-21T13:59:19.457 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d5c90 2024-03-21T13:59:19.457 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d5dc0 2024-03-21T13:59:19.458 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d6380 2024-03-21T13:59:19.458 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d58f0 2024-03-21T13:59:19.458 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d5a00 2024-03-21T13:59:19.458 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d5b50 2024-03-21T13:59:19.458 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d64d0 2024-03-21T13:59:19.458 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d5ed0 2024-03-21T13:59:19.458 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d6020 2024-03-21T13:59:19.458 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d6150 2024-03-21T13:59:19.458 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d65e0 2024-03-21T13:59:19.458 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d6ab0 2024-03-21T13:59:19.458 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d6c00 2024-03-21T13:59:19.458 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d4a30 2024-03-21T13:59:19.458 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d4b60 2024-03-21T13:59:19.458 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d4c90 2024-03-21T13:59:19.459 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d6d30 2024-03-21T13:59:19.459 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d7200 2024-03-21T13:59:19.459 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d7330 2024-03-21T13:59:19.459 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d78f0 2024-03-21T13:59:19.459 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d6e60 2024-03-21T13:59:19.459 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d6f90 2024-03-21T13:59:19.459 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d70e0 2024-03-21T13:59:19.459 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d7a40 2024-03-21T13:59:19.459 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d7440 2024-03-21T13:59:19.459 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d7570 2024-03-21T13:59:19.459 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d7680 2024-03-21T13:59:19.459 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d77d0 2024-03-21T13:59:19.459 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d8020 2024-03-21T13:59:19.459 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d7b70 2024-03-21T13:59:19.459 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d7cc0 2024-03-21T13:59:19.459 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d7dd0 2024-03-21T13:59:19.460 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d7f00 2024-03-21T13:59:19.460 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d8620 2024-03-21T13:59:19.460 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d8170 2024-03-21T13:59:19.460 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d82a0 2024-03-21T13:59:19.460 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d83b0 2024-03-21T13:59:19.460 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d8500 2024-03-21T13:59:19.460 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d8c20 2024-03-21T13:59:19.460 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d8770 2024-03-21T13:59:19.460 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d88c0 2024-03-21T13:59:19.460 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: waiting 0x55ca2a5d89f0 2024-03-21T13:59:19.461 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: done. 2024-03-21T13:59:19.461 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: [ OK ] LibRadosMiscPool.PoolCreationRace (1810771 ms) 2024-03-21T13:59:19.461 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: [----------] 1 test from LibRadosMiscPool (1810771 ms total) 2024-03-21T13:59:19.461 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: 2024-03-21T13:59:19.461 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: [----------] 8 tests from LibRadosMisc 2024-03-21T13:59:19.461 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: [ RUN ] LibRadosMisc.ClusterFSID 2024-03-21T13:59:19.461 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: [ OK ] LibRadosMisc.ClusterFSID (0 ms) 2024-03-21T13:59:19.461 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: [ RUN ] LibRadosMisc.Exec 2024-03-21T13:59:19.461 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: [ OK ] LibRadosMisc.Exec (43 ms) 2024-03-21T13:59:19.461 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: [ RUN ] LibRadosMisc.WriteSame 2024-03-21T13:59:19.461 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: [ OK ] LibRadosMisc.WriteSame (4 ms) 2024-03-21T13:59:19.461 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: [ RUN ] LibRadosMisc.CmpExt 2024-03-21T13:59:19.461 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: [ OK ] LibRadosMisc.CmpExt (2 ms) 2024-03-21T13:59:19.461 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: [ RUN ] LibRadosMisc.Applications 2024-03-21T13:59:19.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:19 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi097-79867-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-03-21T13:59:19.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:19 smithi097 ceph-mon[30040]: osdmap e765: 8 total, 8 up, 8 in 2024-03-21T13:59:19.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:19 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/3143203452' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi097-79867-1","app":"app1","key":"key1"}]: dispatch 2024-03-21T13:59:19.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:19 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi097-79867-1","app":"app1","key":"key1"}]: dispatch 2024-03-21T13:59:19.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:19 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi097-79867-1","app":"app1","key":"key1"}]': finished 2024-03-21T13:59:19.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:19 smithi097 ceph-mon[30040]: osdmap e766: 8 total, 8 up, 8 in 2024-03-21T13:59:19.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:19 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi097-79867-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-03-21T13:59:19.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:19 smithi097 ceph-mon[39416]: osdmap e765: 8 total, 8 up, 8 in 2024-03-21T13:59:19.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:19 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/3143203452' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi097-79867-1","app":"app1","key":"key1"}]: dispatch 2024-03-21T13:59:19.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:19 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi097-79867-1","app":"app1","key":"key1"}]: dispatch 2024-03-21T13:59:19.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:19 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi097-79867-1","app":"app1","key":"key1"}]': finished 2024-03-21T13:59:19.806 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:19 smithi097 ceph-mon[39416]: osdmap e766: 8 total, 8 up, 8 in 2024-03-21T13:59:19.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:19 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi097-79867-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-03-21T13:59:19.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:19 smithi144 ceph-mon[33633]: osdmap e765: 8 total, 8 up, 8 in 2024-03-21T13:59:19.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:19 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/3143203452' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi097-79867-1","app":"app1","key":"key1"}]: dispatch 2024-03-21T13:59:19.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:19 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi097-79867-1","app":"app1","key":"key1"}]: dispatch 2024-03-21T13:59:19.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:19 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi097-79867-1","app":"app1","key":"key1"}]': finished 2024-03-21T13:59:19.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:19 smithi144 ceph-mon[33633]: osdmap e766: 8 total, 8 up, 8 in 2024-03-21T13:59:20.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:20 smithi097 ceph-mon[30040]: pgmap v1671: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:59:20.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:20 smithi097 ceph-mon[30040]: osdmap e767: 8 total, 8 up, 8 in 2024-03-21T13:59:20.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:20 smithi097 ceph-mon[39416]: pgmap v1671: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:59:20.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:20 smithi097 ceph-mon[39416]: osdmap e767: 8 total, 8 up, 8 in 2024-03-21T13:59:20.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:20 smithi144 ceph-mon[33633]: pgmap v1671: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:59:20.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:20 smithi144 ceph-mon[33633]: osdmap e767: 8 total, 8 up, 8 in 2024-03-21T13:59:21.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:21 smithi097 ceph-mon[30040]: pgmap v1674: 196 pgs: 196 active+clean; 583 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:59:21.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:21 smithi097 ceph-mon[30040]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:59:21.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:21 smithi097 ceph-mon[30040]: osdmap e768: 8 total, 8 up, 8 in 2024-03-21T13:59:21.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:21 smithi097 ceph-mon[39416]: pgmap v1674: 196 pgs: 196 active+clean; 583 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:59:21.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:21 smithi097 ceph-mon[39416]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:59:21.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:21 smithi097 ceph-mon[39416]: osdmap e768: 8 total, 8 up, 8 in 2024-03-21T13:59:21.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:21 smithi144 ceph-mon[33633]: pgmap v1674: 196 pgs: 196 active+clean; 583 KiB data, 874 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:59:21.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:21 smithi144 ceph-mon[33633]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:59:21.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:21 smithi144 ceph-mon[33633]: osdmap e768: 8 total, 8 up, 8 in 2024-03-21T13:59:22.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:59:21 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:59:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:59:23.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:23 smithi097 ceph-mon[39416]: osdmap e769: 8 total, 8 up, 8 in 2024-03-21T13:59:23.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:23 smithi097 ceph-mon[30040]: osdmap e769: 8 total, 8 up, 8 in 2024-03-21T13:59:23.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:23 smithi144 ceph-mon[33633]: osdmap e769: 8 total, 8 up, 8 in 2024-03-21T13:59:24.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:24 smithi097 ceph-mon[30040]: pgmap v1677: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 874 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:59:24.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:24 smithi097 ceph-mon[30040]: osdmap e770: 8 total, 8 up, 8 in 2024-03-21T13:59:24.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:24 smithi097 ceph-mon[39416]: pgmap v1677: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 874 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:59:24.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:24 smithi097 ceph-mon[39416]: osdmap e770: 8 total, 8 up, 8 in 2024-03-21T13:59:24.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:24 smithi144 ceph-mon[33633]: pgmap v1677: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 874 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:59:24.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:24 smithi144 ceph-mon[33633]: osdmap e770: 8 total, 8 up, 8 in 2024-03-21T13:59:25.371 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:59:25 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:59:25.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:25 smithi097 ceph-mon[30040]: osdmap e771: 8 total, 8 up, 8 in 2024-03-21T13:59:25.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:25 smithi097 ceph-mon[30040]: pgmap v1680: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 881 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:59:25.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:25 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:59:25.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:25 smithi097 ceph-mon[39416]: osdmap e771: 8 total, 8 up, 8 in 2024-03-21T13:59:25.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:25 smithi097 ceph-mon[39416]: pgmap v1680: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 881 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:59:25.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:25 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:59:25.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:25 smithi144 ceph-mon[33633]: osdmap e771: 8 total, 8 up, 8 in 2024-03-21T13:59:25.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:25 smithi144 ceph-mon[33633]: pgmap v1680: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 881 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:59:25.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:25 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:59:26.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:26 smithi097 ceph-mon[30040]: osdmap e772: 8 total, 8 up, 8 in 2024-03-21T13:59:26.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:26 smithi097 ceph-mon[39416]: osdmap e772: 8 total, 8 up, 8 in 2024-03-21T13:59:26.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:26 smithi144 ceph-mon[33633]: osdmap e772: 8 total, 8 up, 8 in 2024-03-21T13:59:27.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:27 smithi097 ceph-mon[39416]: osdmap e773: 8 total, 8 up, 8 in 2024-03-21T13:59:27.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:27 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:59:27.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:27 smithi097 ceph-mon[39416]: pgmap v1683: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 881 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:59:27.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:27 smithi097 ceph-mon[30040]: osdmap e773: 8 total, 8 up, 8 in 2024-03-21T13:59:27.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:27 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:59:27.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:27 smithi097 ceph-mon[30040]: pgmap v1683: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 881 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:59:27.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:27 smithi144 ceph-mon[33633]: osdmap e773: 8 total, 8 up, 8 in 2024-03-21T13:59:27.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:27 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:59:27.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:27 smithi144 ceph-mon[33633]: pgmap v1683: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 881 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:59:28.580 INFO:tasks.workunit.client.0.smithi097.stdout: write_operations: Running main() from gmock_main.cc 2024-03-21T13:59:28.580 INFO:tasks.workunit.client.0.smithi097.stdout: write_operations: [==========] Running 7 tests from 1 test suite. 2024-03-21T13:59:28.580 INFO:tasks.workunit.client.0.smithi097.stdout: write_operations: [----------] Global test environment set-up. 2024-03-21T13:59:28.580 INFO:tasks.workunit.client.0.smithi097.stdout: write_operations: [----------] 7 tests from NeoRadosWriteOps 2024-03-21T13:59:28.580 INFO:tasks.workunit.client.0.smithi097.stdout: write_operations: [ RUN ] NeoRadosWriteOps.AssertExists 2024-03-21T13:59:28.581 INFO:tasks.workunit.client.0.smithi097.stdout: write_operations: [ OK ] NeoRadosWriteOps.AssertExists (1808500 ms) 2024-03-21T13:59:28.581 INFO:tasks.workunit.client.0.smithi097.stdout: write_operations: [ RUN ] NeoRadosWriteOps.AssertVersion 2024-03-21T13:59:28.581 INFO:tasks.workunit.client.0.smithi097.stdout: write_operations: [ OK ] NeoRadosWriteOps.AssertVersion (3036 ms) 2024-03-21T13:59:28.581 INFO:tasks.workunit.client.0.smithi097.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Xattrs 2024-03-21T13:59:28.581 INFO:tasks.workunit.client.0.smithi097.stdout: write_operations: [ OK ] NeoRadosWriteOps.Xattrs (3014 ms) 2024-03-21T13:59:28.581 INFO:tasks.workunit.client.0.smithi097.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Write 2024-03-21T13:59:28.581 INFO:tasks.workunit.client.0.smithi097.stdout: write_operations: [ OK ] NeoRadosWriteOps.Write (3033 ms) 2024-03-21T13:59:28.581 INFO:tasks.workunit.client.0.smithi097.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Exec 2024-03-21T13:59:28.581 INFO:tasks.workunit.client.0.smithi097.stdout: write_operations: [ OK ] NeoRadosWriteOps.Exec (3015 ms) 2024-03-21T13:59:28.581 INFO:tasks.workunit.client.0.smithi097.stdout: write_operations: [ RUN ] NeoRadosWriteOps.WriteSame 2024-03-21T13:59:28.581 INFO:tasks.workunit.client.0.smithi097.stdout: write_operations: [ OK ] NeoRadosWriteOps.WriteSame (3025 ms) 2024-03-21T13:59:28.581 INFO:tasks.workunit.client.0.smithi097.stdout: write_operations: [ RUN ] NeoRadosWriteOps.CmpExt 2024-03-21T13:59:28.581 INFO:tasks.workunit.client.0.smithi097.stdout: write_operations: [ OK ] NeoRadosWriteOps.CmpExt (5084 ms) 2024-03-21T13:59:28.581 INFO:tasks.workunit.client.0.smithi097.stdout: write_operations: [----------] 7 tests from NeoRadosWriteOps (1828707 ms total) 2024-03-21T13:59:28.581 INFO:tasks.workunit.client.0.smithi097.stdout: write_operations: 2024-03-21T13:59:28.582 INFO:tasks.workunit.client.0.smithi097.stdout: write_operations: [----------] Global test environment tear-down 2024-03-21T13:59:28.582 INFO:tasks.workunit.client.0.smithi097.stdout: write_operations: [==========] 7 tests from 1 test suite ran. (1828707 ms total) 2024-03-21T13:59:28.582 INFO:tasks.workunit.client.0.smithi097.stdout: write_operations: [ PASSED ] 7 tests. 2024-03-21T13:59:28.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:28 smithi097 ceph-mon[30040]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:59:28.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:28 smithi097 ceph-mon[39416]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:59:28.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:28 smithi144 ceph-mon[33633]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:59:29.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:29 smithi144 ceph-mon[33633]: osdmap e774: 8 total, 8 up, 8 in 2024-03-21T13:59:29.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:29 smithi144 ceph-mon[33633]: pgmap v1685: 196 pgs: 196 active+clean; 583 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-21T13:59:30.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:29 smithi097 ceph-mon[39416]: osdmap e774: 8 total, 8 up, 8 in 2024-03-21T13:59:30.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:29 smithi097 ceph-mon[39416]: pgmap v1685: 196 pgs: 196 active+clean; 583 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-21T13:59:30.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:29 smithi097 ceph-mon[30040]: osdmap e774: 8 total, 8 up, 8 in 2024-03-21T13:59:30.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:29 smithi097 ceph-mon[30040]: pgmap v1685: 196 pgs: 196 active+clean; 583 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-21T13:59:32.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:32 smithi097 ceph-mon[30040]: pgmap v1686: 196 pgs: 196 active+clean; 583 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:59:32.305 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:59:31 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:59:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:59:32.305 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:32 smithi097 ceph-mon[39416]: pgmap v1686: 196 pgs: 196 active+clean; 583 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:59:32.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:32 smithi144 ceph-mon[33633]: pgmap v1686: 196 pgs: 196 active+clean; 583 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:59:33.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:33 smithi097 ceph-mon[30040]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:59:33.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:33 smithi097 ceph-mon[39416]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:59:33.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:33 smithi144 ceph-mon[33633]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:59:34.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:34 smithi097 ceph-mon[39416]: pgmap v1687: 196 pgs: 196 active+clean; 583 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 678 B/s rd, 0 op/s 2024-03-21T13:59:34.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:34 smithi097 ceph-mon[30040]: pgmap v1687: 196 pgs: 196 active+clean; 583 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 678 B/s rd, 0 op/s 2024-03-21T13:59:34.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:34 smithi144 ceph-mon[33633]: pgmap v1687: 196 pgs: 196 active+clean; 583 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 678 B/s rd, 0 op/s 2024-03-21T13:59:35.371 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:59:35 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:59:36.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:36 smithi097 ceph-mon[30040]: pgmap v1688: 196 pgs: 196 active+clean; 583 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:59:36.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:36 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:59:36.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:36 smithi097 ceph-mon[39416]: pgmap v1688: 196 pgs: 196 active+clean; 583 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:59:36.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:36 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:59:36.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:36 smithi144 ceph-mon[33633]: pgmap v1688: 196 pgs: 196 active+clean; 583 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:59:36.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:36 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:59:38.161 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: [ OK ] LibRadosMisc.Applications (4989 ms) 2024-03-21T13:59:38.162 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatOSD 2024-03-21T13:59:38.162 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatOSD (0 ms) 2024-03-21T13:59:38.162 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatClient 2024-03-21T13:59:38.162 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatClient (0 ms) 2024-03-21T13:59:38.162 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: [ RUN ] LibRadosMisc.ShutdownRace 2024-03-21T13:59:38.162 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: [ OK ] LibRadosMisc.ShutdownRace (18159 ms) 2024-03-21T13:59:38.162 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: [----------] 8 tests from LibRadosMisc (23197 ms total) 2024-03-21T13:59:38.162 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: 2024-03-21T13:59:38.162 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: [----------] Global test environment tear-down 2024-03-21T13:59:38.162 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: [==========] 12 tests from 4 test suites ran. (1838613 ms total) 2024-03-21T13:59:38.162 INFO:tasks.workunit.client.0.smithi097.stdout: api_misc: [ PASSED ] 12 tests. 2024-03-21T13:59:38.167 INFO:tasks.workunit.client.0.smithi097.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:59:38.167 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=79919 2024-03-21T13:59:38.167 INFO:tasks.workunit.client.0.smithi097.stderr:+ wait 79919 2024-03-21T13:59:38.167 INFO:tasks.workunit.client.0.smithi097.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:59:38.167 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=80441 2024-03-21T13:59:38.167 INFO:tasks.workunit.client.0.smithi097.stderr:+ wait 80441 2024-03-21T13:59:38.167 INFO:tasks.workunit.client.0.smithi097.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:59:38.167 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=80509 2024-03-21T13:59:38.167 INFO:tasks.workunit.client.0.smithi097.stderr:+ wait 80509 2024-03-21T13:59:38.167 INFO:tasks.workunit.client.0.smithi097.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:59:38.167 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=79691 2024-03-21T13:59:38.167 INFO:tasks.workunit.client.0.smithi097.stderr:+ wait 79691 2024-03-21T13:59:38.167 INFO:tasks.workunit.client.0.smithi097.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:59:38.168 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=80353 2024-03-21T13:59:38.168 INFO:tasks.workunit.client.0.smithi097.stderr:+ wait 80353 2024-03-21T13:59:38.168 INFO:tasks.workunit.client.0.smithi097.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:59:38.168 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=80253 2024-03-21T13:59:38.168 INFO:tasks.workunit.client.0.smithi097.stderr:+ wait 80253 2024-03-21T13:59:38.168 INFO:tasks.workunit.client.0.smithi097.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:59:38.168 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=79567 2024-03-21T13:59:38.168 INFO:tasks.workunit.client.0.smithi097.stderr:+ wait 79567 2024-03-21T13:59:38.168 INFO:tasks.workunit.client.0.smithi097.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:59:38.168 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=79629 2024-03-21T13:59:38.168 INFO:tasks.workunit.client.0.smithi097.stderr:+ wait 79629 2024-03-21T13:59:38.169 INFO:tasks.workunit.client.0.smithi097.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:59:38.169 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=80272 2024-03-21T13:59:38.169 INFO:tasks.workunit.client.0.smithi097.stderr:+ wait 80272 2024-03-21T13:59:38.169 INFO:tasks.workunit.client.0.smithi097.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:59:38.169 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=79558 2024-03-21T13:59:38.169 INFO:tasks.workunit.client.0.smithi097.stderr:+ wait 79558 2024-03-21T13:59:38.169 INFO:tasks.workunit.client.0.smithi097.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:59:38.169 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=79579 2024-03-21T13:59:38.169 INFO:tasks.workunit.client.0.smithi097.stderr:+ wait 79579 2024-03-21T13:59:38.169 INFO:tasks.workunit.client.0.smithi097.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:59:38.169 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=80660 2024-03-21T13:59:38.169 INFO:tasks.workunit.client.0.smithi097.stderr:+ wait 80660 2024-03-21T13:59:38.170 INFO:tasks.workunit.client.0.smithi097.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:59:38.170 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=79767 2024-03-21T13:59:38.170 INFO:tasks.workunit.client.0.smithi097.stderr:+ wait 79767 2024-03-21T13:59:38.170 INFO:tasks.workunit.client.0.smithi097.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:59:38.170 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=80760 2024-03-21T13:59:38.170 INFO:tasks.workunit.client.0.smithi097.stderr:+ wait 80760 2024-03-21T13:59:38.170 INFO:tasks.workunit.client.0.smithi097.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:59:38.170 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=79889 2024-03-21T13:59:38.170 INFO:tasks.workunit.client.0.smithi097.stderr:+ wait 79889 2024-03-21T13:59:38.170 INFO:tasks.workunit.client.0.smithi097.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:59:38.170 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=80402 2024-03-21T13:59:38.170 INFO:tasks.workunit.client.0.smithi097.stderr:+ wait 80402 2024-03-21T13:59:38.170 INFO:tasks.workunit.client.0.smithi097.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:59:38.170 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=80832 2024-03-21T13:59:38.171 INFO:tasks.workunit.client.0.smithi097.stderr:+ wait 80832 2024-03-21T13:59:38.171 INFO:tasks.workunit.client.0.smithi097.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:59:38.171 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=79819 2024-03-21T13:59:38.171 INFO:tasks.workunit.client.0.smithi097.stderr:+ wait 79819 2024-03-21T13:59:38.171 INFO:tasks.workunit.client.0.smithi097.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:59:38.171 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=80310 2024-03-21T13:59:38.171 INFO:tasks.workunit.client.0.smithi097.stderr:+ wait 80310 2024-03-21T13:59:38.171 INFO:tasks.workunit.client.0.smithi097.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:59:38.171 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=80592 2024-03-21T13:59:38.171 INFO:tasks.workunit.client.0.smithi097.stderr:+ wait 80592 2024-03-21T13:59:38.171 INFO:tasks.workunit.client.0.smithi097.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:59:38.171 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=80374 2024-03-21T13:59:38.171 INFO:tasks.workunit.client.0.smithi097.stderr:+ wait 80374 2024-03-21T13:59:38.171 INFO:tasks.workunit.client.0.smithi097.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:59:38.171 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=80724 2024-03-21T13:59:38.171 INFO:tasks.workunit.client.0.smithi097.stderr:+ wait 80724 2024-03-21T13:59:38.172 INFO:tasks.workunit.client.0.smithi097.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:59:38.172 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=80242 2024-03-21T13:59:38.172 INFO:tasks.workunit.client.0.smithi097.stderr:+ wait 80242 2024-03-21T13:59:38.172 INFO:tasks.workunit.client.0.smithi097.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:59:38.172 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=79725 2024-03-21T13:59:38.172 INFO:tasks.workunit.client.0.smithi097.stderr:+ wait 79725 2024-03-21T13:59:38.172 INFO:tasks.workunit.client.0.smithi097.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:59:38.172 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=80801 2024-03-21T13:59:38.172 INFO:tasks.workunit.client.0.smithi097.stderr:+ wait 80801 2024-03-21T13:59:38.172 INFO:tasks.workunit.client.0.smithi097.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:59:38.172 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=80102 2024-03-21T13:59:38.172 INFO:tasks.workunit.client.0.smithi097.stderr:+ wait 80102 2024-03-21T13:59:38.172 INFO:tasks.workunit.client.0.smithi097.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:59:38.172 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=79551 2024-03-21T13:59:38.172 INFO:tasks.workunit.client.0.smithi097.stderr:+ wait 79551 2024-03-21T13:59:38.173 INFO:tasks.workunit.client.0.smithi097.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:59:38.173 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=79611 2024-03-21T13:59:38.173 INFO:tasks.workunit.client.0.smithi097.stderr:+ wait 79611 2024-03-21T13:59:38.173 INFO:tasks.workunit.client.0.smithi097.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:59:38.173 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=79997 2024-03-21T13:59:38.173 INFO:tasks.workunit.client.0.smithi097.stderr:+ wait 79997 2024-03-21T13:59:38.173 INFO:tasks.workunit.client.0.smithi097.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:59:38.173 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=79653 2024-03-21T13:59:38.173 INFO:tasks.workunit.client.0.smithi097.stderr:+ wait 79653 2024-03-21T13:59:38.173 INFO:tasks.workunit.client.0.smithi097.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:59:38.173 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=80146 2024-03-21T13:59:38.173 INFO:tasks.workunit.client.0.smithi097.stderr:+ wait 80146 2024-03-21T13:59:38.173 INFO:tasks.workunit.client.0.smithi097.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:59:38.173 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=80205 2024-03-21T13:59:38.174 INFO:tasks.workunit.client.0.smithi097.stderr:+ wait 80205 2024-03-21T13:59:38.174 INFO:tasks.workunit.client.0.smithi097.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:59:38.174 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=80545 2024-03-21T13:59:38.174 INFO:tasks.workunit.client.0.smithi097.stderr:+ wait 80545 2024-03-21T13:59:38.174 INFO:tasks.workunit.client.0.smithi097.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:59:38.174 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=80859 2024-03-21T13:59:38.174 INFO:tasks.workunit.client.0.smithi097.stderr:+ wait 80859 2024-03-21T13:59:38.174 INFO:tasks.workunit.client.0.smithi097.stderr:+ exit 0 2024-03-21T13:59:38.174 INFO:tasks.workunit.client.0.smithi097.stderr:+ cleanup 2024-03-21T13:59:38.174 INFO:tasks.workunit.client.0.smithi097.stderr:+ pkill -P 79545 2024-03-21T13:59:38.180 INFO:tasks.workunit.client.0.smithi097.stderr:+ true 2024-03-21T13:59:38.181 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-21T13:59:38.181 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-21T13:59:38.211 INFO:tasks.workunit:Running workunits matching rados/test_pool_quota.sh on client.0... 2024-03-21T13:59:38.213 INFO:tasks.workunit:Running workunit rados/test_pool_quota.sh... 2024-03-21T13:59:38.213 DEBUG:teuthology.orchestra.run.smithi097:workunit test rados/test_pool_quota.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=960fbbe4645c2becc8084669be194c95183c7753 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_pool_quota.sh 2024-03-21T13:59:38.273 INFO:tasks.workunit.client.0.smithi097.stderr:++ uuidgen 2024-03-21T13:59:38.281 INFO:tasks.workunit.client.0.smithi097.stderr:+ p=b5a5e36b-8518-4a8b-80b9-4898af5bd2dd 2024-03-21T13:59:38.281 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph osd pool create b5a5e36b-8518-4a8b-80b9-4898af5bd2dd 12 2024-03-21T13:59:38.366 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:38.363+0000 7f9e74886640 1 -- 172.21.15.97:0/1119994437 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9e701005a0 msgr2=0x7f9e70100a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:59:38.366 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:38.363+0000 7f9e74886640 1 --2- 172.21.15.97:0/1119994437 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9e701005a0 0x7f9e70100a20 secure :-1 s=READY pgs=3190 cs=0 l=1 rev1=1 crypto rx=0x7f9e58009930 tx=0x7f9e5802bc20 comp rx=0 tx=0).stop 2024-03-21T13:59:38.366 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:38.363+0000 7f9e74886640 1 -- 172.21.15.97:0/1119994437 shutdown_connections 2024-03-21T13:59:38.366 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:38.363+0000 7f9e74886640 1 --2- 172.21.15.97:0/1119994437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e70100f60 0x7f9e7010d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:59:38.366 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:38.363+0000 7f9e74886640 1 --2- 172.21.15.97:0/1119994437 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9e701005a0 0x7f9e70100a20 unknown :-1 s=CLOSED pgs=3190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:59:38.366 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:38.363+0000 7f9e74886640 1 --2- 172.21.15.97:0/1119994437 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e70106660 0x7f9e70106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:59:38.367 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:38.363+0000 7f9e74886640 1 -- 172.21.15.97:0/1119994437 >> 172.21.15.97:0/1119994437 conn(0x7f9e70077690 msgr2=0x7f9e70077aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:59:38.367 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:38.363+0000 7f9e74886640 1 -- 172.21.15.97:0/1119994437 shutdown_connections 2024-03-21T13:59:38.367 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:38.363+0000 7f9e74886640 1 -- 172.21.15.97:0/1119994437 wait complete. 2024-03-21T13:59:38.367 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:38.364+0000 7f9e74886640 1 Processor -- start 2024-03-21T13:59:38.367 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:38.364+0000 7f9e74886640 1 -- start start 2024-03-21T13:59:38.367 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:38.364+0000 7f9e74886640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e701005a0 0x7f9e70196260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:59:38.367 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:38.364+0000 7f9e74886640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9e70100f60 0x7f9e701967a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:59:38.367 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:38.364+0000 7f9e74886640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e70106660 0x7f9e70196ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:59:38.367 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:38.364+0000 7f9e74886640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e70110070 con 0x7f9e701005a0 2024-03-21T13:59:38.367 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:38.364+0000 7f9e74886640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7010fef0 con 0x7f9e70106660 2024-03-21T13:59:38.367 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:38.364+0000 7f9e74886640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e701101f0 con 0x7f9e70100f60 2024-03-21T13:59:38.367 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:38.364+0000 7f9e67fff640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9e70100f60 0x7f9e701967a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:59:38.367 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:38.364+0000 7f9e6e59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e701005a0 0x7f9e70196260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:59:38.368 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:38.364+0000 7f9e67fff640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9e70100f60 0x7f9e701967a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.97:45604/0 (socket says 172.21.15.97:45604) 2024-03-21T13:59:38.368 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:38.364+0000 7f9e67fff640 1 -- 172.21.15.97:0/4063893569 learned_addr learned my addr 172.21.15.97:0/4063893569 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:59:38.368 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:38.364+0000 7f9e6e59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e701005a0 0x7f9e70196260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42260/0 (socket says 172.21.15.97:42260) 2024-03-21T13:59:38.368 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:38.365+0000 7f9e6ed9d640 1 --2- 172.21.15.97:0/4063893569 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e70106660 0x7f9e70196ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:59:38.368 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:38.365+0000 7f9e67fff640 1 -- 172.21.15.97:0/4063893569 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e70106660 msgr2=0x7f9e70196ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:59:38.368 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:38.365+0000 7f9e67fff640 1 --2- 172.21.15.97:0/4063893569 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e70106660 0x7f9e70196ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:59:38.368 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:38.365+0000 7f9e67fff640 1 -- 172.21.15.97:0/4063893569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e701005a0 msgr2=0x7f9e70196260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:59:38.368 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:38.365+0000 7f9e67fff640 1 --2- 172.21.15.97:0/4063893569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e701005a0 0x7f9e70196260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:59:38.368 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:38.365+0000 7f9e67fff640 1 -- 172.21.15.97:0/4063893569 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e701a5ad0 con 0x7f9e70100f60 2024-03-21T13:59:38.368 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:38.365+0000 7f9e6e59c640 1 --2- 172.21.15.97:0/4063893569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e701005a0 0x7f9e70196260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T13:59:38.368 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:38.365+0000 7f9e6ed9d640 1 --2- 172.21.15.97:0/4063893569 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e70106660 0x7f9e70196ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T13:59:38.368 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:38.365+0000 7f9e67fff640 1 --2- 172.21.15.97:0/4063893569 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9e70100f60 0x7f9e701967a0 secure :-1 s=READY pgs=3191 cs=0 l=1 rev1=1 crypto rx=0x7f9e58009230 tx=0x7f9e5802c780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:59:38.368 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:38.365+0000 7f9e677fe640 1 -- 172.21.15.97:0/4063893569 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e58043020 con 0x7f9e70100f60 2024-03-21T13:59:38.368 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:38.365+0000 7f9e677fe640 1 -- 172.21.15.97:0/4063893569 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f9e580043d0 con 0x7f9e70100f60 2024-03-21T13:59:38.368 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:38.365+0000 7f9e74886640 1 -- 172.21.15.97:0/4063893569 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e701a5d60 con 0x7f9e70100f60 2024-03-21T13:59:38.369 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:38.365+0000 7f9e677fe640 1 -- 172.21.15.97:0/4063893569 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e58003b70 con 0x7f9e70100f60 2024-03-21T13:59:38.369 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:38.365+0000 7f9e74886640 1 -- 172.21.15.97:0/4063893569 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e701a6250 con 0x7f9e70100f60 2024-03-21T13:59:38.369 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:38.366+0000 7f9e74886640 1 -- 172.21.15.97:0/4063893569 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e700689d0 con 0x7f9e70100f60 2024-03-21T13:59:38.369 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:38.366+0000 7f9e677fe640 1 -- 172.21.15.97:0/4063893569 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 22) v1 ==== 92152+0+0 (secure 0 0 0) 0x7f9e58046020 con 0x7f9e70100f60 2024-03-21T13:59:38.370 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:38.367+0000 7f9e677fe640 1 --2- 172.21.15.97:0/4063893569 >> v2:172.21.15.97:6800/12494470 conn(0x7f9e5c06e270 0x7f9e5c070730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:59:38.370 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:38.367+0000 7f9e677fe640 1 -- 172.21.15.97:0/4063893569 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(775..775 src has 254..775) v4 ==== 7426+0+0 (secure 0 0 0) 0x7f9e580b03d0 con 0x7f9e70100f60 2024-03-21T13:59:38.370 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:38.367+0000 7f9e6e59c640 1 --2- 172.21.15.97:0/4063893569 >> v2:172.21.15.97:6800/12494470 conn(0x7f9e5c06e270 0x7f9e5c070730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:59:38.370 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:38.367+0000 7f9e6e59c640 1 --2- 172.21.15.97:0/4063893569 >> v2:172.21.15.97:6800/12494470 conn(0x7f9e5c06e270 0x7f9e5c070730 secure :-1 s=READY pgs=4029 cs=0 l=1 rev1=1 crypto rx=0x7f9e600095f0 tx=0x7f9e60005f90 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:59:38.372 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:38.372+0000 7f9e677fe640 1 -- 172.21.15.97:0/4063893569 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9e5804d070 con 0x7f9e70100f60 2024-03-21T13:59:38.550 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:38.549+0000 7f9e74886640 1 -- 172.21.15.97:0/4063893569 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "pg_num": 12} v 0) v1 -- 0x7f9e70108a70 con 0x7f9e70100f60 2024-03-21T13:59:38.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:38 smithi097 ceph-mon[39416]: pgmap v1689: 196 pgs: 196 active+clean; 583 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:59:38.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:38 smithi097 ceph-mon[30040]: pgmap v1689: 196 pgs: 196 active+clean; 583 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:59:38.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:38 smithi144 ceph-mon[33633]: pgmap v1689: 196 pgs: 196 active+clean; 583 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:59:39.164 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.164+0000 7f9e677fe640 1 -- 172.21.15.97:0/4063893569 <== mon.2 v2:172.21.15.97:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "pg_num": 12}]=0 pool 'b5a5e36b-8518-4a8b-80b9-4898af5bd2dd' created v776) v1 ==== 176+0+0 (secure 0 0 0) 0x7f9e5807d690 con 0x7f9e70100f60 2024-03-21T13:59:39.277 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.277+0000 7f9e74886640 1 -- 172.21.15.97:0/4063893569 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "pg_num": 12} v 0) v1 -- 0x7f9e70108600 con 0x7f9e70100f60 2024-03-21T13:59:39.278 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.278+0000 7f9e677fe640 1 -- 172.21.15.97:0/4063893569 <== mon.2 v2:172.21.15.97:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "pg_num": 12}]=0 pool 'b5a5e36b-8518-4a8b-80b9-4898af5bd2dd' already exists v776) v1 ==== 183+0+0 (secure 0 0 0) 0x7f9e580810e0 con 0x7f9e70100f60 2024-03-21T13:59:39.278 INFO:tasks.workunit.client.0.smithi097.stderr:pool 'b5a5e36b-8518-4a8b-80b9-4898af5bd2dd' already exists 2024-03-21T13:59:39.280 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.280+0000 7f9e74886640 1 -- 172.21.15.97:0/4063893569 >> v2:172.21.15.97:6800/12494470 conn(0x7f9e5c06e270 msgr2=0x7f9e5c070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:59:39.281 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.280+0000 7f9e74886640 1 --2- 172.21.15.97:0/4063893569 >> v2:172.21.15.97:6800/12494470 conn(0x7f9e5c06e270 0x7f9e5c070730 secure :-1 s=READY pgs=4029 cs=0 l=1 rev1=1 crypto rx=0x7f9e600095f0 tx=0x7f9e60005f90 comp rx=0 tx=0).stop 2024-03-21T13:59:39.281 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.280+0000 7f9e74886640 1 -- 172.21.15.97:0/4063893569 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9e70100f60 msgr2=0x7f9e701967a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:59:39.281 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.280+0000 7f9e74886640 1 --2- 172.21.15.97:0/4063893569 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9e70100f60 0x7f9e701967a0 secure :-1 s=READY pgs=3191 cs=0 l=1 rev1=1 crypto rx=0x7f9e58009230 tx=0x7f9e5802c780 comp rx=0 tx=0).stop 2024-03-21T13:59:39.281 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.280+0000 7f9e74886640 1 -- 172.21.15.97:0/4063893569 shutdown_connections 2024-03-21T13:59:39.281 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.280+0000 7f9e74886640 1 --2- 172.21.15.97:0/4063893569 >> v2:172.21.15.97:6800/12494470 conn(0x7f9e5c06e270 0x7f9e5c070730 unknown :-1 s=CLOSED pgs=4029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:59:39.281 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.280+0000 7f9e74886640 1 --2- 172.21.15.97:0/4063893569 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e70106660 0x7f9e70196ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:59:39.281 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.280+0000 7f9e74886640 1 --2- 172.21.15.97:0/4063893569 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9e70100f60 0x7f9e701967a0 unknown :-1 s=CLOSED pgs=3191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:59:39.281 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.280+0000 7f9e74886640 1 --2- 172.21.15.97:0/4063893569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e701005a0 0x7f9e70196260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:59:39.281 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.280+0000 7f9e74886640 1 -- 172.21.15.97:0/4063893569 >> 172.21.15.97:0/4063893569 conn(0x7f9e70077690 msgr2=0x7f9e700ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:59:39.281 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.280+0000 7f9e74886640 1 -- 172.21.15.97:0/4063893569 shutdown_connections 2024-03-21T13:59:39.281 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.280+0000 7f9e74886640 1 -- 172.21.15.97:0/4063893569 wait complete. 2024-03-21T13:59:39.290 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph osd pool set-quota b5a5e36b-8518-4a8b-80b9-4898af5bd2dd max_objects 10 2024-03-21T13:59:39.376 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.373+0000 7f704cd8c640 1 -- 172.21.15.97:0/1216053930 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7048100f40 msgr2=0x7f704810d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:59:39.376 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.373+0000 7f704cd8c640 1 --2- 172.21.15.97:0/1216053930 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7048100f40 0x7f704810d400 secure :-1 s=READY pgs=3192 cs=0 l=1 rev1=1 crypto rx=0x7f7040009cd0 tx=0x7f704002c0f0 comp rx=0 tx=0).stop 2024-03-21T13:59:39.376 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.373+0000 7f704cd8c640 1 -- 172.21.15.97:0/1216053930 shutdown_connections 2024-03-21T13:59:39.376 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.373+0000 7f704cd8c640 1 --2- 172.21.15.97:0/1216053930 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7048100f40 0x7f704810d400 unknown :-1 s=CLOSED pgs=3192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:59:39.376 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.373+0000 7f704cd8c640 1 --2- 172.21.15.97:0/1216053930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7048100580 0x7f7048100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:59:39.376 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.373+0000 7f704cd8c640 1 --2- 172.21.15.97:0/1216053930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7048106640 0x7f7048106a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:59:39.377 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.373+0000 7f704cd8c640 1 -- 172.21.15.97:0/1216053930 >> 172.21.15.97:0/1216053930 conn(0x7f70480776c0 msgr2=0x7f7048077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:59:39.377 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.373+0000 7f704cd8c640 1 -- 172.21.15.97:0/1216053930 shutdown_connections 2024-03-21T13:59:39.377 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.373+0000 7f704cd8c640 1 -- 172.21.15.97:0/1216053930 wait complete. 2024-03-21T13:59:39.377 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.374+0000 7f704cd8c640 1 Processor -- start 2024-03-21T13:59:39.377 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.374+0000 7f704cd8c640 1 -- start start 2024-03-21T13:59:39.377 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.374+0000 7f704cd8c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7048100580 0x7f7048197670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:59:39.377 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.374+0000 7f704cd8c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7048100f40 0x7f7048197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:59:39.377 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.374+0000 7f704cd8c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7048106640 0x7f704819bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:59:39.377 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.374+0000 7f704cd8c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7048110090 con 0x7f7048106640 2024-03-21T13:59:39.377 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.374+0000 7f704cd8c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f704810ff10 con 0x7f7048100f40 2024-03-21T13:59:39.377 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.374+0000 7f704cd8c640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7048110210 con 0x7f7048100580 2024-03-21T13:59:39.377 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.374+0000 7f704659c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7048100580 0x7f7048197670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:59:39.377 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.374+0000 7f704659c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7048100580 0x7f7048197670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.97:45628/0 (socket says 172.21.15.97:45628) 2024-03-21T13:59:39.378 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.374+0000 7f704659c640 1 -- 172.21.15.97:0/156714210 learned_addr learned my addr 172.21.15.97:0/156714210 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:59:39.378 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.374+0000 7f7046d9d640 1 --2- 172.21.15.97:0/156714210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7048106640 0x7f704819bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:59:39.378 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.374+0000 7f703ffff640 1 --2- 172.21.15.97:0/156714210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7048100f40 0x7f7048197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:59:39.378 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.375+0000 7f704659c640 1 -- 172.21.15.97:0/156714210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7048100f40 msgr2=0x7f7048197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:59:39.378 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.375+0000 7f704659c640 1 --2- 172.21.15.97:0/156714210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7048100f40 0x7f7048197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:59:39.378 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.375+0000 7f704659c640 1 -- 172.21.15.97:0/156714210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7048106640 msgr2=0x7f704819bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:59:39.378 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.375+0000 7f704659c640 1 --2- 172.21.15.97:0/156714210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7048106640 0x7f704819bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:59:39.378 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.375+0000 7f704659c640 1 -- 172.21.15.97:0/156714210 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f704819c660 con 0x7f7048100580 2024-03-21T13:59:39.378 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.375+0000 7f7046d9d640 1 --2- 172.21.15.97:0/156714210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7048106640 0x7f704819bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T13:59:39.378 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.375+0000 7f704659c640 1 --2- 172.21.15.97:0/156714210 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7048100580 0x7f7048197670 secure :-1 s=READY pgs=3193 cs=0 l=1 rev1=1 crypto rx=0x7f703800ea60 tx=0x7f703800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:59:39.378 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.375+0000 7f703f7fe640 1 -- 172.21.15.97:0/156714210 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703800cde0 con 0x7f7048100580 2024-03-21T13:59:39.378 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.375+0000 7f703f7fe640 1 -- 172.21.15.97:0/156714210 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f70380035f0 con 0x7f7048100580 2024-03-21T13:59:39.378 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.375+0000 7f704cd8c640 1 -- 172.21.15.97:0/156714210 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f704819c950 con 0x7f7048100580 2024-03-21T13:59:39.378 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.375+0000 7f703f7fe640 1 -- 172.21.15.97:0/156714210 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7038010690 con 0x7f7048100580 2024-03-21T13:59:39.379 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.375+0000 7f704cd8c640 1 -- 172.21.15.97:0/156714210 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70481a41e0 con 0x7f7048100580 2024-03-21T13:59:39.379 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.376+0000 7f704cd8c640 1 -- 172.21.15.97:0/156714210 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7048068580 con 0x7f7048100580 2024-03-21T13:59:39.379 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.376+0000 7f703f7fe640 1 -- 172.21.15.97:0/156714210 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 22) v1 ==== 92152+0+0 (secure 0 0 0) 0x7f7038004020 con 0x7f7048100580 2024-03-21T13:59:39.379 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.377+0000 7f703f7fe640 1 --2- 172.21.15.97:0/156714210 >> v2:172.21.15.97:6800/12494470 conn(0x7f703406e270 0x7f7034070730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:59:39.380 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.377+0000 7f703f7fe640 1 -- 172.21.15.97:0/156714210 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(776..776 src has 254..776) v4 ==== 7801+0+0 (secure 0 0 0) 0x7f703808ed50 con 0x7f7048100580 2024-03-21T13:59:39.380 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.377+0000 7f703ffff640 1 --2- 172.21.15.97:0/156714210 >> v2:172.21.15.97:6800/12494470 conn(0x7f703406e270 0x7f7034070730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:59:39.380 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.377+0000 7f703ffff640 1 --2- 172.21.15.97:0/156714210 >> v2:172.21.15.97:6800/12494470 conn(0x7f703406e270 0x7f7034070730 secure :-1 s=READY pgs=4030 cs=0 l=1 rev1=1 crypto rx=0x7f7048069d20 tx=0x7f7030008040 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:59:39.381 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.381+0000 7f703f7fe640 1 -- 172.21.15.97:0/156714210 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7038092030 con 0x7f7048100580 2024-03-21T13:59:39.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:39 smithi097 ceph-mon[39416]: osdmap e775: 8 total, 8 up, 8 in 2024-03-21T13:59:39.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:39 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4063893569' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "pg_num": 12}]: dispatch 2024-03-21T13:59:39.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:39 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "pg_num": 12}]: dispatch 2024-03-21T13:59:39.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:39 smithi097 ceph-mon[30040]: osdmap e775: 8 total, 8 up, 8 in 2024-03-21T13:59:39.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:39 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4063893569' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "pg_num": 12}]: dispatch 2024-03-21T13:59:39.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:39 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "pg_num": 12}]: dispatch 2024-03-21T13:59:39.560 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:39.560+0000 7f704cd8c640 1 -- 172.21.15.97:0/156714210 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7f70481a4410 con 0x7f7048100580 2024-03-21T13:59:39.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:39 smithi144 ceph-mon[33633]: osdmap e775: 8 total, 8 up, 8 in 2024-03-21T13:59:39.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:39 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4063893569' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "pg_num": 12}]: dispatch 2024-03-21T13:59:39.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:39 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "pg_num": 12}]: dispatch 2024-03-21T13:59:40.161 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:40.160+0000 7f703f7fe640 1 -- 172.21.15.97:0/156714210 <== mon.2 v2:172.21.15.97:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool b5a5e36b-8518-4a8b-80b9-4898af5bd2dd v777) v1 ==== 223+0+0 (secure 0 0 0) 0x7f703805bea0 con 0x7f7048100580 2024-03-21T13:59:40.275 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:40.275+0000 7f704cd8c640 1 -- 172.21.15.97:0/156714210 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7f70481a45e0 con 0x7f7048100580 2024-03-21T13:59:40.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:40 smithi097 ceph-mon[39416]: pgmap v1691: 164 pgs: 164 active+clean; 582 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 818 B/s rd, 0 op/s 2024-03-21T13:59:40.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:40 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "pg_num": 12}]': finished 2024-03-21T13:59:40.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:40 smithi097 ceph-mon[39416]: osdmap e776: 8 total, 8 up, 8 in 2024-03-21T13:59:40.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:40 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4063893569' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "pg_num": 12}]: dispatch 2024-03-21T13:59:40.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:40 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "pg_num": 12}]: dispatch 2024-03-21T13:59:40.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:40 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/156714210' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "10"}]: dispatch 2024-03-21T13:59:40.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:40 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "10"}]: dispatch 2024-03-21T13:59:40.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:40 smithi097 ceph-mon[30040]: pgmap v1691: 164 pgs: 164 active+clean; 582 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 818 B/s rd, 0 op/s 2024-03-21T13:59:40.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:40 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "pg_num": 12}]': finished 2024-03-21T13:59:40.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:40 smithi097 ceph-mon[30040]: osdmap e776: 8 total, 8 up, 8 in 2024-03-21T13:59:40.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:40 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4063893569' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "pg_num": 12}]: dispatch 2024-03-21T13:59:40.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:40 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "pg_num": 12}]: dispatch 2024-03-21T13:59:40.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:40 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/156714210' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "10"}]: dispatch 2024-03-21T13:59:40.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:40 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "10"}]: dispatch 2024-03-21T13:59:40.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:40 smithi144 ceph-mon[33633]: pgmap v1691: 164 pgs: 164 active+clean; 582 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 818 B/s rd, 0 op/s 2024-03-21T13:59:40.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:40 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "pg_num": 12}]': finished 2024-03-21T13:59:40.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:40 smithi144 ceph-mon[33633]: osdmap e776: 8 total, 8 up, 8 in 2024-03-21T13:59:40.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:40 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4063893569' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "pg_num": 12}]: dispatch 2024-03-21T13:59:40.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:40 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "pg_num": 12}]: dispatch 2024-03-21T13:59:40.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:40 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/156714210' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "10"}]: dispatch 2024-03-21T13:59:40.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:40 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "10"}]: dispatch 2024-03-21T13:59:41.176 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.175+0000 7f703f7fe640 1 -- 172.21.15.97:0/156714210 <== mon.2 v2:172.21.15.97:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool b5a5e36b-8518-4a8b-80b9-4898af5bd2dd v778) v1 ==== 223+0+0 (secure 0 0 0) 0x7f703805f8f0 con 0x7f7048100580 2024-03-21T13:59:41.176 INFO:tasks.workunit.client.0.smithi097.stderr:set-quota max_objects = 10 for pool b5a5e36b-8518-4a8b-80b9-4898af5bd2dd 2024-03-21T13:59:41.178 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.178+0000 7f704cd8c640 1 -- 172.21.15.97:0/156714210 >> v2:172.21.15.97:6800/12494470 conn(0x7f703406e270 msgr2=0x7f7034070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:59:41.178 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.178+0000 7f704cd8c640 1 --2- 172.21.15.97:0/156714210 >> v2:172.21.15.97:6800/12494470 conn(0x7f703406e270 0x7f7034070730 secure :-1 s=READY pgs=4030 cs=0 l=1 rev1=1 crypto rx=0x7f7048069d20 tx=0x7f7030008040 comp rx=0 tx=0).stop 2024-03-21T13:59:41.178 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.178+0000 7f704cd8c640 1 -- 172.21.15.97:0/156714210 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7048100580 msgr2=0x7f7048197670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:59:41.178 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.178+0000 7f704cd8c640 1 --2- 172.21.15.97:0/156714210 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7048100580 0x7f7048197670 secure :-1 s=READY pgs=3193 cs=0 l=1 rev1=1 crypto rx=0x7f703800ea60 tx=0x7f703800ef20 comp rx=0 tx=0).stop 2024-03-21T13:59:41.179 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.178+0000 7f704cd8c640 1 -- 172.21.15.97:0/156714210 shutdown_connections 2024-03-21T13:59:41.179 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.178+0000 7f704cd8c640 1 --2- 172.21.15.97:0/156714210 >> v2:172.21.15.97:6800/12494470 conn(0x7f703406e270 0x7f7034070730 unknown :-1 s=CLOSED pgs=4030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:59:41.179 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.178+0000 7f704cd8c640 1 --2- 172.21.15.97:0/156714210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7048106640 0x7f704819bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:59:41.179 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.178+0000 7f704cd8c640 1 --2- 172.21.15.97:0/156714210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7048100f40 0x7f7048197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:59:41.179 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.178+0000 7f704cd8c640 1 --2- 172.21.15.97:0/156714210 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7048100580 0x7f7048197670 unknown :-1 s=CLOSED pgs=3193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:59:41.179 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.178+0000 7f704cd8c640 1 -- 172.21.15.97:0/156714210 >> 172.21.15.97:0/156714210 conn(0x7f70480776c0 msgr2=0x7f70480689d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:59:41.179 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.178+0000 7f704cd8c640 1 -- 172.21.15.97:0/156714210 shutdown_connections 2024-03-21T13:59:41.179 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.178+0000 7f704cd8c640 1 -- 172.21.15.97:0/156714210 wait complete. 2024-03-21T13:59:41.190 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph osd pool application enable b5a5e36b-8518-4a8b-80b9-4898af5bd2dd rados 2024-03-21T13:59:41.279 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.276+0000 7f107753e640 1 -- 172.21.15.97:0/2504341157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1070103230 msgr2=0x7f107010b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:59:41.280 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.276+0000 7f107753e640 1 --2- 172.21.15.97:0/2504341157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1070103230 0x7f107010b710 secure :-1 s=READY pgs=2995 cs=0 l=1 rev1=1 crypto rx=0x7f1060009900 tx=0x7f106002bd60 comp rx=0 tx=0).stop 2024-03-21T13:59:41.280 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.276+0000 7f107753e640 1 -- 172.21.15.97:0/2504341157 shutdown_connections 2024-03-21T13:59:41.280 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.276+0000 7f107753e640 1 --2- 172.21.15.97:0/2504341157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1070103230 0x7f107010b710 unknown :-1 s=CLOSED pgs=2995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:59:41.280 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.277+0000 7f107753e640 1 --2- 172.21.15.97:0/2504341157 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f107010bc50 0x7f107010e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:59:41.280 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.277+0000 7f107753e640 1 --2- 172.21.15.97:0/2504341157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1070102910 0x7f1070102cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:59:41.280 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.277+0000 7f107753e640 1 -- 172.21.15.97:0/2504341157 >> 172.21.15.97:0/2504341157 conn(0x7f10700776c0 msgr2=0x7f1070077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:59:41.280 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.277+0000 7f107753e640 1 -- 172.21.15.97:0/2504341157 shutdown_connections 2024-03-21T13:59:41.280 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.277+0000 7f107753e640 1 -- 172.21.15.97:0/2504341157 wait complete. 2024-03-21T13:59:41.280 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.277+0000 7f107753e640 1 Processor -- start 2024-03-21T13:59:41.280 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.277+0000 7f107753e640 1 -- start start 2024-03-21T13:59:41.280 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.277+0000 7f107753e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1070102910 0x7f1070197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:59:41.280 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.277+0000 7f107753e640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1070103230 0x7f1070197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:59:41.281 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.277+0000 7f107753e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f107010bc50 0x7f107019bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:59:41.281 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.277+0000 7f107753e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1070111160 con 0x7f1070102910 2024-03-21T13:59:41.281 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.277+0000 7f107753e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1070110fe0 con 0x7f107010bc50 2024-03-21T13:59:41.281 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.277+0000 7f107753e640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10701112e0 con 0x7f1070103230 2024-03-21T13:59:41.281 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.277+0000 7f1074ad9640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1070103230 0x7f1070197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:59:41.281 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.277+0000 7f10752da640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1070102910 0x7f1070197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:59:41.281 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.278+0000 7f1074ad9640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1070103230 0x7f1070197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.97:60696/0 (socket says 172.21.15.97:60696) 2024-03-21T13:59:41.281 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.278+0000 7f10752da640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1070102910 0x7f1070197620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48998/0 (socket says 172.21.15.97:48998) 2024-03-21T13:59:41.281 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.278+0000 7f1074ad9640 1 -- 172.21.15.97:0/179779542 learned_addr learned my addr 172.21.15.97:0/179779542 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T13:59:41.281 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.278+0000 7f1074ad9640 1 -- 172.21.15.97:0/179779542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f107010bc50 msgr2=0x7f107019bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:59:41.281 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.278+0000 7f1075adb640 1 --2- 172.21.15.97:0/179779542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f107010bc50 0x7f107019bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:59:41.281 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.278+0000 7f1074ad9640 1 --2- 172.21.15.97:0/179779542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f107010bc50 0x7f107019bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:59:41.281 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.278+0000 7f1074ad9640 1 -- 172.21.15.97:0/179779542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1070102910 msgr2=0x7f1070197620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:59:41.281 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.278+0000 7f1074ad9640 1 --2- 172.21.15.97:0/179779542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1070102910 0x7f1070197620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:59:41.282 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.278+0000 7f1074ad9640 1 -- 172.21.15.97:0/179779542 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f107019c610 con 0x7f1070103230 2024-03-21T13:59:41.282 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.278+0000 7f10752da640 1 --2- 172.21.15.97:0/179779542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1070102910 0x7f1070197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T13:59:41.282 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.278+0000 7f1074ad9640 1 --2- 172.21.15.97:0/179779542 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1070103230 0x7f1070197b60 secure :-1 s=READY pgs=3194 cs=0 l=1 rev1=1 crypto rx=0x7f1060005940 tx=0x7f106002c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:59:41.282 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.278+0000 7f10667fc640 1 -- 172.21.15.97:0/179779542 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1060043020 con 0x7f1070103230 2024-03-21T13:59:41.282 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.278+0000 7f107753e640 1 -- 172.21.15.97:0/179779542 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f107019c8a0 con 0x7f1070103230 2024-03-21T13:59:41.282 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.278+0000 7f10667fc640 1 -- 172.21.15.97:0/179779542 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f10600043d0 con 0x7f1070103230 2024-03-21T13:59:41.282 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.278+0000 7f107753e640 1 -- 172.21.15.97:0/179779542 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10701a4190 con 0x7f1070103230 2024-03-21T13:59:41.282 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.278+0000 7f10667fc640 1 -- 172.21.15.97:0/179779542 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1060003ae0 con 0x7f1070103230 2024-03-21T13:59:41.282 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.279+0000 7f107753e640 1 -- 172.21.15.97:0/179779542 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1070068580 con 0x7f1070103230 2024-03-21T13:59:41.282 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.280+0000 7f10667fc640 1 -- 172.21.15.97:0/179779542 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 22) v1 ==== 92152+0+0 (secure 0 0 0) 0x7f1060046020 con 0x7f1070103230 2024-03-21T13:59:41.283 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.280+0000 7f10667fc640 1 --2- 172.21.15.97:0/179779542 >> v2:172.21.15.97:6800/12494470 conn(0x7f105c06e270 0x7f105c070730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T13:59:41.283 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.280+0000 7f10752da640 1 --2- 172.21.15.97:0/179779542 >> v2:172.21.15.97:6800/12494470 conn(0x7f105c06e270 0x7f105c070730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T13:59:41.283 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.280+0000 7f10667fc640 1 -- 172.21.15.97:0/179779542 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(778..778 src has 254..778) v4 ==== 7801+0+0 (secure 0 0 0) 0x7f10600b0c20 con 0x7f1070103230 2024-03-21T13:59:41.283 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.280+0000 7f10752da640 1 --2- 172.21.15.97:0/179779542 >> v2:172.21.15.97:6800/12494470 conn(0x7f105c06e270 0x7f105c070730 secure :-1 s=READY pgs=4031 cs=0 l=1 rev1=1 crypto rx=0x7f106c00f650 tx=0x7f106c0075b0 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T13:59:41.285 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.285+0000 7f10667fc640 1 -- 172.21.15.97:0/179779542 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f106004d020 con 0x7f1070103230 2024-03-21T13:59:41.465 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:41.465+0000 7f107753e640 1 -- 172.21.15.97:0/179779542 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "app": "rados"} v 0) v1 -- 0x7f1070106dd0 con 0x7f1070103230 2024-03-21T13:59:41.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:41 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "10"}]': finished 2024-03-21T13:59:41.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:41 smithi097 ceph-mon[30040]: osdmap e777: 8 total, 8 up, 8 in 2024-03-21T13:59:41.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:41 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/156714210' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "10"}]: dispatch 2024-03-21T13:59:41.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:41 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "10"}]: dispatch 2024-03-21T13:59:41.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:41 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "10"}]': finished 2024-03-21T13:59:41.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:41 smithi097 ceph-mon[39416]: osdmap e777: 8 total, 8 up, 8 in 2024-03-21T13:59:41.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:41 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/156714210' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "10"}]: dispatch 2024-03-21T13:59:41.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:41 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "10"}]: dispatch 2024-03-21T13:59:41.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:41 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "10"}]': finished 2024-03-21T13:59:41.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:41 smithi144 ceph-mon[33633]: osdmap e777: 8 total, 8 up, 8 in 2024-03-21T13:59:41.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:41 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/156714210' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "10"}]: dispatch 2024-03-21T13:59:41.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:41 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "10"}]: dispatch 2024-03-21T13:59:42.180 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:42.179+0000 7f10667fc640 1 -- 172.21.15.97:0/179779542 <== mon.2 v2:172.21.15.97:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "app": "rados"}]=0 enabled application 'rados' on pool 'b5a5e36b-8518-4a8b-80b9-4898af5bd2dd' v779) v1 ==== 213+0+0 (secure 0 0 0) 0x7f106007dd70 con 0x7f1070103230 2024-03-21T13:59:42.289 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:42.289+0000 7f107753e640 1 -- 172.21.15.97:0/179779542 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "app": "rados"} v 0) v1 -- 0x7f1070106840 con 0x7f1070103230 2024-03-21T13:59:42.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:59:41 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:59:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:59:42.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:42 smithi097 ceph-mon[39416]: pgmap v1694: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-03-21T13:59:42.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:42 smithi097 ceph-mon[39416]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:59:42.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:42 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "10"}]': finished 2024-03-21T13:59:42.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:42 smithi097 ceph-mon[39416]: osdmap e778: 8 total, 8 up, 8 in 2024-03-21T13:59:42.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:42 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/179779542' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "app": "rados"}]: dispatch 2024-03-21T13:59:42.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:42 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "app": "rados"}]: dispatch 2024-03-21T13:59:42.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:42 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:59:42.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:42 smithi097 ceph-mon[30040]: pgmap v1694: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-03-21T13:59:42.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:42 smithi097 ceph-mon[30040]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:59:42.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:42 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "10"}]': finished 2024-03-21T13:59:42.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:42 smithi097 ceph-mon[30040]: osdmap e778: 8 total, 8 up, 8 in 2024-03-21T13:59:42.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:42 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/179779542' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "app": "rados"}]: dispatch 2024-03-21T13:59:42.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:42 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "app": "rados"}]: dispatch 2024-03-21T13:59:42.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:42 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:59:42.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:42 smithi144 ceph-mon[33633]: pgmap v1694: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-03-21T13:59:42.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:42 smithi144 ceph-mon[33633]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:59:42.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:42 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "10"}]': finished 2024-03-21T13:59:42.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:42 smithi144 ceph-mon[33633]: osdmap e778: 8 total, 8 up, 8 in 2024-03-21T13:59:42.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:42 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/179779542' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "app": "rados"}]: dispatch 2024-03-21T13:59:42.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:42 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "app": "rados"}]: dispatch 2024-03-21T13:59:42.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:42 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:59:43.189 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:43.188+0000 7f10667fc640 1 -- 172.21.15.97:0/179779542 <== mon.2 v2:172.21.15.97:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "app": "rados"}]=0 enabled application 'rados' on pool 'b5a5e36b-8518-4a8b-80b9-4898af5bd2dd' v780) v1 ==== 213+0+0 (secure 0 0 0) 0x7f10600817c0 con 0x7f1070103230 2024-03-21T13:59:43.189 INFO:tasks.workunit.client.0.smithi097.stderr:enabled application 'rados' on pool 'b5a5e36b-8518-4a8b-80b9-4898af5bd2dd' 2024-03-21T13:59:43.191 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:43.191+0000 7f107753e640 1 -- 172.21.15.97:0/179779542 >> v2:172.21.15.97:6800/12494470 conn(0x7f105c06e270 msgr2=0x7f105c070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:59:43.192 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:43.191+0000 7f107753e640 1 --2- 172.21.15.97:0/179779542 >> v2:172.21.15.97:6800/12494470 conn(0x7f105c06e270 0x7f105c070730 secure :-1 s=READY pgs=4031 cs=0 l=1 rev1=1 crypto rx=0x7f106c00f650 tx=0x7f106c0075b0 comp rx=0 tx=0).stop 2024-03-21T13:59:43.192 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:43.191+0000 7f107753e640 1 -- 172.21.15.97:0/179779542 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1070103230 msgr2=0x7f1070197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:59:43.192 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:43.191+0000 7f107753e640 1 --2- 172.21.15.97:0/179779542 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1070103230 0x7f1070197b60 secure :-1 s=READY pgs=3194 cs=0 l=1 rev1=1 crypto rx=0x7f1060005940 tx=0x7f106002c900 comp rx=0 tx=0).stop 2024-03-21T13:59:43.192 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:43.191+0000 7f107753e640 1 -- 172.21.15.97:0/179779542 shutdown_connections 2024-03-21T13:59:43.192 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:43.191+0000 7f107753e640 1 --2- 172.21.15.97:0/179779542 >> v2:172.21.15.97:6800/12494470 conn(0x7f105c06e270 0x7f105c070730 unknown :-1 s=CLOSED pgs=4031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:59:43.192 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:43.191+0000 7f107753e640 1 --2- 172.21.15.97:0/179779542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f107010bc50 0x7f107019bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:59:43.192 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:43.191+0000 7f107753e640 1 --2- 172.21.15.97:0/179779542 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1070103230 0x7f1070197b60 unknown :-1 s=CLOSED pgs=3194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:59:43.192 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:43.191+0000 7f107753e640 1 --2- 172.21.15.97:0/179779542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1070102910 0x7f1070197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T13:59:43.192 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:43.191+0000 7f107753e640 1 -- 172.21.15.97:0/179779542 >> 172.21.15.97:0/179779542 conn(0x7f10700776c0 msgr2=0x7f10700fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:59:43.192 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:43.191+0000 7f107753e640 1 -- 172.21.15.97:0/179779542 shutdown_connections 2024-03-21T13:59:43.192 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T13:59:43.191+0000 7f107753e640 1 -- 172.21.15.97:0/179779542 wait complete. 2024-03-21T13:59:43.203 INFO:tasks.workunit.client.0.smithi097.stderr:++ seq 1 10 2024-03-21T13:59:43.215 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in `seq 1 10` 2024-03-21T13:59:43.215 INFO:tasks.workunit.client.0.smithi097.stderr:+ rados -p b5a5e36b-8518-4a8b-80b9-4898af5bd2dd put obj1 /etc/passwd 2024-03-21T13:59:43.254 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in `seq 1 10` 2024-03-21T13:59:43.254 INFO:tasks.workunit.client.0.smithi097.stderr:+ rados -p b5a5e36b-8518-4a8b-80b9-4898af5bd2dd put obj2 /etc/passwd 2024-03-21T13:59:43.291 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in `seq 1 10` 2024-03-21T13:59:43.292 INFO:tasks.workunit.client.0.smithi097.stderr:+ rados -p b5a5e36b-8518-4a8b-80b9-4898af5bd2dd put obj3 /etc/passwd 2024-03-21T13:59:43.328 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in `seq 1 10` 2024-03-21T13:59:43.329 INFO:tasks.workunit.client.0.smithi097.stderr:+ rados -p b5a5e36b-8518-4a8b-80b9-4898af5bd2dd put obj4 /etc/passwd 2024-03-21T13:59:43.364 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in `seq 1 10` 2024-03-21T13:59:43.364 INFO:tasks.workunit.client.0.smithi097.stderr:+ rados -p b5a5e36b-8518-4a8b-80b9-4898af5bd2dd put obj5 /etc/passwd 2024-03-21T13:59:43.398 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in `seq 1 10` 2024-03-21T13:59:43.399 INFO:tasks.workunit.client.0.smithi097.stderr:+ rados -p b5a5e36b-8518-4a8b-80b9-4898af5bd2dd put obj6 /etc/passwd 2024-03-21T13:59:43.433 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in `seq 1 10` 2024-03-21T13:59:43.434 INFO:tasks.workunit.client.0.smithi097.stderr:+ rados -p b5a5e36b-8518-4a8b-80b9-4898af5bd2dd put obj7 /etc/passwd 2024-03-21T13:59:43.468 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in `seq 1 10` 2024-03-21T13:59:43.468 INFO:tasks.workunit.client.0.smithi097.stderr:+ rados -p b5a5e36b-8518-4a8b-80b9-4898af5bd2dd put obj8 /etc/passwd 2024-03-21T13:59:43.502 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in `seq 1 10` 2024-03-21T13:59:43.502 INFO:tasks.workunit.client.0.smithi097.stderr:+ rados -p b5a5e36b-8518-4a8b-80b9-4898af5bd2dd put obj9 /etc/passwd 2024-03-21T13:59:43.537 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in `seq 1 10` 2024-03-21T13:59:43.538 INFO:tasks.workunit.client.0.smithi097.stderr:+ rados -p b5a5e36b-8518-4a8b-80b9-4898af5bd2dd put obj10 /etc/passwd 2024-03-21T13:59:43.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:43 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "app": "rados"}]': finished 2024-03-21T13:59:43.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:43 smithi097 ceph-mon[30040]: osdmap e779: 8 total, 8 up, 8 in 2024-03-21T13:59:43.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:43 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/179779542' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "app": "rados"}]: dispatch 2024-03-21T13:59:43.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:43 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "app": "rados"}]: dispatch 2024-03-21T13:59:43.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:43 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "app": "rados"}]': finished 2024-03-21T13:59:43.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:43 smithi097 ceph-mon[39416]: osdmap e779: 8 total, 8 up, 8 in 2024-03-21T13:59:43.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:43 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/179779542' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "app": "rados"}]: dispatch 2024-03-21T13:59:43.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:43 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "app": "rados"}]: dispatch 2024-03-21T13:59:43.571 INFO:tasks.workunit.client.0.smithi097.stderr:+ sleep 30 2024-03-21T13:59:43.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:43 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "app": "rados"}]': finished 2024-03-21T13:59:43.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:43 smithi144 ceph-mon[33633]: osdmap e779: 8 total, 8 up, 8 in 2024-03-21T13:59:43.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:43 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/179779542' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "app": "rados"}]: dispatch 2024-03-21T13:59:43.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:43 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "app": "rados"}]: dispatch 2024-03-21T13:59:44.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:44 smithi097 ceph-mon[39416]: pgmap v1697: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 899 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:59:44.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:44 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "app": "rados"}]': finished 2024-03-21T13:59:44.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:44 smithi097 ceph-mon[39416]: osdmap e780: 8 total, 8 up, 8 in 2024-03-21T13:59:44.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:44 smithi097 ceph-mon[30040]: pgmap v1697: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 899 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:59:44.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:44 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "app": "rados"}]': finished 2024-03-21T13:59:44.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:44 smithi097 ceph-mon[30040]: osdmap e780: 8 total, 8 up, 8 in 2024-03-21T13:59:44.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:44 smithi144 ceph-mon[33633]: pgmap v1697: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 899 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:59:44.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:44 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "app": "rados"}]': finished 2024-03-21T13:59:44.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:44 smithi144 ceph-mon[33633]: osdmap e780: 8 total, 8 up, 8 in 2024-03-21T13:59:45.371 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:59:45 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:59:46.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:46 smithi097 ceph-mon[39416]: pgmap v1699: 176 pgs: 176 active+clean; 605 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 4.9 KiB/s wr, 1 op/s 2024-03-21T13:59:46.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:46 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:59:46.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:46 smithi097 ceph-mon[30040]: pgmap v1699: 176 pgs: 176 active+clean; 605 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 4.9 KiB/s wr, 1 op/s 2024-03-21T13:59:46.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:46 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:59:46.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:46 smithi144 ceph-mon[33633]: pgmap v1699: 176 pgs: 176 active+clean; 605 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 4.9 KiB/s wr, 1 op/s 2024-03-21T13:59:46.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:46 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:59:48.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:48 smithi097 ceph-mon[39416]: pgmap v1700: 176 pgs: 176 active+clean; 605 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-03-21T13:59:48.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:48 smithi097 ceph-mon[39416]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:59:48.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:48 smithi097 ceph-mon[30040]: pgmap v1700: 176 pgs: 176 active+clean; 605 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-03-21T13:59:48.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:48 smithi097 ceph-mon[30040]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:59:48.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:48 smithi144 ceph-mon[33633]: pgmap v1700: 176 pgs: 176 active+clean; 605 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-03-21T13:59:48.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:48 smithi144 ceph-mon[33633]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:59:50.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:50 smithi097 ceph-mon[39416]: pgmap v1701: 176 pgs: 176 active+clean; 698 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 15 KiB/s wr, 2 op/s 2024-03-21T13:59:50.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:50 smithi097 ceph-mon[30040]: pgmap v1701: 176 pgs: 176 active+clean; 698 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 15 KiB/s wr, 2 op/s 2024-03-21T13:59:50.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:50 smithi144 ceph-mon[33633]: pgmap v1701: 176 pgs: 176 active+clean; 698 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 15 KiB/s wr, 2 op/s 2024-03-21T13:59:52.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:52 smithi097 ceph-mon[30040]: pgmap v1702: 176 pgs: 176 active+clean; 698 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 13 KiB/s wr, 2 op/s 2024-03-21T13:59:52.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:52 smithi097 ceph-mon[39416]: pgmap v1702: 176 pgs: 176 active+clean; 698 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 13 KiB/s wr, 2 op/s 2024-03-21T13:59:52.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 13:59:51 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:13:59:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:59:52.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:52 smithi144 ceph-mon[33633]: pgmap v1702: 176 pgs: 176 active+clean; 698 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 13 KiB/s wr, 2 op/s 2024-03-21T13:59:53.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:53 smithi097 ceph-mon[39416]: pool 'b5a5e36b-8518-4a8b-80b9-4898af5bd2dd' is full (reached quota's max_objects: 10) 2024-03-21T13:59:53.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:53 smithi097 ceph-mon[39416]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-21T13:59:53.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:53 smithi097 ceph-mon[39416]: osdmap e781: 8 total, 8 up, 8 in 2024-03-21T13:59:53.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:53 smithi097 ceph-mon[30040]: pool 'b5a5e36b-8518-4a8b-80b9-4898af5bd2dd' is full (reached quota's max_objects: 10) 2024-03-21T13:59:53.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:53 smithi097 ceph-mon[30040]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-21T13:59:53.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:53 smithi097 ceph-mon[30040]: osdmap e781: 8 total, 8 up, 8 in 2024-03-21T13:59:53.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:53 smithi144 ceph-mon[33633]: pool 'b5a5e36b-8518-4a8b-80b9-4898af5bd2dd' is full (reached quota's max_objects: 10) 2024-03-21T13:59:53.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:53 smithi144 ceph-mon[33633]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-21T13:59:53.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:53 smithi144 ceph-mon[33633]: osdmap e781: 8 total, 8 up, 8 in 2024-03-21T13:59:54.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:54 smithi097 ceph-mon[39416]: pgmap v1704: 176 pgs: 176 active+clean; 698 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 12 KiB/s wr, 2 op/s 2024-03-21T13:59:54.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:54 smithi097 ceph-mon[30040]: pgmap v1704: 176 pgs: 176 active+clean; 698 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 12 KiB/s wr, 2 op/s 2024-03-21T13:59:54.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:54 smithi144 ceph-mon[33633]: pgmap v1704: 176 pgs: 176 active+clean; 698 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 12 KiB/s wr, 2 op/s 2024-03-21T13:59:55.371 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 13:59:55 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T13:59:56.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:56 smithi097 ceph-mon[30040]: pgmap v1705: 176 pgs: 176 active+clean; 698 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 9.6 KiB/s wr, 1 op/s 2024-03-21T13:59:56.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:56 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:59:56.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:56 smithi097 ceph-mon[39416]: pgmap v1705: 176 pgs: 176 active+clean; 698 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 9.6 KiB/s wr, 1 op/s 2024-03-21T13:59:56.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:56 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:59:56.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:56 smithi144 ceph-mon[33633]: pgmap v1705: 176 pgs: 176 active+clean; 698 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 9.6 KiB/s wr, 1 op/s 2024-03-21T13:59:56.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:56 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:59:58.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:57 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:59:58.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:57 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:59:58.053 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 13:59:57 smithi097 ceph-mon[30040]: pgmap v1706: 176 pgs: 176 active+clean; 698 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 9.6 KiB/s wr, 1 op/s 2024-03-21T13:59:58.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:57 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:59:58.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:57 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:59:58.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 13:59:57 smithi097 ceph-mon[39416]: pgmap v1706: 176 pgs: 176 active+clean; 698 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 9.6 KiB/s wr, 1 op/s 2024-03-21T13:59:58.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:57 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T13:59:58.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:57 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:59:58.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 13:59:57 smithi144 ceph-mon[33633]: pgmap v1706: 176 pgs: 176 active+clean; 698 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 9.6 KiB/s wr, 1 op/s 2024-03-21T14:00:00.481 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:00 smithi144 ceph-mon[33633]: pgmap v1707: 176 pgs: 176 active+clean; 698 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T14:00:00.482 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:00 smithi144 ceph-mon[33633]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled; 1 pool(s) full 2024-03-21T14:00:00.482 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:00 smithi144 ceph-mon[33633]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-03-21T14:00:00.482 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:00 smithi144 ceph-mon[33633]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-03-21T14:00:00.482 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:00 smithi144 ceph-mon[33633]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-03-21T14:00:00.482 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:00 smithi144 ceph-mon[33633]: [WRN] POOL_FULL: 1 pool(s) full 2024-03-21T14:00:00.482 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:00 smithi144 ceph-mon[33633]: pool 'b5a5e36b-8518-4a8b-80b9-4898af5bd2dd' is full (running out of quota) 2024-03-21T14:00:00.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:00 smithi097 ceph-mon[39416]: pgmap v1707: 176 pgs: 176 active+clean; 698 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T14:00:00.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:00 smithi097 ceph-mon[39416]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled; 1 pool(s) full 2024-03-21T14:00:00.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:00 smithi097 ceph-mon[39416]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-03-21T14:00:00.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:00 smithi097 ceph-mon[39416]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-03-21T14:00:00.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:00 smithi097 ceph-mon[39416]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-03-21T14:00:00.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:00 smithi097 ceph-mon[39416]: [WRN] POOL_FULL: 1 pool(s) full 2024-03-21T14:00:00.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:00 smithi097 ceph-mon[39416]: pool 'b5a5e36b-8518-4a8b-80b9-4898af5bd2dd' is full (running out of quota) 2024-03-21T14:00:00.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:00 smithi097 ceph-mon[30040]: pgmap v1707: 176 pgs: 176 active+clean; 698 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T14:00:00.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:00 smithi097 ceph-mon[30040]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled; 1 pool(s) full 2024-03-21T14:00:00.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:00 smithi097 ceph-mon[30040]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-03-21T14:00:00.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:00 smithi097 ceph-mon[30040]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-03-21T14:00:00.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:00 smithi097 ceph-mon[30040]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-03-21T14:00:00.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:00 smithi097 ceph-mon[30040]: [WRN] POOL_FULL: 1 pool(s) full 2024-03-21T14:00:00.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:00 smithi097 ceph-mon[30040]: pool 'b5a5e36b-8518-4a8b-80b9-4898af5bd2dd' is full (running out of quota) 2024-03-21T14:00:02.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:02 smithi097 ceph-mon[39416]: pgmap v1708: 176 pgs: 176 active+clean; 698 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T14:00:02.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:02 smithi097 ceph-mon[30040]: pgmap v1708: 176 pgs: 176 active+clean; 698 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T14:00:02.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 14:00:01 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:14:00:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T14:00:02.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:02 smithi144 ceph-mon[33633]: pgmap v1708: 176 pgs: 176 active+clean; 698 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T14:00:04.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:04 smithi097 ceph-mon[39416]: pgmap v1709: 176 pgs: 176 active+clean; 698 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 988 B/s rd, 0 op/s 2024-03-21T14:00:04.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:04 smithi097 ceph-mon[30040]: pgmap v1709: 176 pgs: 176 active+clean; 698 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 988 B/s rd, 0 op/s 2024-03-21T14:00:04.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:04 smithi144 ceph-mon[33633]: pgmap v1709: 176 pgs: 176 active+clean; 698 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 988 B/s rd, 0 op/s 2024-03-21T14:00:05.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 14:00:05 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T14:00:06.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:06 smithi097 ceph-mon[30040]: pgmap v1710: 176 pgs: 176 active+clean; 698 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T14:00:06.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:06 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T14:00:06.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:06 smithi097 ceph-mon[39416]: pgmap v1710: 176 pgs: 176 active+clean; 698 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T14:00:06.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:06 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T14:00:06.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:06 smithi144 ceph-mon[33633]: pgmap v1710: 176 pgs: 176 active+clean; 698 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T14:00:06.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:06 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T14:00:08.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:08 smithi097 ceph-mon[39416]: pgmap v1711: 176 pgs: 176 active+clean; 698 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T14:00:08.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:08 smithi097 ceph-mon[30040]: pgmap v1711: 176 pgs: 176 active+clean; 698 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T14:00:08.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:08 smithi144 ceph-mon[33633]: pgmap v1711: 176 pgs: 176 active+clean; 698 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T14:00:10.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:10 smithi097 ceph-mon[30040]: pgmap v1712: 176 pgs: 176 active+clean; 698 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T14:00:10.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:10 smithi097 ceph-mon[39416]: pgmap v1712: 176 pgs: 176 active+clean; 698 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T14:00:10.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:10 smithi144 ceph-mon[33633]: pgmap v1712: 176 pgs: 176 active+clean; 698 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T14:00:12.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 14:00:11 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:14:00:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T14:00:12.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:12 smithi097 ceph-mon[39416]: pgmap v1713: 176 pgs: 176 active+clean; 698 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:00:12.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:12 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T14:00:12.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:12 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T14:00:12.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:12 smithi097 ceph-mon[30040]: pgmap v1713: 176 pgs: 176 active+clean; 698 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:00:12.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:12 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T14:00:12.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:12 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T14:00:12.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:12 smithi144 ceph-mon[33633]: pgmap v1713: 176 pgs: 176 active+clean; 698 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:00:12.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:12 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T14:00:12.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:12 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T14:00:13.573 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=156148 2024-03-21T14:00:13.573 INFO:tasks.workunit.client.0.smithi097.stderr:+ rados -p b5a5e36b-8518-4a8b-80b9-4898af5bd2dd put onemore /etc/passwd 2024-03-21T14:00:13.573 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph osd pool set-quota b5a5e36b-8518-4a8b-80b9-4898af5bd2dd max_objects 100 2024-03-21T14:00:13.671 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:13.667+0000 7fee86ae1640 1 -- 172.21.15.97:0/1179503611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee80100580 msgr2=0x7fee80100a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:00:13.671 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:13.667+0000 7fee86ae1640 1 --2- 172.21.15.97:0/1179503611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee80100580 0x7fee80100a00 secure :-1 s=READY pgs=3008 cs=0 l=1 rev1=1 crypto rx=0x7fee6c009900 tx=0x7fee6c02bc30 comp rx=0 tx=0).stop 2024-03-21T14:00:13.671 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:13.667+0000 7fee86ae1640 1 -- 172.21.15.97:0/1179503611 shutdown_connections 2024-03-21T14:00:13.671 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:13.667+0000 7fee86ae1640 1 --2- 172.21.15.97:0/1179503611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee80100580 0x7fee80100a00 unknown :-1 s=CLOSED pgs=3008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:13.671 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:13.667+0000 7fee86ae1640 1 --2- 172.21.15.97:0/1179503611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee80100f40 0x7fee8010d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:13.671 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:13.667+0000 7fee86ae1640 1 --2- 172.21.15.97:0/1179503611 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fee80106640 0x7fee80106a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:13.672 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:13.668+0000 7fee86ae1640 1 -- 172.21.15.97:0/1179503611 >> 172.21.15.97:0/1179503611 conn(0x7fee800776c0 msgr2=0x7fee80077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T14:00:13.672 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:13.668+0000 7fee86ae1640 1 -- 172.21.15.97:0/1179503611 shutdown_connections 2024-03-21T14:00:13.672 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:13.668+0000 7fee86ae1640 1 -- 172.21.15.97:0/1179503611 wait complete. 2024-03-21T14:00:13.672 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:13.668+0000 7fee86ae1640 1 Processor -- start 2024-03-21T14:00:13.673 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:13.668+0000 7fee86ae1640 1 -- start start 2024-03-21T14:00:13.673 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:13.668+0000 7fee86ae1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee80100580 0x7fee80198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T14:00:13.673 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:13.668+0000 7fee86ae1640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fee80100f40 0x7fee801989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T14:00:13.673 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:13.668+0000 7fee86ae1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee80106640 0x7fee801a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T14:00:13.673 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:13.668+0000 7fee86ae1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee8010fff0 con 0x7fee80100580 2024-03-21T14:00:13.673 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:13.668+0000 7fee86ae1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee8010fe70 con 0x7fee80106640 2024-03-21T14:00:13.673 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:13.668+0000 7fee86ae1640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee80110170 con 0x7fee80100f40 2024-03-21T14:00:13.673 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:13.669+0000 7fee7ffff640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fee80100f40 0x7fee801989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T14:00:13.673 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:13.669+0000 7fee8487d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee80100580 0x7fee80198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T14:00:13.673 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:13.669+0000 7fee7ffff640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fee80100f40 0x7fee801989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.97:35682/0 (socket says 172.21.15.97:35682) 2024-03-21T14:00:13.673 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:13.669+0000 7fee8487d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee80100580 0x7fee80198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54554/0 (socket says 172.21.15.97:54554) 2024-03-21T14:00:13.673 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:13.669+0000 7fee7ffff640 1 -- 172.21.15.97:0/48265811 learned_addr learned my addr 172.21.15.97:0/48265811 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T14:00:13.674 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:13.669+0000 7fee8507e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee80106640 0x7fee801a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T14:00:13.674 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:13.669+0000 7fee8487d640 1 -- 172.21.15.97:0/48265811 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fee80100f40 msgr2=0x7fee801989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:00:13.674 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:13.669+0000 7fee8487d640 1 --2- 172.21.15.97:0/48265811 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fee80100f40 0x7fee801989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:13.674 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:13.669+0000 7fee8487d640 1 -- 172.21.15.97:0/48265811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee80106640 msgr2=0x7fee801a34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:00:13.674 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:13.669+0000 7fee8487d640 1 --2- 172.21.15.97:0/48265811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee80106640 0x7fee801a34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:13.674 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:13.669+0000 7fee8487d640 1 -- 172.21.15.97:0/48265811 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee801a3a20 con 0x7fee80100580 2024-03-21T14:00:13.674 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:13.669+0000 7fee7ffff640 1 --2- 172.21.15.97:0/48265811 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fee80100f40 0x7fee801989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T14:00:13.674 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:13.669+0000 7fee8507e640 1 --2- 172.21.15.97:0/48265811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee80106640 0x7fee801a34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T14:00:13.674 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:13.669+0000 7fee8487d640 1 --2- 172.21.15.97:0/48265811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee80100580 0x7fee80198460 secure :-1 s=READY pgs=3009 cs=0 l=1 rev1=1 crypto rx=0x7fee7400e9f0 tx=0x7fee7400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T14:00:13.674 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:13.669+0000 7fee7dffb640 1 -- 172.21.15.97:0/48265811 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7400cd80 con 0x7fee80100580 2024-03-21T14:00:13.674 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:13.669+0000 7fee86ae1640 1 -- 172.21.15.97:0/48265811 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee801a3bf0 con 0x7fee80100580 2024-03-21T14:00:13.675 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:13.669+0000 7fee7dffb640 1 -- 172.21.15.97:0/48265811 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fee740035f0 con 0x7fee80100580 2024-03-21T14:00:13.675 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:13.669+0000 7fee7dffb640 1 -- 172.21.15.97:0/48265811 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee74010650 con 0x7fee80100580 2024-03-21T14:00:13.675 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:13.669+0000 7fee86ae1640 1 -- 172.21.15.97:0/48265811 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee801a4100 con 0x7fee80100580 2024-03-21T14:00:13.675 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:13.670+0000 7fee86ae1640 1 -- 172.21.15.97:0/48265811 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee80068580 con 0x7fee80100580 2024-03-21T14:00:13.675 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:13.671+0000 7fee7dffb640 1 -- 172.21.15.97:0/48265811 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 22) v1 ==== 92152+0+0 (secure 0 0 0) 0x7fee74004000 con 0x7fee80100580 2024-03-21T14:00:13.676 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:13.671+0000 7fee7dffb640 1 --2- 172.21.15.97:0/48265811 >> v2:172.21.15.97:6800/12494470 conn(0x7fee7006e270 0x7fee70070730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T14:00:13.676 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:13.671+0000 7fee7dffb640 1 -- 172.21.15.97:0/48265811 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(781..781 src has 254..781) v4 ==== 7814+0+0 (secure 0 0 0) 0x7fee7408ec30 con 0x7fee80100580 2024-03-21T14:00:13.676 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:13.671+0000 7fee7ffff640 1 --2- 172.21.15.97:0/48265811 >> v2:172.21.15.97:6800/12494470 conn(0x7fee7006e270 0x7fee70070730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T14:00:13.676 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:13.671+0000 7fee7dffb640 1 -- 172.21.15.97:0/48265811 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=782}) v3 -- 0x7fee700797c0 con 0x7fee80100580 2024-03-21T14:00:13.676 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:13.672+0000 7fee7ffff640 1 --2- 172.21.15.97:0/48265811 >> v2:172.21.15.97:6800/12494470 conn(0x7fee7006e270 0x7fee70070730 secure :-1 s=READY pgs=4043 cs=0 l=1 rev1=1 crypto rx=0x7fee6c000c00 tx=0x7fee6c0058d0 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T14:00:13.677 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:13.677+0000 7fee7dffb640 1 -- 172.21.15.97:0/48265811 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fee7405f7c0 con 0x7fee80100580 2024-03-21T14:00:13.878 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:13.878+0000 7fee86ae1640 1 -- 172.21.15.97:0/48265811 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "100"} v 0) v1 -- 0x7fee801a4330 con 0x7fee80100580 2024-03-21T14:00:14.150 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:14.150+0000 7fee7dffb640 1 -- 172.21.15.97:0/48265811 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool b5a5e36b-8518-4a8b-80b9-4898af5bd2dd v782) v1 ==== 225+0+0 (secure 0 0 0) 0x7fee74063ff0 con 0x7fee80100580 2024-03-21T14:00:14.151 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:14.151+0000 7fee7dffb640 1 -- 172.21.15.97:0/48265811 <== mon.0 v2:172.21.15.97:3300/0 8 ==== osd_map(782..782 src has 254..782) v4 ==== 628+0+0 (secure 0 0 0) 0x7fee7408ca30 con 0x7fee80100580 2024-03-21T14:00:14.151 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:14.151+0000 7fee7dffb640 1 -- 172.21.15.97:0/48265811 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=783}) v3 -- 0x7fee7007a6e0 con 0x7fee80100580 2024-03-21T14:00:14.263 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:14.262+0000 7fee86ae1640 1 -- 172.21.15.97:0/48265811 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "100"} v 0) v1 -- 0x7fee801a4580 con 0x7fee80100580 2024-03-21T14:00:14.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:14 smithi097 ceph-mon[39416]: pgmap v1714: 176 pgs: 176 active+clean; 698 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:00:14.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:14 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/48265811' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "100"}]: dispatch 2024-03-21T14:00:14.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:14 smithi097 ceph-mon[30040]: pgmap v1714: 176 pgs: 176 active+clean; 698 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:00:14.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:14 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/48265811' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "100"}]: dispatch 2024-03-21T14:00:14.526 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:14 smithi144 ceph-mon[33633]: pgmap v1714: 176 pgs: 176 active+clean; 698 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:00:14.526 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:14 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/48265811' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "100"}]: dispatch 2024-03-21T14:00:15.163 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:15.163+0000 7fee7dffb640 1 -- 172.21.15.97:0/48265811 <== mon.0 v2:172.21.15.97:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool b5a5e36b-8518-4a8b-80b9-4898af5bd2dd v783) v1 ==== 225+0+0 (secure 0 0 0) 0x7fee74003aa0 con 0x7fee80100580 2024-03-21T14:00:15.164 INFO:tasks.workunit.client.0.smithi097.stderr:set-quota max_objects = 100 for pool b5a5e36b-8518-4a8b-80b9-4898af5bd2dd 2024-03-21T14:00:15.165 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:15.165+0000 7fee86ae1640 1 -- 172.21.15.97:0/48265811 >> v2:172.21.15.97:6800/12494470 conn(0x7fee7006e270 msgr2=0x7fee70070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:00:15.166 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:15.165+0000 7fee86ae1640 1 --2- 172.21.15.97:0/48265811 >> v2:172.21.15.97:6800/12494470 conn(0x7fee7006e270 0x7fee70070730 secure :-1 s=READY pgs=4043 cs=0 l=1 rev1=1 crypto rx=0x7fee6c000c00 tx=0x7fee6c0058d0 comp rx=0 tx=0).stop 2024-03-21T14:00:15.166 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:15.165+0000 7fee86ae1640 1 -- 172.21.15.97:0/48265811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee80100580 msgr2=0x7fee80198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:00:15.166 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:15.165+0000 7fee86ae1640 1 --2- 172.21.15.97:0/48265811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee80100580 0x7fee80198460 secure :-1 s=READY pgs=3009 cs=0 l=1 rev1=1 crypto rx=0x7fee7400e9f0 tx=0x7fee7400eeb0 comp rx=0 tx=0).stop 2024-03-21T14:00:15.166 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:15.165+0000 7fee86ae1640 1 -- 172.21.15.97:0/48265811 shutdown_connections 2024-03-21T14:00:15.166 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:15.165+0000 7fee86ae1640 1 --2- 172.21.15.97:0/48265811 >> v2:172.21.15.97:6800/12494470 conn(0x7fee7006e270 0x7fee70070730 unknown :-1 s=CLOSED pgs=4043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:15.166 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:15.165+0000 7fee86ae1640 1 --2- 172.21.15.97:0/48265811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee80106640 0x7fee801a34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:15.166 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:15.165+0000 7fee86ae1640 1 --2- 172.21.15.97:0/48265811 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fee80100f40 0x7fee801989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:15.166 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:15.165+0000 7fee86ae1640 1 --2- 172.21.15.97:0/48265811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee80100580 0x7fee80198460 unknown :-1 s=CLOSED pgs=3009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:15.166 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:15.165+0000 7fee86ae1640 1 -- 172.21.15.97:0/48265811 >> 172.21.15.97:0/48265811 conn(0x7fee800776c0 msgr2=0x7fee80104130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T14:00:15.166 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:15.165+0000 7fee86ae1640 1 -- 172.21.15.97:0/48265811 shutdown_connections 2024-03-21T14:00:15.167 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:15.165+0000 7fee86ae1640 1 -- 172.21.15.97:0/48265811 wait complete. 2024-03-21T14:00:15.180 INFO:tasks.workunit.client.0.smithi097.stderr:+ wait 156148 2024-03-21T14:00:15.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:15 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/48265811' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "100"}]': finished 2024-03-21T14:00:15.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:15 smithi097 ceph-mon[39416]: osdmap e782: 8 total, 8 up, 8 in 2024-03-21T14:00:15.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:15 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/48265811' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "100"}]: dispatch 2024-03-21T14:00:15.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:15 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/48265811' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "100"}]': finished 2024-03-21T14:00:15.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:15 smithi097 ceph-mon[30040]: osdmap e782: 8 total, 8 up, 8 in 2024-03-21T14:00:15.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:15 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/48265811' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "100"}]: dispatch 2024-03-21T14:00:15.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:15 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/48265811' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "100"}]': finished 2024-03-21T14:00:15.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:15 smithi144 ceph-mon[33633]: osdmap e782: 8 total, 8 up, 8 in 2024-03-21T14:00:15.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:15 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/48265811' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "100"}]: dispatch 2024-03-21T14:00:15.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 14:00:15 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T14:00:16.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:16 smithi144 ceph-mon[33633]: pgmap v1716: 176 pgs: 176 active+clean; 698 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T14:00:16.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:16 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T14:00:16.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:16 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/48265811' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "100"}]': finished 2024-03-21T14:00:16.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:16 smithi144 ceph-mon[33633]: osdmap e783: 8 total, 8 up, 8 in 2024-03-21T14:00:16.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:16 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T14:00:16.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:16 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T14:00:16.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:16 smithi097 ceph-mon[39416]: pgmap v1716: 176 pgs: 176 active+clean; 698 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T14:00:16.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:16 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T14:00:16.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:16 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/48265811' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "100"}]': finished 2024-03-21T14:00:16.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:16 smithi097 ceph-mon[39416]: osdmap e783: 8 total, 8 up, 8 in 2024-03-21T14:00:16.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:16 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T14:00:16.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:16 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T14:00:16.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:16 smithi097 ceph-mon[30040]: pgmap v1716: 176 pgs: 176 active+clean; 698 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T14:00:16.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:16 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T14:00:16.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:16 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/48265811' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "100"}]': finished 2024-03-21T14:00:16.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:16 smithi097 ceph-mon[30040]: osdmap e783: 8 total, 8 up, 8 in 2024-03-21T14:00:16.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:16 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T14:00:16.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:16 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T14:00:17.739 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 0 -ne 0 ']' 2024-03-21T14:00:17.739 INFO:tasks.workunit.client.0.smithi097.stderr:+ true 2024-03-21T14:00:17.739 INFO:tasks.workunit.client.0.smithi097.stderr:+ rados -p b5a5e36b-8518-4a8b-80b9-4898af5bd2dd put twomore /etc/passwd 2024-03-21T14:00:17.777 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph osd pool set-quota b5a5e36b-8518-4a8b-80b9-4898af5bd2dd max_bytes 100 2024-03-21T14:00:17.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:17 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T14:00:17.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:17 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T14:00:17.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:17 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T14:00:17.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:17 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T14:00:17.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:17 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T14:00:17.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:17 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T14:00:17.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:17 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T14:00:17.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:17 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T14:00:17.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:17 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T14:00:17.805 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:17 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T14:00:17.865 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:17.862+0000 7fcecbc58640 1 -- 172.21.15.97:0/229090366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcec4074360 msgr2=0x7fcec4112140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:00:17.865 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:17.862+0000 7fcecbc58640 1 --2- 172.21.15.97:0/229090366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcec4074360 0x7fcec4112140 secure :-1 s=READY pgs=3011 cs=0 l=1 rev1=1 crypto rx=0x7fceb4009900 tx=0x7fceb402bcd0 comp rx=0 tx=0).stop 2024-03-21T14:00:17.865 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:17.862+0000 7fcecbc58640 1 -- 172.21.15.97:0/229090366 shutdown_connections 2024-03-21T14:00:17.865 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:17.862+0000 7fcecbc58640 1 --2- 172.21.15.97:0/229090366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcec4074360 0x7fcec4112140 unknown :-1 s=CLOSED pgs=3011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:17.866 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:17.862+0000 7fcecbc58640 1 --2- 172.21.15.97:0/229090366 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcec4112680 0x7fcec4114a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:17.866 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:17.862+0000 7fcecbc58640 1 --2- 172.21.15.97:0/229090366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcec4073a40 0x7fcec4073e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:17.866 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:17.862+0000 7fcecbc58640 1 -- 172.21.15.97:0/229090366 >> 172.21.15.97:0/229090366 conn(0x7fcec40fd180 msgr2=0x7fcec40ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T14:00:17.866 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:17.862+0000 7fcecbc58640 1 -- 172.21.15.97:0/229090366 shutdown_connections 2024-03-21T14:00:17.866 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:17.862+0000 7fcecbc58640 1 -- 172.21.15.97:0/229090366 wait complete. 2024-03-21T14:00:17.866 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:17.863+0000 7fcecbc58640 1 Processor -- start 2024-03-21T14:00:17.866 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:17.863+0000 7fcecbc58640 1 -- start start 2024-03-21T14:00:17.866 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:17.863+0000 7fcecbc58640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcec4073a40 0x7fcec41a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T14:00:17.866 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:17.863+0000 7fcecbc58640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcec4074360 0x7fcec41a3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T14:00:17.866 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:17.863+0000 7fcecbc58640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcec4112680 0x7fcec419db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T14:00:17.866 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:17.863+0000 7fcecbc58640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec4117790 con 0x7fcec4074360 2024-03-21T14:00:17.866 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:17.863+0000 7fcecbc58640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec4117610 con 0x7fcec4073a40 2024-03-21T14:00:17.866 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:17.863+0000 7fcecbc58640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcec4117910 con 0x7fcec4112680 2024-03-21T14:00:17.867 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:17.863+0000 7fcec91f3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcec4074360 0x7fcec41a3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T14:00:17.867 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:17.863+0000 7fceca1f5640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcec4112680 0x7fcec419db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T14:00:17.867 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:17.863+0000 7fcec91f3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcec4074360 0x7fcec41a3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54604/0 (socket says 172.21.15.97:54604) 2024-03-21T14:00:17.867 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:17.863+0000 7fcec91f3640 1 -- 172.21.15.97:0/123700993 learned_addr learned my addr 172.21.15.97:0/123700993 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T14:00:17.867 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:17.863+0000 7fcec99f4640 1 --2- 172.21.15.97:0/123700993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcec4073a40 0x7fcec41a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T14:00:17.867 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:17.863+0000 7fcec91f3640 1 -- 172.21.15.97:0/123700993 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcec4112680 msgr2=0x7fcec419db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:00:17.867 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:17.863+0000 7fcec91f3640 1 --2- 172.21.15.97:0/123700993 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcec4112680 0x7fcec419db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:17.867 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:17.863+0000 7fcec91f3640 1 -- 172.21.15.97:0/123700993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcec4073a40 msgr2=0x7fcec41a3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:00:17.867 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:17.863+0000 7fcec91f3640 1 --2- 172.21.15.97:0/123700993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcec4073a40 0x7fcec41a3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:17.867 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:17.863+0000 7fcec91f3640 1 -- 172.21.15.97:0/123700993 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec419e3d0 con 0x7fcec4074360 2024-03-21T14:00:17.867 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:17.864+0000 7fceca1f5640 1 --2- 172.21.15.97:0/123700993 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcec4112680 0x7fcec419db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T14:00:17.867 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:17.864+0000 7fcec91f3640 1 --2- 172.21.15.97:0/123700993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcec4074360 0x7fcec41a3f90 secure :-1 s=READY pgs=3012 cs=0 l=1 rev1=1 crypto rx=0x7fceb4005940 tx=0x7fceb402c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T14:00:17.867 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:17.864+0000 7fcebaffd640 1 -- 172.21.15.97:0/123700993 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb4043020 con 0x7fcec4074360 2024-03-21T14:00:17.867 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:17.864+0000 7fcebaffd640 1 -- 172.21.15.97:0/123700993 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fceb4004350 con 0x7fcec4074360 2024-03-21T14:00:17.867 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:17.864+0000 7fcebaffd640 1 -- 172.21.15.97:0/123700993 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb4003970 con 0x7fcec4074360 2024-03-21T14:00:17.868 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:17.864+0000 7fcecbc58640 1 -- 172.21.15.97:0/123700993 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcec419e660 con 0x7fcec4074360 2024-03-21T14:00:17.868 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:17.864+0000 7fcecbc58640 1 -- 172.21.15.97:0/123700993 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcec41aa9d0 con 0x7fcec4074360 2024-03-21T14:00:17.868 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:17.865+0000 7fcecbc58640 1 -- 172.21.15.97:0/123700993 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcec4075180 con 0x7fcec4074360 2024-03-21T14:00:17.868 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:17.865+0000 7fcebaffd640 1 -- 172.21.15.97:0/123700993 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 22) v1 ==== 92152+0+0 (secure 0 0 0) 0x7fceb4046020 con 0x7fcec4074360 2024-03-21T14:00:17.868 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:17.866+0000 7fcebaffd640 1 --2- 172.21.15.97:0/123700993 >> v2:172.21.15.97:6800/12494470 conn(0x7fceb006e270 0x7fceb0070730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T14:00:17.869 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:17.866+0000 7fcebaffd640 1 -- 172.21.15.97:0/123700993 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(784..784 src has 254..784) v4 ==== 7814+0+0 (secure 0 0 0) 0x7fceb40b0a70 con 0x7fcec4074360 2024-03-21T14:00:17.869 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:17.866+0000 7fcec99f4640 1 --2- 172.21.15.97:0/123700993 >> v2:172.21.15.97:6800/12494470 conn(0x7fceb006e270 0x7fceb0070730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T14:00:17.869 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:17.866+0000 7fcec99f4640 1 --2- 172.21.15.97:0/123700993 >> v2:172.21.15.97:6800/12494470 conn(0x7fceb006e270 0x7fceb0070730 secure :-1 s=READY pgs=4045 cs=0 l=1 rev1=1 crypto rx=0x7fcec0009bd0 tx=0x7fcec0005f90 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T14:00:17.870 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:17.870+0000 7fcebaffd640 1 -- 172.21.15.97:0/123700993 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fceb404d020 con 0x7fcec4074360 2024-03-21T14:00:17.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:17 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T14:00:17.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:17 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T14:00:17.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:17 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T14:00:17.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:17 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T14:00:17.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:17 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T14:00:18.048 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:18.048+0000 7fcecbc58640 1 -- 172.21.15.97:0/123700993 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "100"} v 0) v1 -- 0x7fcec40689d0 con 0x7fcec4074360 2024-03-21T14:00:18.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:18 smithi097 ceph-mon[39416]: pgmap v1718: 176 pgs: 176 active+clean; 698 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-21T14:00:18.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:18 smithi097 ceph-mon[39416]: pool 'b5a5e36b-8518-4a8b-80b9-4898af5bd2dd' no longer out of quota; removing NO_QUOTA flag 2024-03-21T14:00:18.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:18 smithi097 ceph-mon[39416]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-21T14:00:18.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:18 smithi097 ceph-mon[39416]: osdmap e784: 8 total, 8 up, 8 in 2024-03-21T14:00:18.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:18 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/123700993' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "100"}]: dispatch 2024-03-21T14:00:18.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:18 smithi097 ceph-mon[30040]: pgmap v1718: 176 pgs: 176 active+clean; 698 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-21T14:00:18.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:18 smithi097 ceph-mon[30040]: pool 'b5a5e36b-8518-4a8b-80b9-4898af5bd2dd' no longer out of quota; removing NO_QUOTA flag 2024-03-21T14:00:18.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:18 smithi097 ceph-mon[30040]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-21T14:00:18.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:18 smithi097 ceph-mon[30040]: osdmap e784: 8 total, 8 up, 8 in 2024-03-21T14:00:18.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:18 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/123700993' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "100"}]: dispatch 2024-03-21T14:00:18.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:18 smithi144 ceph-mon[33633]: pgmap v1718: 176 pgs: 176 active+clean; 698 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-21T14:00:18.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:18 smithi144 ceph-mon[33633]: pool 'b5a5e36b-8518-4a8b-80b9-4898af5bd2dd' no longer out of quota; removing NO_QUOTA flag 2024-03-21T14:00:18.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:18 smithi144 ceph-mon[33633]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-21T14:00:18.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:18 smithi144 ceph-mon[33633]: osdmap e784: 8 total, 8 up, 8 in 2024-03-21T14:00:18.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:18 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/123700993' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "100"}]: dispatch 2024-03-21T14:00:19.022 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:19.022+0000 7fcebaffd640 1 -- 172.21.15.97:0/123700993 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool b5a5e36b-8518-4a8b-80b9-4898af5bd2dd v785) v1 ==== 221+0+0 (secure 0 0 0) 0x7fceb402ce20 con 0x7fcec4074360 2024-03-21T14:00:19.134 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:19.133+0000 7fcecbc58640 1 -- 172.21.15.97:0/123700993 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "100"} v 0) v1 -- 0x7fcec4104af0 con 0x7fcec4074360 2024-03-21T14:00:20.030 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:20.030+0000 7fcebaffd640 1 -- 172.21.15.97:0/123700993 <== mon.0 v2:172.21.15.97:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool b5a5e36b-8518-4a8b-80b9-4898af5bd2dd v786) v1 ==== 221+0+0 (secure 0 0 0) 0x7fceb407dc30 con 0x7fcec4074360 2024-03-21T14:00:20.030 INFO:tasks.workunit.client.0.smithi097.stderr:set-quota max_bytes = 100 for pool b5a5e36b-8518-4a8b-80b9-4898af5bd2dd 2024-03-21T14:00:20.033 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:20.032+0000 7fcecbc58640 1 -- 172.21.15.97:0/123700993 >> v2:172.21.15.97:6800/12494470 conn(0x7fceb006e270 msgr2=0x7fceb0070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:00:20.033 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:20.032+0000 7fcecbc58640 1 --2- 172.21.15.97:0/123700993 >> v2:172.21.15.97:6800/12494470 conn(0x7fceb006e270 0x7fceb0070730 secure :-1 s=READY pgs=4045 cs=0 l=1 rev1=1 crypto rx=0x7fcec0009bd0 tx=0x7fcec0005f90 comp rx=0 tx=0).stop 2024-03-21T14:00:20.033 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:20.032+0000 7fcecbc58640 1 -- 172.21.15.97:0/123700993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcec4074360 msgr2=0x7fcec41a3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:00:20.033 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:20.032+0000 7fcecbc58640 1 --2- 172.21.15.97:0/123700993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcec4074360 0x7fcec41a3f90 secure :-1 s=READY pgs=3012 cs=0 l=1 rev1=1 crypto rx=0x7fceb4005940 tx=0x7fceb402c7c0 comp rx=0 tx=0).stop 2024-03-21T14:00:20.033 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:20.033+0000 7fcecbc58640 1 -- 172.21.15.97:0/123700993 shutdown_connections 2024-03-21T14:00:20.033 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:20.033+0000 7fcecbc58640 1 --2- 172.21.15.97:0/123700993 >> v2:172.21.15.97:6800/12494470 conn(0x7fceb006e270 0x7fceb0070730 unknown :-1 s=CLOSED pgs=4045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:20.034 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:20.033+0000 7fcecbc58640 1 --2- 172.21.15.97:0/123700993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcec4074360 0x7fcec41a3f90 unknown :-1 s=CLOSED pgs=3012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:20.034 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:20.033+0000 7fcecbc58640 1 --2- 172.21.15.97:0/123700993 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcec4112680 0x7fcec419db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:20.034 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:20.033+0000 7fcecbc58640 1 --2- 172.21.15.97:0/123700993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcec4073a40 0x7fcec41a3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:20.034 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:20.033+0000 7fcecbc58640 1 -- 172.21.15.97:0/123700993 >> 172.21.15.97:0/123700993 conn(0x7fcec40fd180 msgr2=0x7fcec40fea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T14:00:20.034 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:20.033+0000 7fcecbc58640 1 -- 172.21.15.97:0/123700993 shutdown_connections 2024-03-21T14:00:20.034 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:20.033+0000 7fcecbc58640 1 -- 172.21.15.97:0/123700993 wait complete. 2024-03-21T14:00:20.047 INFO:tasks.workunit.client.0.smithi097.stderr:+ sleep 30 2024-03-21T14:00:20.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:20 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/123700993' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "100"}]': finished 2024-03-21T14:00:20.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:20 smithi097 ceph-mon[39416]: osdmap e785: 8 total, 8 up, 8 in 2024-03-21T14:00:20.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:20 smithi097 ceph-mon[39416]: pgmap v1721: 176 pgs: 176 active+clean; 721 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 4.9 KiB/s wr, 1 op/s 2024-03-21T14:00:20.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:20 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/123700993' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "100"}]: dispatch 2024-03-21T14:00:20.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:20 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/123700993' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "100"}]': finished 2024-03-21T14:00:20.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:20 smithi097 ceph-mon[30040]: osdmap e785: 8 total, 8 up, 8 in 2024-03-21T14:00:20.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:20 smithi097 ceph-mon[30040]: pgmap v1721: 176 pgs: 176 active+clean; 721 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 4.9 KiB/s wr, 1 op/s 2024-03-21T14:00:20.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:20 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/123700993' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "100"}]: dispatch 2024-03-21T14:00:20.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:20 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/123700993' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "100"}]': finished 2024-03-21T14:00:20.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:20 smithi144 ceph-mon[33633]: osdmap e785: 8 total, 8 up, 8 in 2024-03-21T14:00:20.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:20 smithi144 ceph-mon[33633]: pgmap v1721: 176 pgs: 176 active+clean; 721 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 4.9 KiB/s wr, 1 op/s 2024-03-21T14:00:20.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:20 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/123700993' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "100"}]: dispatch 2024-03-21T14:00:21.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:21 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/123700993' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "100"}]': finished 2024-03-21T14:00:21.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:21 smithi097 ceph-mon[39416]: osdmap e786: 8 total, 8 up, 8 in 2024-03-21T14:00:21.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:21 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/123700993' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "100"}]': finished 2024-03-21T14:00:21.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:21 smithi097 ceph-mon[30040]: osdmap e786: 8 total, 8 up, 8 in 2024-03-21T14:00:21.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:21 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/123700993' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "100"}]': finished 2024-03-21T14:00:21.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:21 smithi144 ceph-mon[33633]: osdmap e786: 8 total, 8 up, 8 in 2024-03-21T14:00:22.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:22 smithi097 ceph-mon[39416]: pgmap v1723: 176 pgs: 176 active+clean; 721 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 865 B/s rd, 4.1 KiB/s wr, 1 op/s 2024-03-21T14:00:22.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:22 smithi097 ceph-mon[30040]: pgmap v1723: 176 pgs: 176 active+clean; 721 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 865 B/s rd, 4.1 KiB/s wr, 1 op/s 2024-03-21T14:00:22.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 14:00:21 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:14:00:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T14:00:22.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:22 smithi144 ceph-mon[33633]: pgmap v1723: 176 pgs: 176 active+clean; 721 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 865 B/s rd, 4.1 KiB/s wr, 1 op/s 2024-03-21T14:00:23.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:23 smithi097 ceph-mon[39416]: pool 'b5a5e36b-8518-4a8b-80b9-4898af5bd2dd' is full (reached quota's max_bytes: 100 B) 2024-03-21T14:00:23.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:23 smithi097 ceph-mon[39416]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-21T14:00:23.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:23 smithi097 ceph-mon[39416]: osdmap e787: 8 total, 8 up, 8 in 2024-03-21T14:00:23.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:23 smithi097 ceph-mon[30040]: pool 'b5a5e36b-8518-4a8b-80b9-4898af5bd2dd' is full (reached quota's max_bytes: 100 B) 2024-03-21T14:00:23.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:23 smithi097 ceph-mon[30040]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-21T14:00:23.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:23 smithi097 ceph-mon[30040]: osdmap e787: 8 total, 8 up, 8 in 2024-03-21T14:00:23.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:23 smithi144 ceph-mon[33633]: pool 'b5a5e36b-8518-4a8b-80b9-4898af5bd2dd' is full (reached quota's max_bytes: 100 B) 2024-03-21T14:00:23.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:23 smithi144 ceph-mon[33633]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-21T14:00:23.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:23 smithi144 ceph-mon[33633]: osdmap e787: 8 total, 8 up, 8 in 2024-03-21T14:00:24.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:24 smithi097 ceph-mon[30040]: pgmap v1725: 176 pgs: 176 active+clean; 721 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 956 B/s rd, 4.5 KiB/s wr, 1 op/s 2024-03-21T14:00:24.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:24 smithi097 ceph-mon[39416]: pgmap v1725: 176 pgs: 176 active+clean; 721 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 956 B/s rd, 4.5 KiB/s wr, 1 op/s 2024-03-21T14:00:24.373 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:24 smithi144 ceph-mon[33633]: pgmap v1725: 176 pgs: 176 active+clean; 721 KiB data, 917 MiB used, 714 GiB / 715 GiB avail; 956 B/s rd, 4.5 KiB/s wr, 1 op/s 2024-03-21T14:00:25.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 14:00:25 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T14:00:26.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:26 smithi097 ceph-mon[39416]: pgmap v1726: 176 pgs: 176 active+clean; 721 KiB data, 918 MiB used, 714 GiB / 715 GiB avail; 844 B/s rd, 0 op/s 2024-03-21T14:00:26.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:26 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T14:00:26.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:26 smithi097 ceph-mon[30040]: pgmap v1726: 176 pgs: 176 active+clean; 721 KiB data, 918 MiB used, 714 GiB / 715 GiB avail; 844 B/s rd, 0 op/s 2024-03-21T14:00:26.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:26 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T14:00:26.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:26 smithi144 ceph-mon[33633]: pgmap v1726: 176 pgs: 176 active+clean; 721 KiB data, 918 MiB used, 714 GiB / 715 GiB avail; 844 B/s rd, 0 op/s 2024-03-21T14:00:26.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:26 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T14:00:27.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:27 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T14:00:27.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:27 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T14:00:27.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:27 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T14:00:28.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:28 smithi097 ceph-mon[39416]: pgmap v1727: 176 pgs: 176 active+clean; 721 KiB data, 918 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-21T14:00:28.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:28 smithi097 ceph-mon[30040]: pgmap v1727: 176 pgs: 176 active+clean; 721 KiB data, 918 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-21T14:00:28.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:28 smithi144 ceph-mon[33633]: pgmap v1727: 176 pgs: 176 active+clean; 721 KiB data, 918 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-21T14:00:30.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:30 smithi097 ceph-mon[39416]: pgmap v1728: 176 pgs: 176 active+clean; 721 KiB data, 918 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-21T14:00:30.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:30 smithi097 ceph-mon[30040]: pgmap v1728: 176 pgs: 176 active+clean; 721 KiB data, 918 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-21T14:00:30.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:30 smithi144 ceph-mon[33633]: pgmap v1728: 176 pgs: 176 active+clean; 721 KiB data, 918 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-21T14:00:32.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:32 smithi097 ceph-mon[39416]: pgmap v1729: 176 pgs: 176 active+clean; 721 KiB data, 918 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T14:00:32.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:32 smithi097 ceph-mon[30040]: pgmap v1729: 176 pgs: 176 active+clean; 721 KiB data, 918 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T14:00:32.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 14:00:31 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:14:00:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T14:00:32.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:32 smithi144 ceph-mon[33633]: pgmap v1729: 176 pgs: 176 active+clean; 721 KiB data, 918 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T14:00:34.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:34 smithi097 ceph-mon[30040]: pgmap v1730: 176 pgs: 176 active+clean; 721 KiB data, 918 MiB used, 714 GiB / 715 GiB avail; 988 B/s rd, 0 op/s 2024-03-21T14:00:34.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:34 smithi097 ceph-mon[39416]: pgmap v1730: 176 pgs: 176 active+clean; 721 KiB data, 918 MiB used, 714 GiB / 715 GiB avail; 988 B/s rd, 0 op/s 2024-03-21T14:00:34.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:34 smithi144 ceph-mon[33633]: pgmap v1730: 176 pgs: 176 active+clean; 721 KiB data, 918 MiB used, 714 GiB / 715 GiB avail; 988 B/s rd, 0 op/s 2024-03-21T14:00:35.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 14:00:35 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T14:00:36.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:36 smithi097 ceph-mon[39416]: pgmap v1731: 176 pgs: 176 active+clean; 721 KiB data, 918 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T14:00:36.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:36 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T14:00:36.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:36 smithi097 ceph-mon[30040]: pgmap v1731: 176 pgs: 176 active+clean; 721 KiB data, 918 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T14:00:36.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:36 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T14:00:36.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:36 smithi144 ceph-mon[33633]: pgmap v1731: 176 pgs: 176 active+clean; 721 KiB data, 918 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T14:00:36.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:36 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T14:00:38.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:38 smithi097 ceph-mon[39416]: pgmap v1732: 176 pgs: 176 active+clean; 721 KiB data, 918 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:00:38.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:38 smithi097 ceph-mon[30040]: pgmap v1732: 176 pgs: 176 active+clean; 721 KiB data, 918 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:00:38.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:38 smithi144 ceph-mon[33633]: pgmap v1732: 176 pgs: 176 active+clean; 721 KiB data, 918 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:00:40.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:40 smithi097 ceph-mon[39416]: pgmap v1733: 176 pgs: 176 active+clean; 721 KiB data, 918 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T14:00:40.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:40 smithi097 ceph-mon[30040]: pgmap v1733: 176 pgs: 176 active+clean; 721 KiB data, 918 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T14:00:40.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:40 smithi144 ceph-mon[33633]: pgmap v1733: 176 pgs: 176 active+clean; 721 KiB data, 918 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T14:00:42.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:42 smithi097 ceph-mon[39416]: pgmap v1734: 176 pgs: 176 active+clean; 721 KiB data, 918 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:00:42.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:42 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T14:00:42.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:42 smithi097 ceph-mon[30040]: pgmap v1734: 176 pgs: 176 active+clean; 721 KiB data, 918 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:00:42.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:42 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T14:00:42.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 14:00:41 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:14:00:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T14:00:42.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:42 smithi144 ceph-mon[33633]: pgmap v1734: 176 pgs: 176 active+clean; 721 KiB data, 918 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:00:42.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:42 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T14:00:44.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:44 smithi097 ceph-mon[39416]: pgmap v1735: 176 pgs: 176 active+clean; 721 KiB data, 918 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:00:44.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:44 smithi097 ceph-mon[30040]: pgmap v1735: 176 pgs: 176 active+clean; 721 KiB data, 918 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:00:44.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:44 smithi144 ceph-mon[33633]: pgmap v1735: 176 pgs: 176 active+clean; 721 KiB data, 918 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:00:45.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 14:00:45 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T14:00:46.522 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:46 smithi144 ceph-mon[33633]: pgmap v1736: 176 pgs: 176 active+clean; 721 KiB data, 918 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T14:00:46.523 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:46 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T14:00:46.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:46 smithi097 ceph-mon[39416]: pgmap v1736: 176 pgs: 176 active+clean; 721 KiB data, 918 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T14:00:46.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:46 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T14:00:46.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:46 smithi097 ceph-mon[30040]: pgmap v1736: 176 pgs: 176 active+clean; 721 KiB data, 918 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T14:00:46.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:46 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T14:00:48.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:48 smithi097 ceph-mon[39416]: pgmap v1737: 176 pgs: 176 active+clean; 721 KiB data, 918 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:00:48.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:48 smithi097 ceph-mon[30040]: pgmap v1737: 176 pgs: 176 active+clean; 721 KiB data, 918 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:00:48.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:48 smithi144 ceph-mon[33633]: pgmap v1737: 176 pgs: 176 active+clean; 721 KiB data, 918 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:00:50.049 INFO:tasks.workunit.client.0.smithi097.stderr:+ pid=156563 2024-03-21T14:00:50.049 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph osd pool set-quota b5a5e36b-8518-4a8b-80b9-4898af5bd2dd max_bytes 0 2024-03-21T14:00:50.049 INFO:tasks.workunit.client.0.smithi097.stderr:+ rados -p b5a5e36b-8518-4a8b-80b9-4898af5bd2dd put two /etc/passwd 2024-03-21T14:00:50.145 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:50.142+0000 7fe63b88f640 1 -- 172.21.15.97:0/3029534436 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe6340fff50 msgr2=0x7fe634100330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:00:50.145 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:50.142+0000 7fe63b88f640 1 --2- 172.21.15.97:0/3029534436 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe6340fff50 0x7fe634100330 secure :-1 s=READY pgs=3219 cs=0 l=1 rev1=1 crypto rx=0x7fe62c009900 tx=0x7fe62c02bc30 comp rx=0 tx=0).stop 2024-03-21T14:00:50.145 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:50.142+0000 7fe63b88f640 1 -- 172.21.15.97:0/3029534436 shutdown_connections 2024-03-21T14:00:50.145 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:50.142+0000 7fe63b88f640 1 --2- 172.21.15.97:0/3029534436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe634105020 0x7fe63410cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:50.145 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:50.142+0000 7fe63b88f640 1 --2- 172.21.15.97:0/3029534436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe634100900 0x7fe634104940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:50.145 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:50.142+0000 7fe63b88f640 1 --2- 172.21.15.97:0/3029534436 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe6340fff50 0x7fe634100330 unknown :-1 s=CLOSED pgs=3219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:50.145 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:50.142+0000 7fe63b88f640 1 -- 172.21.15.97:0/3029534436 >> 172.21.15.97:0/3029534436 conn(0x7fe634076ab0 msgr2=0x7fe6340fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T14:00:50.145 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:50.142+0000 7fe63b88f640 1 -- 172.21.15.97:0/3029534436 shutdown_connections 2024-03-21T14:00:50.146 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:50.142+0000 7fe63b88f640 1 -- 172.21.15.97:0/3029534436 wait complete. 2024-03-21T14:00:50.146 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:50.143+0000 7fe63b88f640 1 Processor -- start 2024-03-21T14:00:50.146 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:50.143+0000 7fe63b88f640 1 -- start start 2024-03-21T14:00:50.146 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:50.143+0000 7fe63b88f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6340fff50 0x7fe6341984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T14:00:50.146 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:50.143+0000 7fe63b88f640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe634105020 0x7fe6341989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T14:00:50.146 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:50.143+0000 7fe63b88f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe634198f80 0x7fe6341a3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T14:00:50.146 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:50.143+0000 7fe63b88f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe634110060 con 0x7fe634198f80 2024-03-21T14:00:50.146 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:50.143+0000 7fe63b88f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe63410fee0 con 0x7fe6340fff50 2024-03-21T14:00:50.146 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:50.143+0000 7fe63b88f640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6341101e0 con 0x7fe634105020 2024-03-21T14:00:50.146 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:50.143+0000 7fe638e2a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe634105020 0x7fe6341989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T14:00:50.146 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:50.143+0000 7fe639e2c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe634198f80 0x7fe6341a3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T14:00:50.146 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:50.143+0000 7fe638e2a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe634105020 0x7fe6341989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.97:44032/0 (socket says 172.21.15.97:44032) 2024-03-21T14:00:50.147 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:50.143+0000 7fe638e2a640 1 -- 172.21.15.97:0/126003720 learned_addr learned my addr 172.21.15.97:0/126003720 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T14:00:50.147 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:50.143+0000 7fe63962b640 1 --2- 172.21.15.97:0/126003720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6340fff50 0x7fe6341984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T14:00:50.147 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:50.143+0000 7fe639e2c640 1 -- 172.21.15.97:0/126003720 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe634105020 msgr2=0x7fe6341989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:00:50.147 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:50.143+0000 7fe639e2c640 1 --2- 172.21.15.97:0/126003720 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe634105020 0x7fe6341989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:50.147 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:50.143+0000 7fe639e2c640 1 -- 172.21.15.97:0/126003720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6340fff50 msgr2=0x7fe6341984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:00:50.147 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:50.143+0000 7fe639e2c640 1 --2- 172.21.15.97:0/126003720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6340fff50 0x7fe6341984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:50.147 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:50.143+0000 7fe639e2c640 1 -- 172.21.15.97:0/126003720 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6341a3d30 con 0x7fe634198f80 2024-03-21T14:00:50.147 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:50.143+0000 7fe638e2a640 1 --2- 172.21.15.97:0/126003720 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe634105020 0x7fe6341989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T14:00:50.147 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:50.144+0000 7fe639e2c640 1 --2- 172.21.15.97:0/126003720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe634198f80 0x7fe6341a3530 secure :-1 s=READY pgs=3014 cs=0 l=1 rev1=1 crypto rx=0x7fe62400ec50 tx=0x7fe62400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T14:00:50.147 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:50.144+0000 7fe6327fc640 1 -- 172.21.15.97:0/126003720 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe624003b70 con 0x7fe634198f80 2024-03-21T14:00:50.147 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:50.144+0000 7fe6327fc640 1 -- 172.21.15.97:0/126003720 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fe624003d10 con 0x7fe634198f80 2024-03-21T14:00:50.147 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:50.144+0000 7fe63b88f640 1 -- 172.21.15.97:0/126003720 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6341a4020 con 0x7fe634198f80 2024-03-21T14:00:50.147 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:50.144+0000 7fe6327fc640 1 -- 172.21.15.97:0/126003720 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6240107b0 con 0x7fe634198f80 2024-03-21T14:00:50.147 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:50.144+0000 7fe63b88f640 1 -- 172.21.15.97:0/126003720 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6341a44f0 con 0x7fe634198f80 2024-03-21T14:00:50.147 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:50.145+0000 7fe63b88f640 1 -- 172.21.15.97:0/126003720 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe634101420 con 0x7fe634198f80 2024-03-21T14:00:50.148 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:50.145+0000 7fe6327fc640 1 -- 172.21.15.97:0/126003720 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 22) v1 ==== 92152+0+0 (secure 0 0 0) 0x7fe624003fc0 con 0x7fe634198f80 2024-03-21T14:00:50.148 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:50.146+0000 7fe6327fc640 1 --2- 172.21.15.97:0/126003720 >> v2:172.21.15.97:6800/12494470 conn(0x7fe61c06e270 0x7fe61c070730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T14:00:50.148 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:50.146+0000 7fe63962b640 1 --2- 172.21.15.97:0/126003720 >> v2:172.21.15.97:6800/12494470 conn(0x7fe61c06e270 0x7fe61c070730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T14:00:50.149 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:50.146+0000 7fe6327fc640 1 -- 172.21.15.97:0/126003720 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(787..787 src has 254..787) v4 ==== 7814+0+0 (secure 0 0 0) 0x7fe62408ef20 con 0x7fe634198f80 2024-03-21T14:00:50.149 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:50.146+0000 7fe6327fc640 1 -- 172.21.15.97:0/126003720 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=788}) v3 -- 0x7fe61c0797c0 con 0x7fe634198f80 2024-03-21T14:00:50.149 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:50.146+0000 7fe63962b640 1 --2- 172.21.15.97:0/126003720 >> v2:172.21.15.97:6800/12494470 conn(0x7fe61c06e270 0x7fe61c070730 secure :-1 s=READY pgs=4047 cs=0 l=1 rev1=1 crypto rx=0x7fe62c006010 tx=0x7fe62c005e40 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T14:00:50.151 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:50.151+0000 7fe6327fc640 1 -- 172.21.15.97:0/126003720 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe62405fab0 con 0x7fe634198f80 2024-03-21T14:00:50.329 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:50.329+0000 7fe63b88f640 1 -- 172.21.15.97:0/126003720 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7fe6340689a0 con 0x7fe634198f80 2024-03-21T14:00:50.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:50 smithi097 ceph-mon[39416]: pgmap v1738: 176 pgs: 176 active+clean; 721 KiB data, 918 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T14:00:50.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:50 smithi097 ceph-mon[30040]: pgmap v1738: 176 pgs: 176 active+clean; 721 KiB data, 918 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T14:00:50.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:50 smithi144 ceph-mon[33633]: pgmap v1738: 176 pgs: 176 active+clean; 721 KiB data, 918 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T14:00:51.193 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:51.192+0000 7fe6327fc640 1 -- 172.21.15.97:0/126003720 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool b5a5e36b-8518-4a8b-80b9-4898af5bd2dd v788) v1 ==== 217+0+0 (secure 0 0 0) 0x7fe6240642e0 con 0x7fe634198f80 2024-03-21T14:00:51.196 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:51.195+0000 7fe6327fc640 1 -- 172.21.15.97:0/126003720 <== mon.0 v2:172.21.15.97:3300/0 8 ==== osd_map(788..788 src has 254..788) v4 ==== 628+0+0 (secure 0 0 0) 0x7fe62408cd20 con 0x7fe634198f80 2024-03-21T14:00:51.196 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:51.195+0000 7fe6327fc640 1 -- 172.21.15.97:0/126003720 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=789}) v3 -- 0x7fe61c07a6e0 con 0x7fe634198f80 2024-03-21T14:00:51.308 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:51.308+0000 7fe63b88f640 1 -- 172.21.15.97:0/126003720 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7fe634100900 con 0x7fe634198f80 2024-03-21T14:00:51.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:51 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/126003720' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-21T14:00:51.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:51 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/126003720' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-21T14:00:51.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:51 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/126003720' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-21T14:00:52.199 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.198+0000 7fe6327fc640 1 -- 172.21.15.97:0/126003720 <== mon.0 v2:172.21.15.97:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool b5a5e36b-8518-4a8b-80b9-4898af5bd2dd v789) v1 ==== 217+0+0 (secure 0 0 0) 0x7fe624059ab0 con 0x7fe634198f80 2024-03-21T14:00:52.199 INFO:tasks.workunit.client.0.smithi097.stderr:set-quota max_bytes = 0 for pool b5a5e36b-8518-4a8b-80b9-4898af5bd2dd 2024-03-21T14:00:52.199 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.199+0000 7fe6327fc640 1 -- 172.21.15.97:0/126003720 <== mon.0 v2:172.21.15.97:3300/0 10 ==== osd_map(789..789 src has 254..789) v4 ==== 628+0+0 (secure 0 0 0) 0x7fe624059d10 con 0x7fe634198f80 2024-03-21T14:00:52.199 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.199+0000 7fe6327fc640 1 -- 172.21.15.97:0/126003720 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=790}) v3 -- 0x7fe61c07a8f0 con 0x7fe634198f80 2024-03-21T14:00:52.203 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.202+0000 7fe63b88f640 1 -- 172.21.15.97:0/126003720 >> v2:172.21.15.97:6800/12494470 conn(0x7fe61c06e270 msgr2=0x7fe61c070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:00:52.203 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.202+0000 7fe63b88f640 1 --2- 172.21.15.97:0/126003720 >> v2:172.21.15.97:6800/12494470 conn(0x7fe61c06e270 0x7fe61c070730 secure :-1 s=READY pgs=4047 cs=0 l=1 rev1=1 crypto rx=0x7fe62c006010 tx=0x7fe62c005e40 comp rx=0 tx=0).stop 2024-03-21T14:00:52.203 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.202+0000 7fe63b88f640 1 -- 172.21.15.97:0/126003720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe634198f80 msgr2=0x7fe6341a3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:00:52.203 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.202+0000 7fe63b88f640 1 --2- 172.21.15.97:0/126003720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe634198f80 0x7fe6341a3530 secure :-1 s=READY pgs=3014 cs=0 l=1 rev1=1 crypto rx=0x7fe62400ec50 tx=0x7fe62400c6a0 comp rx=0 tx=0).stop 2024-03-21T14:00:52.203 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.203+0000 7fe63b88f640 1 -- 172.21.15.97:0/126003720 shutdown_connections 2024-03-21T14:00:52.203 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.203+0000 7fe63b88f640 1 --2- 172.21.15.97:0/126003720 >> v2:172.21.15.97:6800/12494470 conn(0x7fe61c06e270 0x7fe61c070730 secure :-1 s=CLOSED pgs=4047 cs=0 l=1 rev1=1 crypto rx=0x7fe62c006010 tx=0x7fe62c005e40 comp rx=0 tx=0).stop 2024-03-21T14:00:52.203 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.203+0000 7fe63b88f640 1 --2- 172.21.15.97:0/126003720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe634198f80 0x7fe6341a3530 unknown :-1 s=CLOSED pgs=3014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:52.203 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.203+0000 7fe63b88f640 1 --2- 172.21.15.97:0/126003720 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe634105020 0x7fe6341989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:52.203 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.203+0000 7fe63b88f640 1 --2- 172.21.15.97:0/126003720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6340fff50 0x7fe6341984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:52.204 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.203+0000 7fe63b88f640 1 -- 172.21.15.97:0/126003720 >> 172.21.15.97:0/126003720 conn(0x7fe634076ab0 msgr2=0x7fe634102980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T14:00:52.204 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.204+0000 7fe63b88f640 1 -- 172.21.15.97:0/126003720 shutdown_connections 2024-03-21T14:00:52.204 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.204+0000 7fe63b88f640 1 -- 172.21.15.97:0/126003720 wait complete. 2024-03-21T14:00:52.217 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph osd pool set-quota b5a5e36b-8518-4a8b-80b9-4898af5bd2dd max_objects 0 2024-03-21T14:00:52.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 14:00:51 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:14:00:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T14:00:52.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:52 smithi097 ceph-mon[30040]: pgmap v1739: 176 pgs: 176 active+clean; 721 KiB data, 918 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T14:00:52.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:52 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/126003720' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "0"}]': finished 2024-03-21T14:00:52.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:52 smithi097 ceph-mon[30040]: osdmap e788: 8 total, 8 up, 8 in 2024-03-21T14:00:52.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:52 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/126003720' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-21T14:00:52.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:52 smithi097 ceph-mon[39416]: pgmap v1739: 176 pgs: 176 active+clean; 721 KiB data, 918 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T14:00:52.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:52 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/126003720' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "0"}]': finished 2024-03-21T14:00:52.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:52 smithi097 ceph-mon[39416]: osdmap e788: 8 total, 8 up, 8 in 2024-03-21T14:00:52.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:52 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/126003720' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-21T14:00:52.308 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.305+0000 7f5b2493e640 1 -- 172.21.15.97:0/2309240408 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5b20100580 msgr2=0x7f5b20100a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:00:52.308 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.305+0000 7f5b2493e640 1 --2- 172.21.15.97:0/2309240408 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5b20100580 0x7f5b20100a00 secure :-1 s=READY pgs=3220 cs=0 l=1 rev1=1 crypto rx=0x7f5b08009930 tx=0x7f5b0802bc30 comp rx=0 tx=0).stop 2024-03-21T14:00:52.308 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.305+0000 7f5b2493e640 1 -- 172.21.15.97:0/2309240408 shutdown_connections 2024-03-21T14:00:52.308 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.305+0000 7f5b2493e640 1 --2- 172.21.15.97:0/2309240408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b20100f40 0x7f5b2010d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:52.308 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.305+0000 7f5b2493e640 1 --2- 172.21.15.97:0/2309240408 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5b20100580 0x7f5b20100a00 unknown :-1 s=CLOSED pgs=3220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:52.308 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.305+0000 7f5b2493e640 1 --2- 172.21.15.97:0/2309240408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b20106640 0x7f5b20106a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:52.308 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.305+0000 7f5b2493e640 1 -- 172.21.15.97:0/2309240408 >> 172.21.15.97:0/2309240408 conn(0x7f5b200776c0 msgr2=0x7f5b20077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T14:00:52.308 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.305+0000 7f5b2493e640 1 -- 172.21.15.97:0/2309240408 shutdown_connections 2024-03-21T14:00:52.308 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.305+0000 7f5b2493e640 1 -- 172.21.15.97:0/2309240408 wait complete. 2024-03-21T14:00:52.308 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.305+0000 7f5b2493e640 1 Processor -- start 2024-03-21T14:00:52.309 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.305+0000 7f5b2493e640 1 -- start start 2024-03-21T14:00:52.309 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.306+0000 7f5b2493e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b20100580 0x7f5b20197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T14:00:52.309 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.306+0000 7f5b2493e640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5b20100f40 0x7f5b20197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T14:00:52.309 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.306+0000 7f5b2493e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b20106640 0x7f5b2019bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T14:00:52.309 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.306+0000 7f5b2493e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b20111150 con 0x7f5b20100580 2024-03-21T14:00:52.309 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.306+0000 7f5b2493e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b20110fd0 con 0x7f5b20106640 2024-03-21T14:00:52.309 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.306+0000 7f5b2493e640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b201112d0 con 0x7f5b20100f40 2024-03-21T14:00:52.309 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.306+0000 7f5b1e59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b20100580 0x7f5b20197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T14:00:52.309 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.306+0000 7f5b17fff640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5b20100f40 0x7f5b20197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T14:00:52.309 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.306+0000 7f5b1e59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b20100580 0x7f5b20197610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37020/0 (socket says 172.21.15.97:37020) 2024-03-21T14:00:52.309 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.306+0000 7f5b1e59c640 1 -- 172.21.15.97:0/2102382318 learned_addr learned my addr 172.21.15.97:0/2102382318 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T14:00:52.309 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.306+0000 7f5b1ed9d640 1 --2- 172.21.15.97:0/2102382318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b20106640 0x7f5b2019bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T14:00:52.309 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.306+0000 7f5b1e59c640 1 -- 172.21.15.97:0/2102382318 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5b20100f40 msgr2=0x7f5b20197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:00:52.310 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.306+0000 7f5b1e59c640 1 --2- 172.21.15.97:0/2102382318 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5b20100f40 0x7f5b20197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:52.310 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.306+0000 7f5b1e59c640 1 -- 172.21.15.97:0/2102382318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b20106640 msgr2=0x7f5b2019bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:00:52.310 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.306+0000 7f5b1e59c640 1 --2- 172.21.15.97:0/2102382318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b20106640 0x7f5b2019bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:52.310 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.306+0000 7f5b1e59c640 1 -- 172.21.15.97:0/2102382318 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b2019c600 con 0x7f5b20100580 2024-03-21T14:00:52.310 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.306+0000 7f5b17fff640 1 --2- 172.21.15.97:0/2102382318 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5b20100f40 0x7f5b20197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T14:00:52.310 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.306+0000 7f5b1e59c640 1 --2- 172.21.15.97:0/2102382318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b20100580 0x7f5b20197610 secure :-1 s=READY pgs=3015 cs=0 l=1 rev1=1 crypto rx=0x7f5b1000c920 tx=0x7f5b1000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T14:00:52.310 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.306+0000 7f5b177fe640 1 -- 172.21.15.97:0/2102382318 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b10007c70 con 0x7f5b20100580 2024-03-21T14:00:52.310 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.306+0000 7f5b177fe640 1 -- 172.21.15.97:0/2102382318 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f5b10007e10 con 0x7f5b20100580 2024-03-21T14:00:52.310 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.306+0000 7f5b2493e640 1 -- 172.21.15.97:0/2102382318 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b2019c8f0 con 0x7f5b20100580 2024-03-21T14:00:52.310 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.306+0000 7f5b177fe640 1 -- 172.21.15.97:0/2102382318 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b100187a0 con 0x7f5b20100580 2024-03-21T14:00:52.310 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.306+0000 7f5b2493e640 1 -- 172.21.15.97:0/2102382318 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b201a4180 con 0x7f5b20100580 2024-03-21T14:00:52.310 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.307+0000 7f5b2493e640 1 -- 172.21.15.97:0/2102382318 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b20068580 con 0x7f5b20100580 2024-03-21T14:00:52.311 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.308+0000 7f5b177fe640 1 -- 172.21.15.97:0/2102382318 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 22) v1 ==== 92152+0+0 (secure 0 0 0) 0x7f5b1000f450 con 0x7f5b20100580 2024-03-21T14:00:52.311 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.308+0000 7f5b177fe640 1 --2- 172.21.15.97:0/2102382318 >> v2:172.21.15.97:6800/12494470 conn(0x7f5b0c06e270 0x7f5b0c070730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T14:00:52.312 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.308+0000 7f5b17fff640 1 --2- 172.21.15.97:0/2102382318 >> v2:172.21.15.97:6800/12494470 conn(0x7f5b0c06e270 0x7f5b0c070730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T14:00:52.312 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.308+0000 7f5b177fe640 1 -- 172.21.15.97:0/2102382318 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(789..789 src has 254..789) v4 ==== 7814+0+0 (secure 0 0 0) 0x7f5b1008eb20 con 0x7f5b20100580 2024-03-21T14:00:52.312 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.309+0000 7f5b177fe640 1 -- 172.21.15.97:0/2102382318 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=790}) v3 -- 0x7f5b0c0797c0 con 0x7f5b20100580 2024-03-21T14:00:52.312 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.309+0000 7f5b17fff640 1 --2- 172.21.15.97:0/2102382318 >> v2:172.21.15.97:6800/12494470 conn(0x7f5b0c06e270 0x7f5b0c070730 secure :-1 s=READY pgs=4048 cs=0 l=1 rev1=1 crypto rx=0x7f5b08005ab0 tx=0x7f5b08005fd0 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T14:00:52.313 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.313+0000 7f5b177fe640 1 -- 172.21.15.97:0/2102382318 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5b10014030 con 0x7f5b20100580 2024-03-21T14:00:52.491 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.490+0000 7f5b2493e640 1 -- 172.21.15.97:0/2102382318 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7f5b20108a50 con 0x7f5b20100580 2024-03-21T14:00:52.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:52 smithi144 ceph-mon[33633]: pgmap v1739: 176 pgs: 176 active+clean; 721 KiB data, 918 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T14:00:52.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:52 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/126003720' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "0"}]': finished 2024-03-21T14:00:52.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:52 smithi144 ceph-mon[33633]: osdmap e788: 8 total, 8 up, 8 in 2024-03-21T14:00:52.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:52 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/126003720' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-21T14:00:52.731 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.731+0000 7f5b177fe640 1 -- 172.21.15.97:0/2102382318 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool b5a5e36b-8518-4a8b-80b9-4898af5bd2dd v790) v1 ==== 221+0+0 (secure 0 0 0) 0x7f5b1005bc60 con 0x7f5b20100580 2024-03-21T14:00:52.732 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.732+0000 7f5b177fe640 1 -- 172.21.15.97:0/2102382318 <== mon.0 v2:172.21.15.97:3300/0 8 ==== osd_map(790..790 src has 254..790) v4 ==== 628+0+0 (secure 0 0 0) 0x7f5b1008c920 con 0x7f5b20100580 2024-03-21T14:00:52.732 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.732+0000 7f5b177fe640 1 -- 172.21.15.97:0/2102382318 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=791}) v3 -- 0x7f5b0c07a670 con 0x7f5b20100580 2024-03-21T14:00:52.841 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:52.841+0000 7f5b2493e640 1 -- 172.21.15.97:0/2102382318 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7f5b201084c0 con 0x7f5b20100580 2024-03-21T14:00:53.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:53 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/126003720' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "0"}]': finished 2024-03-21T14:00:53.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:53 smithi097 ceph-mon[39416]: osdmap e789: 8 total, 8 up, 8 in 2024-03-21T14:00:53.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:53 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2102382318' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "0"}]: dispatch 2024-03-21T14:00:53.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:53 smithi097 ceph-mon[39416]: pool 'b5a5e36b-8518-4a8b-80b9-4898af5bd2dd' no longer out of quota; removing NO_QUOTA flag 2024-03-21T14:00:53.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:53 smithi097 ceph-mon[39416]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-21T14:00:53.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:53 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2102382318' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "0"}]': finished 2024-03-21T14:00:53.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:53 smithi097 ceph-mon[39416]: osdmap e790: 8 total, 8 up, 8 in 2024-03-21T14:00:53.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:53 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2102382318' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "0"}]: dispatch 2024-03-21T14:00:53.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:53 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/126003720' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "0"}]': finished 2024-03-21T14:00:53.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:53 smithi097 ceph-mon[30040]: osdmap e789: 8 total, 8 up, 8 in 2024-03-21T14:00:53.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:53 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2102382318' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "0"}]: dispatch 2024-03-21T14:00:53.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:53 smithi097 ceph-mon[30040]: pool 'b5a5e36b-8518-4a8b-80b9-4898af5bd2dd' no longer out of quota; removing NO_QUOTA flag 2024-03-21T14:00:53.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:53 smithi097 ceph-mon[30040]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-21T14:00:53.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:53 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2102382318' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "0"}]': finished 2024-03-21T14:00:53.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:53 smithi097 ceph-mon[30040]: osdmap e790: 8 total, 8 up, 8 in 2024-03-21T14:00:53.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:53 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2102382318' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "0"}]: dispatch 2024-03-21T14:00:53.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:53 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/126003720' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "0"}]': finished 2024-03-21T14:00:53.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:53 smithi144 ceph-mon[33633]: osdmap e789: 8 total, 8 up, 8 in 2024-03-21T14:00:53.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:53 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2102382318' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "0"}]: dispatch 2024-03-21T14:00:53.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:53 smithi144 ceph-mon[33633]: pool 'b5a5e36b-8518-4a8b-80b9-4898af5bd2dd' no longer out of quota; removing NO_QUOTA flag 2024-03-21T14:00:53.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:53 smithi144 ceph-mon[33633]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-21T14:00:53.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:53 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2102382318' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "0"}]': finished 2024-03-21T14:00:53.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:53 smithi144 ceph-mon[33633]: osdmap e790: 8 total, 8 up, 8 in 2024-03-21T14:00:53.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:53 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2102382318' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "0"}]: dispatch 2024-03-21T14:00:53.737 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.736+0000 7f5b177fe640 1 -- 172.21.15.97:0/2102382318 <== mon.0 v2:172.21.15.97:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool b5a5e36b-8518-4a8b-80b9-4898af5bd2dd v791) v1 ==== 221+0+0 (secure 0 0 0) 0x7f5b1005f6b0 con 0x7f5b20100580 2024-03-21T14:00:53.737 INFO:tasks.workunit.client.0.smithi097.stderr:set-quota max_objects = 0 for pool b5a5e36b-8518-4a8b-80b9-4898af5bd2dd 2024-03-21T14:00:53.740 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.739+0000 7f5b2493e640 1 -- 172.21.15.97:0/2102382318 >> v2:172.21.15.97:6800/12494470 conn(0x7f5b0c06e270 msgr2=0x7f5b0c070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:00:53.740 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.739+0000 7f5b2493e640 1 --2- 172.21.15.97:0/2102382318 >> v2:172.21.15.97:6800/12494470 conn(0x7f5b0c06e270 0x7f5b0c070730 secure :-1 s=READY pgs=4048 cs=0 l=1 rev1=1 crypto rx=0x7f5b08005ab0 tx=0x7f5b08005fd0 comp rx=0 tx=0).stop 2024-03-21T14:00:53.740 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.739+0000 7f5b2493e640 1 -- 172.21.15.97:0/2102382318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b20100580 msgr2=0x7f5b20197610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:00:53.740 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.739+0000 7f5b177fe640 1 -- 172.21.15.97:0/2102382318 <== mon.0 v2:172.21.15.97:3300/0 10 ==== osd_map(791..791 src has 254..791) v4 ==== 628+0+0 (secure 0 0 0) 0x7f5b10059600 con 0x7f5b20100580 2024-03-21T14:00:53.740 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.739+0000 7f5b2493e640 1 --2- 172.21.15.97:0/2102382318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b20100580 0x7f5b20197610 secure :-1 s=READY pgs=3015 cs=0 l=1 rev1=1 crypto rx=0x7f5b1000c920 tx=0x7f5b1000cde0 comp rx=0 tx=0).stop 2024-03-21T14:00:53.740 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.739+0000 7f5b2493e640 1 -- 172.21.15.97:0/2102382318 shutdown_connections 2024-03-21T14:00:53.740 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.739+0000 7f5b2493e640 1 --2- 172.21.15.97:0/2102382318 >> v2:172.21.15.97:6800/12494470 conn(0x7f5b0c06e270 0x7f5b0c070730 unknown :-1 s=CLOSED pgs=4048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:53.740 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.739+0000 7f5b2493e640 1 --2- 172.21.15.97:0/2102382318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b20106640 0x7f5b2019bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:53.740 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.739+0000 7f5b2493e640 1 --2- 172.21.15.97:0/2102382318 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5b20100f40 0x7f5b20197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:53.740 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.739+0000 7f5b2493e640 1 --2- 172.21.15.97:0/2102382318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b20100580 0x7f5b20197610 unknown :-1 s=CLOSED pgs=3015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:53.741 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.739+0000 7f5b2493e640 1 -- 172.21.15.97:0/2102382318 >> 172.21.15.97:0/2102382318 conn(0x7f5b200776c0 msgr2=0x7f5b200ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T14:00:53.741 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.739+0000 7f5b2493e640 1 -- 172.21.15.97:0/2102382318 shutdown_connections 2024-03-21T14:00:53.741 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.740+0000 7f5b2493e640 1 -- 172.21.15.97:0/2102382318 wait complete. 2024-03-21T14:00:53.754 INFO:tasks.workunit.client.0.smithi097.stderr:+ wait 156563 2024-03-21T14:00:53.754 INFO:tasks.workunit.client.0.smithi097.stderr:+ '[' 0 -ne 0 ']' 2024-03-21T14:00:53.754 INFO:tasks.workunit.client.0.smithi097.stderr:+ true 2024-03-21T14:00:53.755 INFO:tasks.workunit.client.0.smithi097.stderr:+ rados -p b5a5e36b-8518-4a8b-80b9-4898af5bd2dd put three /etc/passwd 2024-03-21T14:00:53.792 INFO:tasks.workunit.client.0.smithi097.stderr:++ uuidgen 2024-03-21T14:00:53.793 INFO:tasks.workunit.client.0.smithi097.stderr:+ pp=d7ed4d26-9388-40f6-9c6c-02152ee33b37 2024-03-21T14:00:53.793 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph osd pool create d7ed4d26-9388-40f6-9c6c-02152ee33b37 12 2024-03-21T14:00:53.877 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.874+0000 7fef39c0e640 1 -- 172.21.15.97:0/3336459623 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fef34107680 msgr2=0x7fef34107a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:00:53.878 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.874+0000 7fef39c0e640 1 --2- 172.21.15.97:0/3336459623 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fef34107680 0x7fef34107a60 secure :-1 s=READY pgs=3224 cs=0 l=1 rev1=1 crypto rx=0x7fef28009930 tx=0x7fef2802bc30 comp rx=0 tx=0).stop 2024-03-21T14:00:53.878 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.875+0000 7fef39c0e640 1 -- 172.21.15.97:0/3336459623 shutdown_connections 2024-03-21T14:00:53.878 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.875+0000 7fef39c0e640 1 --2- 172.21.15.97:0/3336459623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef34101670 0x7fef34101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:53.878 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.875+0000 7fef39c0e640 1 --2- 172.21.15.97:0/3336459623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef34102030 0x7fef3410e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:53.878 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.875+0000 7fef39c0e640 1 --2- 172.21.15.97:0/3336459623 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fef34107680 0x7fef34107a60 unknown :-1 s=CLOSED pgs=3224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:53.878 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.875+0000 7fef39c0e640 1 -- 172.21.15.97:0/3336459623 >> 172.21.15.97:0/3336459623 conn(0x7fef340fd1a0 msgr2=0x7fef340ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T14:00:53.878 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.875+0000 7fef39c0e640 1 -- 172.21.15.97:0/3336459623 shutdown_connections 2024-03-21T14:00:53.878 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.875+0000 7fef39c0e640 1 -- 172.21.15.97:0/3336459623 wait complete. 2024-03-21T14:00:53.878 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.875+0000 7fef39c0e640 1 Processor -- start 2024-03-21T14:00:53.878 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.875+0000 7fef39c0e640 1 -- start start 2024-03-21T14:00:53.878 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.875+0000 7fef39c0e640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fef34101670 0x7fef341976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T14:00:53.878 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.875+0000 7fef39c0e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef34102030 0x7fef34197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T14:00:53.879 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.875+0000 7fef39c0e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef34107680 0x7fef3419bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T14:00:53.879 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.875+0000 7fef39c0e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef341111a0 con 0x7fef34102030 2024-03-21T14:00:53.879 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.875+0000 7fef39c0e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef34111020 con 0x7fef34107680 2024-03-21T14:00:53.879 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.875+0000 7fef39c0e640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef34111320 con 0x7fef34101670 2024-03-21T14:00:53.879 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.876+0000 7fef337fe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fef34101670 0x7fef341976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T14:00:53.879 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.876+0000 7fef32ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef34102030 0x7fef34197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T14:00:53.879 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.876+0000 7fef337fe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fef34101670 0x7fef341976a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.97:44082/0 (socket says 172.21.15.97:44082) 2024-03-21T14:00:53.879 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.876+0000 7fef337fe640 1 -- 172.21.15.97:0/200906259 learned_addr learned my addr 172.21.15.97:0/200906259 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T14:00:53.879 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.876+0000 7fef33fff640 1 --2- 172.21.15.97:0/200906259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef34107680 0x7fef3419bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T14:00:53.879 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.876+0000 7fef337fe640 1 -- 172.21.15.97:0/200906259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef34107680 msgr2=0x7fef3419bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:00:53.879 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.876+0000 7fef337fe640 1 --2- 172.21.15.97:0/200906259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef34107680 0x7fef3419bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:53.879 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.876+0000 7fef337fe640 1 -- 172.21.15.97:0/200906259 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef34102030 msgr2=0x7fef34197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:00:53.880 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.876+0000 7fef337fe640 1 --2- 172.21.15.97:0/200906259 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef34102030 0x7fef34197be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:53.880 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.876+0000 7fef337fe640 1 -- 172.21.15.97:0/200906259 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef3419c690 con 0x7fef34101670 2024-03-21T14:00:53.880 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.876+0000 7fef33fff640 1 --2- 172.21.15.97:0/200906259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef34107680 0x7fef3419bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T14:00:53.880 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.876+0000 7fef32ffd640 1 --2- 172.21.15.97:0/200906259 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef34102030 0x7fef34197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T14:00:53.880 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.876+0000 7fef337fe640 1 --2- 172.21.15.97:0/200906259 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fef34101670 0x7fef341976a0 secure :-1 s=READY pgs=3225 cs=0 l=1 rev1=1 crypto rx=0x7fef28009230 tx=0x7fef2802c790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T14:00:53.880 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.876+0000 7fef30ff9640 1 -- 172.21.15.97:0/200906259 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef28043020 con 0x7fef34101670 2024-03-21T14:00:53.880 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.876+0000 7fef39c0e640 1 -- 172.21.15.97:0/200906259 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef3419c920 con 0x7fef34101670 2024-03-21T14:00:53.880 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.876+0000 7fef30ff9640 1 -- 172.21.15.97:0/200906259 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fef280043d0 con 0x7fef34101670 2024-03-21T14:00:53.880 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.876+0000 7fef30ff9640 1 -- 172.21.15.97:0/200906259 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef28003b70 con 0x7fef34101670 2024-03-21T14:00:53.881 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.876+0000 7fef39c0e640 1 -- 172.21.15.97:0/200906259 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef341a4210 con 0x7fef34101670 2024-03-21T14:00:53.881 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.877+0000 7fef39c0e640 1 -- 172.21.15.97:0/200906259 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef340689d0 con 0x7fef34101670 2024-03-21T14:00:53.882 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.878+0000 7fef30ff9640 1 -- 172.21.15.97:0/200906259 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 22) v1 ==== 92152+0+0 (secure 0 0 0) 0x7fef28046020 con 0x7fef34101670 2024-03-21T14:00:53.882 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.878+0000 7fef30ff9640 1 --2- 172.21.15.97:0/200906259 >> v2:172.21.15.97:6800/12494470 conn(0x7fef1c06e270 0x7fef1c070730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T14:00:53.882 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.878+0000 7fef32ffd640 1 --2- 172.21.15.97:0/200906259 >> v2:172.21.15.97:6800/12494470 conn(0x7fef1c06e270 0x7fef1c070730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T14:00:53.882 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.878+0000 7fef30ff9640 1 -- 172.21.15.97:0/200906259 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(791..791 src has 254..791) v4 ==== 7814+0+0 (secure 0 0 0) 0x7fef280b0550 con 0x7fef34101670 2024-03-21T14:00:53.882 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.878+0000 7fef32ffd640 1 --2- 172.21.15.97:0/200906259 >> v2:172.21.15.97:6800/12494470 conn(0x7fef1c06e270 0x7fef1c070730 secure :-1 s=READY pgs=4050 cs=0 l=1 rev1=1 crypto rx=0x7fef240095f0 tx=0x7fef24005f90 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T14:00:53.883 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:53.883+0000 7fef30ff9640 1 -- 172.21.15.97:0/200906259 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fef2804d070 con 0x7fef34101670 2024-03-21T14:00:54.059 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.059+0000 7fef39c0e640 1 -- 172.21.15.97:0/200906259 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "pg_num": 12} v 0) v1 -- 0x7fef34109be0 con 0x7fef34101670 2024-03-21T14:00:54.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:54 smithi097 ceph-mon[30040]: pgmap v1743: 176 pgs: 176 active+clean; 721 KiB data, 918 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:00:54.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:54 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2102382318' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "0"}]': finished 2024-03-21T14:00:54.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:54 smithi097 ceph-mon[30040]: osdmap e791: 8 total, 8 up, 8 in 2024-03-21T14:00:54.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:54 smithi097 ceph-mon[39416]: pgmap v1743: 176 pgs: 176 active+clean; 721 KiB data, 918 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:00:54.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:54 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2102382318' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "0"}]': finished 2024-03-21T14:00:54.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:54 smithi097 ceph-mon[39416]: osdmap e791: 8 total, 8 up, 8 in 2024-03-21T14:00:54.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:54 smithi144 ceph-mon[33633]: pgmap v1743: 176 pgs: 176 active+clean; 721 KiB data, 918 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:00:54.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:54 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2102382318' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "0"}]': finished 2024-03-21T14:00:54.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:54 smithi144 ceph-mon[33633]: osdmap e791: 8 total, 8 up, 8 in 2024-03-21T14:00:54.743 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.742+0000 7fef30ff9640 1 -- 172.21.15.97:0/200906259 <== mon.2 v2:172.21.15.97:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "pg_num": 12}]=0 pool 'd7ed4d26-9388-40f6-9c6c-02152ee33b37' created v792) v1 ==== 176+0+0 (secure 0 0 0) 0x7fef2807d690 con 0x7fef34101670 2024-03-21T14:00:54.853 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.853+0000 7fef39c0e640 1 -- 172.21.15.97:0/200906259 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "pg_num": 12} v 0) v1 -- 0x7fef34109770 con 0x7fef34101670 2024-03-21T14:00:54.854 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.853+0000 7fef30ff9640 1 -- 172.21.15.97:0/200906259 <== mon.2 v2:172.21.15.97:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "pg_num": 12}]=0 pool 'd7ed4d26-9388-40f6-9c6c-02152ee33b37' already exists v792) v1 ==== 183+0+0 (secure 0 0 0) 0x7fef280810e0 con 0x7fef34101670 2024-03-21T14:00:54.854 INFO:tasks.workunit.client.0.smithi097.stderr:pool 'd7ed4d26-9388-40f6-9c6c-02152ee33b37' already exists 2024-03-21T14:00:54.856 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.855+0000 7fef39c0e640 1 -- 172.21.15.97:0/200906259 >> v2:172.21.15.97:6800/12494470 conn(0x7fef1c06e270 msgr2=0x7fef1c070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:00:54.856 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.855+0000 7fef39c0e640 1 --2- 172.21.15.97:0/200906259 >> v2:172.21.15.97:6800/12494470 conn(0x7fef1c06e270 0x7fef1c070730 secure :-1 s=READY pgs=4050 cs=0 l=1 rev1=1 crypto rx=0x7fef240095f0 tx=0x7fef24005f90 comp rx=0 tx=0).stop 2024-03-21T14:00:54.856 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.855+0000 7fef39c0e640 1 -- 172.21.15.97:0/200906259 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fef34101670 msgr2=0x7fef341976a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:00:54.856 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.855+0000 7fef39c0e640 1 --2- 172.21.15.97:0/200906259 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fef34101670 0x7fef341976a0 secure :-1 s=READY pgs=3225 cs=0 l=1 rev1=1 crypto rx=0x7fef28009230 tx=0x7fef2802c790 comp rx=0 tx=0).stop 2024-03-21T14:00:54.856 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.855+0000 7fef39c0e640 1 -- 172.21.15.97:0/200906259 shutdown_connections 2024-03-21T14:00:54.856 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.855+0000 7fef39c0e640 1 --2- 172.21.15.97:0/200906259 >> v2:172.21.15.97:6800/12494470 conn(0x7fef1c06e270 0x7fef1c070730 unknown :-1 s=CLOSED pgs=4050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:54.856 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.855+0000 7fef39c0e640 1 --2- 172.21.15.97:0/200906259 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef34102030 0x7fef34197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:54.856 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.855+0000 7fef39c0e640 1 --2- 172.21.15.97:0/200906259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef34107680 0x7fef3419bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:54.856 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.855+0000 7fef39c0e640 1 --2- 172.21.15.97:0/200906259 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fef34101670 0x7fef341976a0 unknown :-1 s=CLOSED pgs=3225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:54.856 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.855+0000 7fef39c0e640 1 -- 172.21.15.97:0/200906259 >> 172.21.15.97:0/200906259 conn(0x7fef340fd1a0 msgr2=0x7fef340fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T14:00:54.856 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.856+0000 7fef39c0e640 1 -- 172.21.15.97:0/200906259 shutdown_connections 2024-03-21T14:00:54.857 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.856+0000 7fef39c0e640 1 -- 172.21.15.97:0/200906259 wait complete. 2024-03-21T14:00:54.865 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph osd pool application enable d7ed4d26-9388-40f6-9c6c-02152ee33b37 rados 2024-03-21T14:00:54.952 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.947+0000 7fab43dd0640 1 -- 172.21.15.97:0/3517572489 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fab3c103230 msgr2=0x7fab3c10b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:00:54.953 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.947+0000 7fab43dd0640 1 --2- 172.21.15.97:0/3517572489 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fab3c103230 0x7fab3c10b710 secure :-1 s=READY pgs=3226 cs=0 l=1 rev1=1 crypto rx=0x7fab2c009860 tx=0x7fab2c02bca0 comp rx=0 tx=0).stop 2024-03-21T14:00:54.953 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.947+0000 7fab43dd0640 1 -- 172.21.15.97:0/3517572489 shutdown_connections 2024-03-21T14:00:54.953 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.947+0000 7fab43dd0640 1 --2- 172.21.15.97:0/3517572489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab3c10bc50 0x7fab3c10e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:54.953 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.947+0000 7fab43dd0640 1 --2- 172.21.15.97:0/3517572489 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fab3c103230 0x7fab3c10b710 unknown :-1 s=CLOSED pgs=3226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:54.953 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.947+0000 7fab43dd0640 1 --2- 172.21.15.97:0/3517572489 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab3c102910 0x7fab3c102cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:54.953 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.947+0000 7fab43dd0640 1 -- 172.21.15.97:0/3517572489 >> 172.21.15.97:0/3517572489 conn(0x7fab3c0776c0 msgr2=0x7fab3c077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T14:00:54.953 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.947+0000 7fab43dd0640 1 -- 172.21.15.97:0/3517572489 shutdown_connections 2024-03-21T14:00:54.953 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.948+0000 7fab43dd0640 1 -- 172.21.15.97:0/3517572489 wait complete. 2024-03-21T14:00:54.953 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.948+0000 7fab43dd0640 1 Processor -- start 2024-03-21T14:00:54.953 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.948+0000 7fab43dd0640 1 -- start start 2024-03-21T14:00:54.953 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.948+0000 7fab43dd0640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fab3c102910 0x7fab3c197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T14:00:54.953 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.948+0000 7fab43dd0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab3c103230 0x7fab3c197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T14:00:54.953 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.948+0000 7fab43dd0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab3c10bc50 0x7fab3c19bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T14:00:54.954 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.948+0000 7fab43dd0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3c111160 con 0x7fab3c10bc50 2024-03-21T14:00:54.954 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.948+0000 7fab43dd0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3c110fe0 con 0x7fab3c103230 2024-03-21T14:00:54.954 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.948+0000 7fab43dd0640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab3c1112e0 con 0x7fab3c102910 2024-03-21T14:00:54.954 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.948+0000 7fab41b6c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fab3c102910 0x7fab3c197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T14:00:54.954 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.948+0000 7fab41b6c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fab3c102910 0x7fab3c197690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.97:44106/0 (socket says 172.21.15.97:44106) 2024-03-21T14:00:54.954 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.948+0000 7fab4236d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab3c10bc50 0x7fab3c19bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T14:00:54.954 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.948+0000 7fab41b6c640 1 -- 172.21.15.97:0/1253210109 learned_addr learned my addr 172.21.15.97:0/1253210109 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T14:00:54.954 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.949+0000 7fab4136b640 1 --2- 172.21.15.97:0/1253210109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab3c103230 0x7fab3c197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T14:00:54.954 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.949+0000 7fab41b6c640 1 -- 172.21.15.97:0/1253210109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab3c103230 msgr2=0x7fab3c197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:00:54.954 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.949+0000 7fab41b6c640 1 --2- 172.21.15.97:0/1253210109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab3c103230 0x7fab3c197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:54.954 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.949+0000 7fab41b6c640 1 -- 172.21.15.97:0/1253210109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab3c10bc50 msgr2=0x7fab3c19bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:00:54.954 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.949+0000 7fab41b6c640 1 --2- 172.21.15.97:0/1253210109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab3c10bc50 0x7fab3c19bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:54.954 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.949+0000 7fab41b6c640 1 -- 172.21.15.97:0/1253210109 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab3c19c630 con 0x7fab3c102910 2024-03-21T14:00:54.955 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.949+0000 7fab4236d640 1 --2- 172.21.15.97:0/1253210109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab3c10bc50 0x7fab3c19bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T14:00:54.955 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.949+0000 7fab41b6c640 1 --2- 172.21.15.97:0/1253210109 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fab3c102910 0x7fab3c197690 secure :-1 s=READY pgs=3227 cs=0 l=1 rev1=1 crypto rx=0x7fab3800e9c0 tx=0x7fab3800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T14:00:54.955 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.949+0000 7fab32ffd640 1 -- 172.21.15.97:0/1253210109 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3800cde0 con 0x7fab3c102910 2024-03-21T14:00:54.955 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.949+0000 7fab32ffd640 1 -- 172.21.15.97:0/1253210109 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fab380035f0 con 0x7fab3c102910 2024-03-21T14:00:54.955 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.949+0000 7fab43dd0640 1 -- 172.21.15.97:0/1253210109 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab3c19c920 con 0x7fab3c102910 2024-03-21T14:00:54.955 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.949+0000 7fab32ffd640 1 -- 172.21.15.97:0/1253210109 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab38010690 con 0x7fab3c102910 2024-03-21T14:00:54.955 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.949+0000 7fab43dd0640 1 -- 172.21.15.97:0/1253210109 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab3c1a41b0 con 0x7fab3c102910 2024-03-21T14:00:54.955 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.950+0000 7fab43dd0640 1 -- 172.21.15.97:0/1253210109 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab3c068580 con 0x7fab3c102910 2024-03-21T14:00:54.955 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.950+0000 7fab32ffd640 1 -- 172.21.15.97:0/1253210109 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 22) v1 ==== 92152+0+0 (secure 0 0 0) 0x7fab38004000 con 0x7fab3c102910 2024-03-21T14:00:54.955 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.951+0000 7fab32ffd640 1 --2- 172.21.15.97:0/1253210109 >> v2:172.21.15.97:6800/12494470 conn(0x7fab2806e270 0x7fab28070730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T14:00:54.955 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.951+0000 7fab4136b640 1 --2- 172.21.15.97:0/1253210109 >> v2:172.21.15.97:6800/12494470 conn(0x7fab2806e270 0x7fab28070730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T14:00:54.955 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.951+0000 7fab32ffd640 1 -- 172.21.15.97:0/1253210109 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(792..792 src has 254..792) v4 ==== 8189+0+0 (secure 0 0 0) 0x7fab3808ee90 con 0x7fab3c102910 2024-03-21T14:00:54.955 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.951+0000 7fab4136b640 1 --2- 172.21.15.97:0/1253210109 >> v2:172.21.15.97:6800/12494470 conn(0x7fab2806e270 0x7fab28070730 secure :-1 s=READY pgs=4051 cs=0 l=1 rev1=1 crypto rx=0x7fab2c02c2b0 tx=0x7fab2c0058d0 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T14:00:54.956 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:54.955+0000 7fab32ffd640 1 -- 172.21.15.97:0/1253210109 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fab3805f860 con 0x7fab3c102910 2024-03-21T14:00:55.132 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:55.132+0000 7fab43dd0640 1 -- 172.21.15.97:0/1253210109 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "app": "rados"} v 0) v1 -- 0x7fab3c1064e0 con 0x7fab3c102910 2024-03-21T14:00:55.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:55 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/200906259' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "pg_num": 12}]: dispatch 2024-03-21T14:00:55.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:55 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "pg_num": 12}]: dispatch 2024-03-21T14:00:55.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:55 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "pg_num": 12}]': finished 2024-03-21T14:00:55.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:55 smithi097 ceph-mon[39416]: osdmap e792: 8 total, 8 up, 8 in 2024-03-21T14:00:55.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:55 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/200906259' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "pg_num": 12}]: dispatch 2024-03-21T14:00:55.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:55 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "pg_num": 12}]: dispatch 2024-03-21T14:00:55.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:55 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1253210109' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "app": "rados"}]: dispatch 2024-03-21T14:00:55.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:55 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "app": "rados"}]: dispatch 2024-03-21T14:00:55.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:55 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/200906259' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "pg_num": 12}]: dispatch 2024-03-21T14:00:55.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:55 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "pg_num": 12}]: dispatch 2024-03-21T14:00:55.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:55 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "pg_num": 12}]': finished 2024-03-21T14:00:55.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:55 smithi097 ceph-mon[30040]: osdmap e792: 8 total, 8 up, 8 in 2024-03-21T14:00:55.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:55 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/200906259' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "pg_num": 12}]: dispatch 2024-03-21T14:00:55.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:55 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "pg_num": 12}]: dispatch 2024-03-21T14:00:55.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:55 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1253210109' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "app": "rados"}]: dispatch 2024-03-21T14:00:55.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:55 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "app": "rados"}]: dispatch 2024-03-21T14:00:55.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 14:00:55 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T14:00:55.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:55 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/200906259' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "pg_num": 12}]: dispatch 2024-03-21T14:00:55.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:55 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "pg_num": 12}]: dispatch 2024-03-21T14:00:55.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:55 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "pg_num": 12}]': finished 2024-03-21T14:00:55.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:55 smithi144 ceph-mon[33633]: osdmap e792: 8 total, 8 up, 8 in 2024-03-21T14:00:55.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:55 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/200906259' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "pg_num": 12}]: dispatch 2024-03-21T14:00:55.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:55 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "pg_num": 12}]: dispatch 2024-03-21T14:00:55.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:55 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1253210109' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "app": "rados"}]: dispatch 2024-03-21T14:00:55.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:55 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "app": "rados"}]: dispatch 2024-03-21T14:00:55.751 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:55.750+0000 7fab32ffd640 1 -- 172.21.15.97:0/1253210109 <== mon.2 v2:172.21.15.97:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "app": "rados"}]=0 enabled application 'rados' on pool 'd7ed4d26-9388-40f6-9c6c-02152ee33b37' v793) v1 ==== 213+0+0 (secure 0 0 0) 0x7fab38064090 con 0x7fab3c102910 2024-03-21T14:00:55.861 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:55.861+0000 7fab43dd0640 1 -- 172.21.15.97:0/1253210109 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "app": "rados"} v 0) v1 -- 0x7fab3c105f50 con 0x7fab3c102910 2024-03-21T14:00:56.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:56 smithi097 ceph-mon[39416]: pgmap v1746: 188 pgs: 12 unknown, 176 active+clean; 732 KiB data, 918 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 3.1 KiB/s wr, 1 op/s 2024-03-21T14:00:56.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:56 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T14:00:56.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:56 smithi097 ceph-mon[39416]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T14:00:56.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:56 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "app": "rados"}]': finished 2024-03-21T14:00:56.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:56 smithi097 ceph-mon[39416]: osdmap e793: 8 total, 8 up, 8 in 2024-03-21T14:00:56.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:56 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1253210109' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "app": "rados"}]: dispatch 2024-03-21T14:00:56.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:56 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "app": "rados"}]: dispatch 2024-03-21T14:00:56.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:56 smithi097 ceph-mon[30040]: pgmap v1746: 188 pgs: 12 unknown, 176 active+clean; 732 KiB data, 918 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 3.1 KiB/s wr, 1 op/s 2024-03-21T14:00:56.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:56 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T14:00:56.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:56 smithi097 ceph-mon[30040]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T14:00:56.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:56 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "app": "rados"}]': finished 2024-03-21T14:00:56.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:56 smithi097 ceph-mon[30040]: osdmap e793: 8 total, 8 up, 8 in 2024-03-21T14:00:56.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:56 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1253210109' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "app": "rados"}]: dispatch 2024-03-21T14:00:56.556 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:56 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "app": "rados"}]: dispatch 2024-03-21T14:00:56.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:56 smithi144 ceph-mon[33633]: pgmap v1746: 188 pgs: 12 unknown, 176 active+clean; 732 KiB data, 918 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 3.1 KiB/s wr, 1 op/s 2024-03-21T14:00:56.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:56 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T14:00:56.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:56 smithi144 ceph-mon[33633]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T14:00:56.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:56 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "app": "rados"}]': finished 2024-03-21T14:00:56.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:56 smithi144 ceph-mon[33633]: osdmap e793: 8 total, 8 up, 8 in 2024-03-21T14:00:56.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:56 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1253210109' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "app": "rados"}]: dispatch 2024-03-21T14:00:56.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:56 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "app": "rados"}]: dispatch 2024-03-21T14:00:56.755 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.754+0000 7fab32ffd640 1 -- 172.21.15.97:0/1253210109 <== mon.2 v2:172.21.15.97:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "app": "rados"}]=0 enabled application 'rados' on pool 'd7ed4d26-9388-40f6-9c6c-02152ee33b37' v794) v1 ==== 213+0+0 (secure 0 0 0) 0x7fab38003a80 con 0x7fab3c102910 2024-03-21T14:00:56.755 INFO:tasks.workunit.client.0.smithi097.stderr:enabled application 'rados' on pool 'd7ed4d26-9388-40f6-9c6c-02152ee33b37' 2024-03-21T14:00:56.757 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.757+0000 7fab43dd0640 1 -- 172.21.15.97:0/1253210109 >> v2:172.21.15.97:6800/12494470 conn(0x7fab2806e270 msgr2=0x7fab28070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:00:56.757 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.757+0000 7fab43dd0640 1 --2- 172.21.15.97:0/1253210109 >> v2:172.21.15.97:6800/12494470 conn(0x7fab2806e270 0x7fab28070730 secure :-1 s=READY pgs=4051 cs=0 l=1 rev1=1 crypto rx=0x7fab2c02c2b0 tx=0x7fab2c0058d0 comp rx=0 tx=0).stop 2024-03-21T14:00:56.757 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.757+0000 7fab43dd0640 1 -- 172.21.15.97:0/1253210109 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fab3c102910 msgr2=0x7fab3c197690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:00:56.757 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.757+0000 7fab43dd0640 1 --2- 172.21.15.97:0/1253210109 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fab3c102910 0x7fab3c197690 secure :-1 s=READY pgs=3227 cs=0 l=1 rev1=1 crypto rx=0x7fab3800e9c0 tx=0x7fab3800ee80 comp rx=0 tx=0).stop 2024-03-21T14:00:56.757 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.757+0000 7fab43dd0640 1 -- 172.21.15.97:0/1253210109 shutdown_connections 2024-03-21T14:00:56.757 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.757+0000 7fab43dd0640 1 --2- 172.21.15.97:0/1253210109 >> v2:172.21.15.97:6800/12494470 conn(0x7fab2806e270 0x7fab28070730 unknown :-1 s=CLOSED pgs=4051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:56.758 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.757+0000 7fab43dd0640 1 --2- 172.21.15.97:0/1253210109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab3c10bc50 0x7fab3c19bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:56.758 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.757+0000 7fab43dd0640 1 --2- 172.21.15.97:0/1253210109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab3c103230 0x7fab3c197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:56.758 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.757+0000 7fab43dd0640 1 --2- 172.21.15.97:0/1253210109 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fab3c102910 0x7fab3c197690 unknown :-1 s=CLOSED pgs=3227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:56.758 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.757+0000 7fab43dd0640 1 -- 172.21.15.97:0/1253210109 >> 172.21.15.97:0/1253210109 conn(0x7fab3c0776c0 msgr2=0x7fab3c1036b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T14:00:56.758 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.757+0000 7fab43dd0640 1 -- 172.21.15.97:0/1253210109 shutdown_connections 2024-03-21T14:00:56.758 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.757+0000 7fab43dd0640 1 -- 172.21.15.97:0/1253210109 wait complete. 2024-03-21T14:00:56.768 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph osd pool set-quota d7ed4d26-9388-40f6-9c6c-02152ee33b37 max_objects 10 2024-03-21T14:00:56.862 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.858+0000 7f4a5b00e640 1 -- 172.21.15.97:0/336137212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a54106640 msgr2=0x7f4a54106a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:00:56.862 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.858+0000 7f4a5b00e640 1 --2- 172.21.15.97:0/336137212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a54106640 0x7f4a54106a20 secure :-1 s=READY pgs=3016 cs=0 l=1 rev1=1 crypto rx=0x7f4a44009860 tx=0x7f4a4402bb70 comp rx=0 tx=0).stop 2024-03-21T14:00:56.862 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.859+0000 7f4a5b00e640 1 -- 172.21.15.97:0/336137212 shutdown_connections 2024-03-21T14:00:56.862 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.859+0000 7f4a5b00e640 1 --2- 172.21.15.97:0/336137212 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4a54100f40 0x7f4a5410d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:56.862 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.859+0000 7f4a5b00e640 1 --2- 172.21.15.97:0/336137212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a54100580 0x7f4a54100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:56.862 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.859+0000 7f4a5b00e640 1 --2- 172.21.15.97:0/336137212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a54106640 0x7f4a54106a20 unknown :-1 s=CLOSED pgs=3016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:56.862 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.859+0000 7f4a5b00e640 1 -- 172.21.15.97:0/336137212 >> 172.21.15.97:0/336137212 conn(0x7f4a540776c0 msgr2=0x7f4a54077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T14:00:56.862 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.859+0000 7f4a5b00e640 1 -- 172.21.15.97:0/336137212 shutdown_connections 2024-03-21T14:00:56.862 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.859+0000 7f4a5b00e640 1 -- 172.21.15.97:0/336137212 wait complete. 2024-03-21T14:00:56.862 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.859+0000 7f4a5b00e640 1 Processor -- start 2024-03-21T14:00:56.863 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.859+0000 7f4a5b00e640 1 -- start start 2024-03-21T14:00:56.863 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.859+0000 7f4a5b00e640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4a54100580 0x7f4a541975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T14:00:56.863 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.859+0000 7f4a5b00e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a54100f40 0x7f4a54197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T14:00:56.863 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.859+0000 7f4a5b00e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a54106640 0x7f4a5419beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T14:00:56.863 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.859+0000 7f4a5b00e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a54111150 con 0x7f4a54106640 2024-03-21T14:00:56.863 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.859+0000 7f4a5b00e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a54110fd0 con 0x7f4a54100f40 2024-03-21T14:00:56.863 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.859+0000 7f4a5b00e640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a541112d0 con 0x7f4a54100580 2024-03-21T14:00:56.863 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.859+0000 7f4a58daa640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4a54100580 0x7f4a541975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T14:00:56.863 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.859+0000 7f4a595ab640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a54106640 0x7f4a5419beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T14:00:56.863 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.859+0000 7f4a58daa640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4a54100580 0x7f4a541975f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.97:44122/0 (socket says 172.21.15.97:44122) 2024-03-21T14:00:56.863 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.859+0000 7f4a58daa640 1 -- 172.21.15.97:0/1185692300 learned_addr learned my addr 172.21.15.97:0/1185692300 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T14:00:56.863 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.860+0000 7f4a53fff640 1 --2- 172.21.15.97:0/1185692300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a54100f40 0x7f4a54197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T14:00:56.863 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.860+0000 7f4a58daa640 1 -- 172.21.15.97:0/1185692300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a54100f40 msgr2=0x7f4a54197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:00:56.863 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.860+0000 7f4a58daa640 1 --2- 172.21.15.97:0/1185692300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a54100f40 0x7f4a54197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:56.863 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.860+0000 7f4a58daa640 1 -- 172.21.15.97:0/1185692300 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a54106640 msgr2=0x7f4a5419beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:00:56.864 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.860+0000 7f4a58daa640 1 --2- 172.21.15.97:0/1185692300 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a54106640 0x7f4a5419beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:56.864 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.860+0000 7f4a58daa640 1 -- 172.21.15.97:0/1185692300 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a5419c5e0 con 0x7f4a54100580 2024-03-21T14:00:56.864 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.860+0000 7f4a595ab640 1 --2- 172.21.15.97:0/1185692300 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a54106640 0x7f4a5419beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T14:00:56.864 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.860+0000 7f4a58daa640 1 --2- 172.21.15.97:0/1185692300 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4a54100580 0x7f4a541975f0 secure :-1 s=READY pgs=3228 cs=0 l=1 rev1=1 crypto rx=0x7f4a44037040 tx=0x7f4a440040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T14:00:56.864 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.860+0000 7f4a51ffb640 1 -- 172.21.15.97:0/1185692300 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a44041020 con 0x7f4a54100580 2024-03-21T14:00:56.864 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.860+0000 7f4a5b00e640 1 -- 172.21.15.97:0/1185692300 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a5419c870 con 0x7f4a54100580 2024-03-21T14:00:56.864 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.860+0000 7f4a51ffb640 1 -- 172.21.15.97:0/1185692300 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f4a440044b0 con 0x7f4a54100580 2024-03-21T14:00:56.864 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.860+0000 7f4a51ffb640 1 -- 172.21.15.97:0/1185692300 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a44039040 con 0x7f4a54100580 2024-03-21T14:00:56.864 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.860+0000 7f4a5b00e640 1 -- 172.21.15.97:0/1185692300 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a541a4160 con 0x7f4a54100580 2024-03-21T14:00:56.864 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.861+0000 7f4a5b00e640 1 -- 172.21.15.97:0/1185692300 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a54068580 con 0x7f4a54100580 2024-03-21T14:00:56.864 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.862+0000 7f4a51ffb640 1 -- 172.21.15.97:0/1185692300 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 22) v1 ==== 92152+0+0 (secure 0 0 0) 0x7f4a44044020 con 0x7f4a54100580 2024-03-21T14:00:56.865 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.862+0000 7f4a51ffb640 1 --2- 172.21.15.97:0/1185692300 >> v2:172.21.15.97:6800/12494470 conn(0x7f4a3c06e270 0x7f4a3c070730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T14:00:56.865 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.862+0000 7f4a51ffb640 1 -- 172.21.15.97:0/1185692300 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(794..794 src has 254..794) v4 ==== 8202+0+0 (secure 0 0 0) 0x7f4a440b0450 con 0x7f4a54100580 2024-03-21T14:00:56.865 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.862+0000 7f4a53fff640 1 --2- 172.21.15.97:0/1185692300 >> v2:172.21.15.97:6800/12494470 conn(0x7f4a3c06e270 0x7f4a3c070730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T14:00:56.866 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.862+0000 7f4a53fff640 1 --2- 172.21.15.97:0/1185692300 >> v2:172.21.15.97:6800/12494470 conn(0x7f4a3c06e270 0x7f4a3c070730 secure :-1 s=READY pgs=4052 cs=0 l=1 rev1=1 crypto rx=0x7f4a54077ca0 tx=0x7f4a4800b390 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T14:00:56.867 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:56.867+0000 7f4a51ffb640 1 -- 172.21.15.97:0/1185692300 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4a4404b020 con 0x7f4a54100580 2024-03-21T14:00:57.043 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:57.042+0000 7f4a5b00e640 1 -- 172.21.15.97:0/1185692300 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7f4a54108a30 con 0x7f4a54100580 2024-03-21T14:00:57.803 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:57.803+0000 7f4a51ffb640 1 -- 172.21.15.97:0/1185692300 <== mon.2 v2:172.21.15.97:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool d7ed4d26-9388-40f6-9c6c-02152ee33b37 v795) v1 ==== 223+0+0 (secure 0 0 0) 0x7f4a44080e10 con 0x7f4a54100580 2024-03-21T14:00:57.914 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:57.914+0000 7f4a5b00e640 1 -- 172.21.15.97:0/1185692300 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7f4a5402d200 con 0x7f4a54100580 2024-03-21T14:00:58.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:57 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "app": "rados"}]': finished 2024-03-21T14:00:58.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:57 smithi097 ceph-mon[39416]: osdmap e794: 8 total, 8 up, 8 in 2024-03-21T14:00:58.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:57 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T14:00:58.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:57 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T14:00:58.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:57 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1185692300' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "field": "max_objects", "val": "10"}]: dispatch 2024-03-21T14:00:58.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:57 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "field": "max_objects", "val": "10"}]: dispatch 2024-03-21T14:00:58.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:57 smithi097 ceph-mon[39416]: pgmap v1749: 188 pgs: 12 unknown, 176 active+clean; 732 KiB data, 918 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-03-21T14:00:58.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:57 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "app": "rados"}]': finished 2024-03-21T14:00:58.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:57 smithi097 ceph-mon[30040]: osdmap e794: 8 total, 8 up, 8 in 2024-03-21T14:00:58.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:57 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T14:00:58.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:57 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T14:00:58.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:57 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1185692300' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "field": "max_objects", "val": "10"}]: dispatch 2024-03-21T14:00:58.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:57 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "field": "max_objects", "val": "10"}]: dispatch 2024-03-21T14:00:58.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:57 smithi097 ceph-mon[30040]: pgmap v1749: 188 pgs: 12 unknown, 176 active+clean; 732 KiB data, 918 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-03-21T14:00:58.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:57 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "app": "rados"}]': finished 2024-03-21T14:00:58.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:57 smithi144 ceph-mon[33633]: osdmap e794: 8 total, 8 up, 8 in 2024-03-21T14:00:58.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:57 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T14:00:58.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:57 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T14:00:58.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:57 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1185692300' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "field": "max_objects", "val": "10"}]: dispatch 2024-03-21T14:00:58.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:57 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "field": "max_objects", "val": "10"}]: dispatch 2024-03-21T14:00:58.122 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:57 smithi144 ceph-mon[33633]: pgmap v1749: 188 pgs: 12 unknown, 176 active+clean; 732 KiB data, 918 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-03-21T14:00:58.807 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:58.807+0000 7f4a51ffb640 1 -- 172.21.15.97:0/1185692300 <== mon.2 v2:172.21.15.97:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool d7ed4d26-9388-40f6-9c6c-02152ee33b37 v796) v1 ==== 223+0+0 (secure 0 0 0) 0x7f4a44085640 con 0x7f4a54100580 2024-03-21T14:00:58.808 INFO:tasks.workunit.client.0.smithi097.stderr:set-quota max_objects = 10 for pool d7ed4d26-9388-40f6-9c6c-02152ee33b37 2024-03-21T14:00:58.810 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:58.810+0000 7f4a5b00e640 1 -- 172.21.15.97:0/1185692300 >> v2:172.21.15.97:6800/12494470 conn(0x7f4a3c06e270 msgr2=0x7f4a3c070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:00:58.810 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:58.810+0000 7f4a5b00e640 1 --2- 172.21.15.97:0/1185692300 >> v2:172.21.15.97:6800/12494470 conn(0x7f4a3c06e270 0x7f4a3c070730 secure :-1 s=READY pgs=4052 cs=0 l=1 rev1=1 crypto rx=0x7f4a54077ca0 tx=0x7f4a4800b390 comp rx=0 tx=0).stop 2024-03-21T14:00:58.810 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:58.810+0000 7f4a5b00e640 1 -- 172.21.15.97:0/1185692300 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4a54100580 msgr2=0x7f4a541975f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:00:58.810 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:58.810+0000 7f4a5b00e640 1 --2- 172.21.15.97:0/1185692300 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4a54100580 0x7f4a541975f0 secure :-1 s=READY pgs=3228 cs=0 l=1 rev1=1 crypto rx=0x7f4a44037040 tx=0x7f4a440040c0 comp rx=0 tx=0).stop 2024-03-21T14:00:58.811 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:58.810+0000 7f4a5b00e640 1 -- 172.21.15.97:0/1185692300 shutdown_connections 2024-03-21T14:00:58.811 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:58.810+0000 7f4a5b00e640 1 --2- 172.21.15.97:0/1185692300 >> v2:172.21.15.97:6800/12494470 conn(0x7f4a3c06e270 0x7f4a3c070730 unknown :-1 s=CLOSED pgs=4052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:58.811 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:58.810+0000 7f4a5b00e640 1 --2- 172.21.15.97:0/1185692300 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a54106640 0x7f4a5419beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:58.811 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:58.810+0000 7f4a5b00e640 1 --2- 172.21.15.97:0/1185692300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a54100f40 0x7f4a54197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:58.811 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:58.810+0000 7f4a5b00e640 1 --2- 172.21.15.97:0/1185692300 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4a54100580 0x7f4a541975f0 unknown :-1 s=CLOSED pgs=3228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:00:58.811 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:58.810+0000 7f4a5b00e640 1 -- 172.21.15.97:0/1185692300 >> 172.21.15.97:0/1185692300 conn(0x7f4a540776c0 msgr2=0x7f4a540ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T14:00:58.811 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:58.810+0000 7f4a5b00e640 1 -- 172.21.15.97:0/1185692300 shutdown_connections 2024-03-21T14:00:58.811 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:00:58.810+0000 7f4a5b00e640 1 -- 172.21.15.97:0/1185692300 wait complete. 2024-03-21T14:00:58.822 INFO:tasks.workunit.client.0.smithi097.stderr:+ sleep 30 2024-03-21T14:00:59.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:58 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "field": "max_objects", "val": "10"}]': finished 2024-03-21T14:00:59.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:58 smithi144 ceph-mon[33633]: osdmap e795: 8 total, 8 up, 8 in 2024-03-21T14:00:59.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:58 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1185692300' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "field": "max_objects", "val": "10"}]: dispatch 2024-03-21T14:00:59.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:58 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "field": "max_objects", "val": "10"}]: dispatch 2024-03-21T14:00:59.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:58 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "field": "max_objects", "val": "10"}]': finished 2024-03-21T14:00:59.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:58 smithi097 ceph-mon[39416]: osdmap e795: 8 total, 8 up, 8 in 2024-03-21T14:00:59.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:58 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1185692300' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "field": "max_objects", "val": "10"}]: dispatch 2024-03-21T14:00:59.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:58 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "field": "max_objects", "val": "10"}]: dispatch 2024-03-21T14:00:59.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:58 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "field": "max_objects", "val": "10"}]': finished 2024-03-21T14:00:59.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:58 smithi097 ceph-mon[30040]: osdmap e795: 8 total, 8 up, 8 in 2024-03-21T14:00:59.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:58 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1185692300' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "field": "max_objects", "val": "10"}]: dispatch 2024-03-21T14:00:59.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:58 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "field": "max_objects", "val": "10"}]: dispatch 2024-03-21T14:01:00.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:59 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "field": "max_objects", "val": "10"}]': finished 2024-03-21T14:01:00.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:59 smithi144 ceph-mon[33633]: osdmap e796: 8 total, 8 up, 8 in 2024-03-21T14:01:00.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:00:59 smithi144 ceph-mon[33633]: pgmap v1752: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-03-21T14:01:00.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:59 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "field": "max_objects", "val": "10"}]': finished 2024-03-21T14:01:00.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:59 smithi097 ceph-mon[39416]: osdmap e796: 8 total, 8 up, 8 in 2024-03-21T14:01:00.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:00:59 smithi097 ceph-mon[39416]: pgmap v1752: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-03-21T14:01:00.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:59 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "field": "max_objects", "val": "10"}]': finished 2024-03-21T14:01:00.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:59 smithi097 ceph-mon[30040]: osdmap e796: 8 total, 8 up, 8 in 2024-03-21T14:01:00.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:00:59 smithi097 ceph-mon[30040]: pgmap v1752: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-03-21T14:01:02.306 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:01:02 smithi097 ceph-mon[39416]: pgmap v1753: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 957 B/s rd, 2.2 KiB/s wr, 1 op/s 2024-03-21T14:01:02.306 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:01:02 smithi097 ceph-mon[30040]: pgmap v1753: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 957 B/s rd, 2.2 KiB/s wr, 1 op/s 2024-03-21T14:01:02.307 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 14:01:01 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:14:01:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T14:01:02.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:01:02 smithi144 ceph-mon[33633]: pgmap v1753: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 957 B/s rd, 2.2 KiB/s wr, 1 op/s 2024-03-21T14:01:03.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:01:03 smithi097 ceph-mon[39416]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T14:01:03.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:01:03 smithi097 ceph-mon[30040]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T14:01:03.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:01:03 smithi144 ceph-mon[33633]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T14:01:04.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:01:04 smithi097 ceph-mon[30040]: pgmap v1754: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 806 B/s rd, 1.9 KiB/s wr, 0 op/s 2024-03-21T14:01:04.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:01:04 smithi097 ceph-mon[39416]: pgmap v1754: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 806 B/s rd, 1.9 KiB/s wr, 0 op/s 2024-03-21T14:01:04.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:01:04 smithi144 ceph-mon[33633]: pgmap v1754: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 806 B/s rd, 1.9 KiB/s wr, 0 op/s 2024-03-21T14:01:05.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 14:01:05 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T14:01:06.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:01:06 smithi097 ceph-mon[30040]: pgmap v1755: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 1 op/s 2024-03-21T14:01:06.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:01:06 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T14:01:06.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:01:06 smithi097 ceph-mon[39416]: pgmap v1755: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 1 op/s 2024-03-21T14:01:06.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:01:06 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T14:01:06.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:01:06 smithi144 ceph-mon[33633]: pgmap v1755: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 1 op/s 2024-03-21T14:01:06.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:01:06 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T14:01:08.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:01:08 smithi097 ceph-mon[30040]: pgmap v1756: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 1 op/s 2024-03-21T14:01:08.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:01:08 smithi097 ceph-mon[39416]: pgmap v1756: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 1 op/s 2024-03-21T14:01:08.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:01:08 smithi144 ceph-mon[33633]: pgmap v1756: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 1 op/s 2024-03-21T14:01:10.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:01:10 smithi097 ceph-mon[30040]: pgmap v1757: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 994 B/s rd, 0 op/s 2024-03-21T14:01:10.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:01:10 smithi097 ceph-mon[39416]: pgmap v1757: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 994 B/s rd, 0 op/s 2024-03-21T14:01:10.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:01:10 smithi144 ceph-mon[33633]: pgmap v1757: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 994 B/s rd, 0 op/s 2024-03-21T14:01:12.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:01:12 smithi097 ceph-mon[39416]: pgmap v1758: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:01:12.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:01:12 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T14:01:12.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:01:12 smithi097 ceph-mon[30040]: pgmap v1758: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:01:12.304 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:01:12 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T14:01:12.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 14:01:11 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:14:01:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T14:01:12.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:01:12 smithi144 ceph-mon[33633]: pgmap v1758: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:01:12.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:01:12 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T14:01:14.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:01:14 smithi097 ceph-mon[39416]: pgmap v1759: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:01:14.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:01:14 smithi097 ceph-mon[30040]: pgmap v1759: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:01:14.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:01:14 smithi144 ceph-mon[33633]: pgmap v1759: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:01:15.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 14:01:15 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T14:01:16.522 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:01:16 smithi144 ceph-mon[33633]: pgmap v1760: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T14:01:16.522 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:01:16 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T14:01:16.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:01:16 smithi097 ceph-mon[39416]: pgmap v1760: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T14:01:16.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:01:16 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T14:01:16.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:01:16 smithi097 ceph-mon[30040]: pgmap v1760: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T14:01:16.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:01:16 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T14:01:17.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:01:17 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T14:01:17.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:01:17 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T14:01:17.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:01:17 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T14:01:18.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:01:18 smithi097 ceph-mon[39416]: pgmap v1761: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:01:18.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:01:18 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T14:01:18.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:01:18 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T14:01:18.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:01:18 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T14:01:18.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:01:18 smithi097 ceph-mon[30040]: pgmap v1761: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:01:18.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:01:18 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T14:01:18.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:01:18 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T14:01:18.555 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:01:18 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T14:01:18.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:01:18 smithi144 ceph-mon[33633]: pgmap v1761: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:01:18.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:01:18 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T14:01:18.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:01:18 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T14:01:18.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:01:18 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T14:01:20.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:01:20 smithi097 ceph-mon[39416]: pgmap v1762: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T14:01:20.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:01:20 smithi097 ceph-mon[30040]: pgmap v1762: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T14:01:20.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:01:20 smithi144 ceph-mon[33633]: pgmap v1762: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T14:01:22.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:01:22 smithi097 ceph-mon[39416]: pgmap v1763: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:01:22.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:01:22 smithi097 ceph-mon[30040]: pgmap v1763: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:01:22.304 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 14:01:21 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:14:01:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T14:01:22.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:01:22 smithi144 ceph-mon[33633]: pgmap v1763: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:01:24.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:01:24 smithi097 ceph-mon[39416]: pgmap v1764: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:01:24.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:01:24 smithi097 ceph-mon[30040]: pgmap v1764: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:01:24.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:01:24 smithi144 ceph-mon[33633]: pgmap v1764: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:01:25.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 14:01:25 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T14:01:26.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:01:26 smithi097 ceph-mon[39416]: pgmap v1765: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T14:01:26.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:01:26 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T14:01:26.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:01:26 smithi097 ceph-mon[30040]: pgmap v1765: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T14:01:26.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:01:26 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T14:01:26.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:01:26 smithi144 ceph-mon[33633]: pgmap v1765: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T14:01:26.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:01:26 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T14:01:27.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:01:27 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T14:01:27.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:01:27 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T14:01:27.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:01:27 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T14:01:28.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:01:28 smithi097 ceph-mon[39416]: pgmap v1766: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:01:28.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:01:28 smithi097 ceph-mon[30040]: pgmap v1766: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:01:28.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:01:28 smithi144 ceph-mon[33633]: pgmap v1766: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:01:28.824 INFO:tasks.workunit.client.0.smithi097.stderr:++ seq 1 10 2024-03-21T14:01:28.824 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in `seq 1 10` 2024-03-21T14:01:28.825 INFO:tasks.workunit.client.0.smithi097.stderr:+ rados -p d7ed4d26-9388-40f6-9c6c-02152ee33b37 put obj1 /etc/passwd 2024-03-21T14:01:28.865 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in `seq 1 10` 2024-03-21T14:01:28.865 INFO:tasks.workunit.client.0.smithi097.stderr:+ rados -p d7ed4d26-9388-40f6-9c6c-02152ee33b37 put obj2 /etc/passwd 2024-03-21T14:01:28.903 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in `seq 1 10` 2024-03-21T14:01:28.903 INFO:tasks.workunit.client.0.smithi097.stderr:+ rados -p d7ed4d26-9388-40f6-9c6c-02152ee33b37 put obj3 /etc/passwd 2024-03-21T14:01:28.938 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in `seq 1 10` 2024-03-21T14:01:28.939 INFO:tasks.workunit.client.0.smithi097.stderr:+ rados -p d7ed4d26-9388-40f6-9c6c-02152ee33b37 put obj4 /etc/passwd 2024-03-21T14:01:28.974 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in `seq 1 10` 2024-03-21T14:01:28.974 INFO:tasks.workunit.client.0.smithi097.stderr:+ rados -p d7ed4d26-9388-40f6-9c6c-02152ee33b37 put obj5 /etc/passwd 2024-03-21T14:01:29.008 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in `seq 1 10` 2024-03-21T14:01:29.008 INFO:tasks.workunit.client.0.smithi097.stderr:+ rados -p d7ed4d26-9388-40f6-9c6c-02152ee33b37 put obj6 /etc/passwd 2024-03-21T14:01:29.042 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in `seq 1 10` 2024-03-21T14:01:29.042 INFO:tasks.workunit.client.0.smithi097.stderr:+ rados -p d7ed4d26-9388-40f6-9c6c-02152ee33b37 put obj7 /etc/passwd 2024-03-21T14:01:29.077 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in `seq 1 10` 2024-03-21T14:01:29.077 INFO:tasks.workunit.client.0.smithi097.stderr:+ rados -p d7ed4d26-9388-40f6-9c6c-02152ee33b37 put obj8 /etc/passwd 2024-03-21T14:01:29.112 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in `seq 1 10` 2024-03-21T14:01:29.112 INFO:tasks.workunit.client.0.smithi097.stderr:+ rados -p d7ed4d26-9388-40f6-9c6c-02152ee33b37 put obj9 /etc/passwd 2024-03-21T14:01:29.147 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in `seq 1 10` 2024-03-21T14:01:29.147 INFO:tasks.workunit.client.0.smithi097.stderr:+ rados -p d7ed4d26-9388-40f6-9c6c-02152ee33b37 put obj10 /etc/passwd 2024-03-21T14:01:29.182 INFO:tasks.workunit.client.0.smithi097.stderr:+ sleep 30 2024-03-21T14:01:30.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:01:30 smithi097 ceph-mon[39416]: pgmap v1767: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T14:01:30.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:01:30 smithi097 ceph-mon[30040]: pgmap v1767: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T14:01:30.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:01:30 smithi144 ceph-mon[33633]: pgmap v1767: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T14:01:32.209 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 14:01:31 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:14:01:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T14:01:32.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:01:32 smithi097 ceph-mon[30040]: pgmap v1768: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:01:32.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:01:32 smithi097 ceph-mon[39416]: pgmap v1768: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:01:32.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:01:32 smithi144 ceph-mon[33633]: pgmap v1768: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:01:34.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:01:34 smithi097 ceph-mon[39416]: pgmap v1769: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:01:34.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:01:34 smithi097 ceph-mon[30040]: pgmap v1769: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:01:34.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:01:34 smithi144 ceph-mon[33633]: pgmap v1769: 188 pgs: 188 active+clean; 744 KiB data, 936 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:01:35.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 14:01:35 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T14:01:36.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:01:36 smithi097 ceph-mon[30040]: pgmap v1770: 188 pgs: 188 active+clean; 859 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-03-21T14:01:36.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:01:36 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T14:01:36.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:01:36 smithi097 ceph-mon[39416]: pgmap v1770: 188 pgs: 188 active+clean; 859 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-03-21T14:01:36.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:01:36 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T14:01:36.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:01:36 smithi144 ceph-mon[33633]: pgmap v1770: 188 pgs: 188 active+clean; 859 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-03-21T14:01:36.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:01:36 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T14:01:38.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:01:38 smithi097 ceph-mon[39416]: pgmap v1771: 188 pgs: 188 active+clean; 859 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 10 KiB/s wr, 1 op/s 2024-03-21T14:01:38.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:01:38 smithi097 ceph-mon[39416]: pool 'd7ed4d26-9388-40f6-9c6c-02152ee33b37' is full (reached quota's max_objects: 10) 2024-03-21T14:01:38.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:01:38 smithi097 ceph-mon[39416]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-21T14:01:38.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:01:38 smithi097 ceph-mon[39416]: osdmap e797: 8 total, 8 up, 8 in 2024-03-21T14:01:38.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:01:38 smithi097 ceph-mon[30040]: pgmap v1771: 188 pgs: 188 active+clean; 859 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 10 KiB/s wr, 1 op/s 2024-03-21T14:01:38.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:01:38 smithi097 ceph-mon[30040]: pool 'd7ed4d26-9388-40f6-9c6c-02152ee33b37' is full (reached quota's max_objects: 10) 2024-03-21T14:01:38.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:01:38 smithi097 ceph-mon[30040]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-21T14:01:38.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:01:38 smithi097 ceph-mon[30040]: osdmap e797: 8 total, 8 up, 8 in 2024-03-21T14:01:38.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:01:38 smithi144 ceph-mon[33633]: pgmap v1771: 188 pgs: 188 active+clean; 859 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 10 KiB/s wr, 1 op/s 2024-03-21T14:01:38.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:01:38 smithi144 ceph-mon[33633]: pool 'd7ed4d26-9388-40f6-9c6c-02152ee33b37' is full (reached quota's max_objects: 10) 2024-03-21T14:01:38.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:01:38 smithi144 ceph-mon[33633]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-21T14:01:38.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:01:38 smithi144 ceph-mon[33633]: osdmap e797: 8 total, 8 up, 8 in 2024-03-21T14:01:40.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:01:40 smithi097 ceph-mon[39416]: pgmap v1773: 188 pgs: 188 active+clean; 859 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-21T14:01:40.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:01:40 smithi097 ceph-mon[30040]: pgmap v1773: 188 pgs: 188 active+clean; 859 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-21T14:01:40.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:01:40 smithi144 ceph-mon[33633]: pgmap v1773: 188 pgs: 188 active+clean; 859 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-21T14:01:42.223 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 14:01:41 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:14:01:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T14:01:42.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:01:42 smithi097 ceph-mon[39416]: pgmap v1774: 188 pgs: 188 active+clean; 859 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-21T14:01:42.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:01:42 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T14:01:42.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:01:42 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T14:01:42.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:01:42 smithi097 ceph-mon[30040]: pgmap v1774: 188 pgs: 188 active+clean; 859 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-21T14:01:42.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:01:42 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T14:01:42.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:01:42 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T14:01:42.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:01:42 smithi144 ceph-mon[33633]: pgmap v1774: 188 pgs: 188 active+clean; 859 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-21T14:01:42.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:01:42 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T14:01:42.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:01:42 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T14:01:44.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:01:44 smithi097 ceph-mon[30040]: pgmap v1775: 188 pgs: 188 active+clean; 859 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-21T14:01:44.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:01:44 smithi097 ceph-mon[39416]: pgmap v1775: 188 pgs: 188 active+clean; 859 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-21T14:01:44.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:01:44 smithi144 ceph-mon[33633]: pgmap v1775: 188 pgs: 188 active+clean; 859 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-21T14:01:45.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 14:01:45 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T14:01:46.522 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:01:46 smithi144 ceph-mon[33633]: pgmap v1776: 188 pgs: 188 active+clean; 859 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T14:01:46.523 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:01:46 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T14:01:46.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:01:46 smithi097 ceph-mon[39416]: pgmap v1776: 188 pgs: 188 active+clean; 859 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T14:01:46.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:01:46 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T14:01:46.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:01:46 smithi097 ceph-mon[30040]: pgmap v1776: 188 pgs: 188 active+clean; 859 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T14:01:46.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:01:46 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T14:01:48.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:01:48 smithi097 ceph-mon[39416]: pgmap v1777: 188 pgs: 188 active+clean; 859 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T14:01:48.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:01:48 smithi097 ceph-mon[30040]: pgmap v1777: 188 pgs: 188 active+clean; 859 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T14:01:48.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:01:48 smithi144 ceph-mon[33633]: pgmap v1777: 188 pgs: 188 active+clean; 859 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T14:01:50.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:01:50 smithi097 ceph-mon[39416]: pgmap v1778: 188 pgs: 188 active+clean; 859 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-21T14:01:50.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:01:50 smithi097 ceph-mon[30040]: pgmap v1778: 188 pgs: 188 active+clean; 859 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-21T14:01:50.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:01:50 smithi144 ceph-mon[33633]: pgmap v1778: 188 pgs: 188 active+clean; 859 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-21T14:01:52.240 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 14:01:51 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:14:01:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T14:01:52.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:01:52 smithi097 ceph-mon[39416]: pgmap v1779: 188 pgs: 188 active+clean; 859 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T14:01:52.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:01:52 smithi097 ceph-mon[30040]: pgmap v1779: 188 pgs: 188 active+clean; 859 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T14:01:52.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:01:52 smithi144 ceph-mon[33633]: pgmap v1779: 188 pgs: 188 active+clean; 859 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T14:01:54.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:01:54 smithi097 ceph-mon[39416]: pgmap v1780: 188 pgs: 188 active+clean; 859 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:01:54.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:01:54 smithi097 ceph-mon[30040]: pgmap v1780: 188 pgs: 188 active+clean; 859 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:01:54.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:01:54 smithi144 ceph-mon[33633]: pgmap v1780: 188 pgs: 188 active+clean; 859 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:01:55.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 14:01:55 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T14:01:56.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:01:56 smithi097 ceph-mon[39416]: pgmap v1781: 188 pgs: 188 active+clean; 859 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T14:01:56.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:01:56 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T14:01:56.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:01:56 smithi097 ceph-mon[30040]: pgmap v1781: 188 pgs: 188 active+clean; 859 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T14:01:56.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:01:56 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T14:01:56.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:01:56 smithi144 ceph-mon[33633]: pgmap v1781: 188 pgs: 188 active+clean; 859 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T14:01:56.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:01:56 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T14:01:57.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:01:57 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T14:01:57.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:01:57 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T14:01:57.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:01:57 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T14:01:58.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:01:58 smithi097 ceph-mon[39416]: pgmap v1782: 188 pgs: 188 active+clean; 859 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:01:58.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:01:58 smithi097 ceph-mon[30040]: pgmap v1782: 188 pgs: 188 active+clean; 859 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:01:58.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:01:58 smithi144 ceph-mon[33633]: pgmap v1782: 188 pgs: 188 active+clean; 859 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:01:59.183 INFO:tasks.workunit.client.0.smithi097.stderr:+ rados -p b5a5e36b-8518-4a8b-80b9-4898af5bd2dd put threemore /etc/passwd 2024-03-21T14:01:59.223 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph osd pool set-quota b5a5e36b-8518-4a8b-80b9-4898af5bd2dd max_bytes 0 2024-03-21T14:01:59.312 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:01:59.310+0000 7f7250c2e640 1 -- 172.21.15.97:0/1688093942 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f724c10e040 msgr2=0x7f724c110430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:01:59.312 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:01:59.310+0000 7f7250c2e640 1 --2- 172.21.15.97:0/1688093942 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f724c10e040 0x7f724c110430 secure :-1 s=READY pgs=3249 cs=0 l=1 rev1=1 crypto rx=0x7f7244009900 tx=0x7f724402bc30 comp rx=0 tx=0).stop 2024-03-21T14:01:59.312 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:01:59.311+0000 7f7250c2e640 1 -- 172.21.15.97:0/1688093942 shutdown_connections 2024-03-21T14:01:59.312 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:01:59.311+0000 7f7250c2e640 1 --2- 172.21.15.97:0/1688093942 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f724c10e040 0x7f724c110430 unknown :-1 s=CLOSED pgs=3249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:01:59.313 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:01:59.311+0000 7f7250c2e640 1 --2- 172.21.15.97:0/1688093942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f724c100db0 0x7f724c10db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:01:59.313 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:01:59.311+0000 7f7250c2e640 1 --2- 172.21.15.97:0/1688093942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f724c100490 0x7f724c100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:01:59.313 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:01:59.311+0000 7f7250c2e640 1 -- 172.21.15.97:0/1688093942 >> 172.21.15.97:0/1688093942 conn(0x7f724c0776c0 msgr2=0x7f724c077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T14:01:59.313 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:01:59.311+0000 7f7250c2e640 1 -- 172.21.15.97:0/1688093942 shutdown_connections 2024-03-21T14:01:59.313 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:01:59.311+0000 7f7250c2e640 1 -- 172.21.15.97:0/1688093942 wait complete. 2024-03-21T14:01:59.313 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:01:59.311+0000 7f7250c2e640 1 Processor -- start 2024-03-21T14:01:59.313 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:01:59.311+0000 7f7250c2e640 1 -- start start 2024-03-21T14:01:59.313 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:01:59.312+0000 7f7250c2e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f724c100490 0x7f724c19f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T14:01:59.313 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:01:59.312+0000 7f7250c2e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f724c100db0 0x7f724c19fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T14:01:59.313 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:01:59.312+0000 7f7250c2e640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f724c10e040 0x7f724c199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T14:01:59.313 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:01:59.312+0000 7f7250c2e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f724c113380 con 0x7f724c100db0 2024-03-21T14:01:59.313 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:01:59.312+0000 7f7250c2e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f724c113200 con 0x7f724c100490 2024-03-21T14:01:59.313 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:01:59.312+0000 7f7250c2e640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f724c113500 con 0x7f724c10e040 2024-03-21T14:01:59.313 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:01:59.312+0000 7f724ad9d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f724c10e040 0x7f724c199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T14:01:59.314 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:01:59.312+0000 7f7249d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f724c100db0 0x7f724c19fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T14:01:59.314 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:01:59.312+0000 7f724ad9d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f724c10e040 0x7f724c199650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.97:43440/0 (socket says 172.21.15.97:43440) 2024-03-21T14:01:59.314 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:01:59.312+0000 7f724ad9d640 1 -- 172.21.15.97:0/2104641486 learned_addr learned my addr 172.21.15.97:0/2104641486 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T14:01:59.314 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:01:59.312+0000 7f724a59c640 1 --2- 172.21.15.97:0/2104641486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f724c100490 0x7f724c19f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T14:01:59.316 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:01:59.312+0000 7f724ad9d640 1 -- 172.21.15.97:0/2104641486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f724c100490 msgr2=0x7f724c19f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:01:59.316 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:01:59.312+0000 7f724ad9d640 1 --2- 172.21.15.97:0/2104641486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f724c100490 0x7f724c19f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:01:59.316 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:01:59.312+0000 7f724ad9d640 1 -- 172.21.15.97:0/2104641486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f724c100db0 msgr2=0x7f724c19fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:01:59.316 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:01:59.312+0000 7f724ad9d640 1 --2- 172.21.15.97:0/2104641486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f724c100db0 0x7f724c19fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:01:59.316 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:01:59.312+0000 7f724ad9d640 1 -- 172.21.15.97:0/2104641486 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f724c199f10 con 0x7f724c10e040 2024-03-21T14:01:59.316 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:01:59.313+0000 7f724ad9d640 1 --2- 172.21.15.97:0/2104641486 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f724c10e040 0x7f724c199650 secure :-1 s=READY pgs=3250 cs=0 l=1 rev1=1 crypto rx=0x7f7244005940 tx=0x7f724402c790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T14:01:59.316 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:01:59.313+0000 7f723b7fe640 1 -- 172.21.15.97:0/2104641486 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7244043020 con 0x7f724c10e040 2024-03-21T14:01:59.316 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:01:59.313+0000 7f723b7fe640 1 -- 172.21.15.97:0/2104641486 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f72440043d0 con 0x7f724c10e040 2024-03-21T14:01:59.316 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:01:59.313+0000 7f7250c2e640 1 -- 172.21.15.97:0/2104641486 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f724c19a1a0 con 0x7f724c10e040 2024-03-21T14:01:59.316 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:01:59.313+0000 7f723b7fe640 1 -- 172.21.15.97:0/2104641486 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7244003ae0 con 0x7f724c10e040 2024-03-21T14:01:59.316 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:01:59.313+0000 7f7250c2e640 1 -- 172.21.15.97:0/2104641486 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f724c1a6510 con 0x7f724c10e040 2024-03-21T14:01:59.316 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:01:59.314+0000 7f7250c2e640 1 -- 172.21.15.97:0/2104641486 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f724c0691d0 con 0x7f724c10e040 2024-03-21T14:01:59.317 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:01:59.315+0000 7f723b7fe640 1 -- 172.21.15.97:0/2104641486 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 22) v1 ==== 92152+0+0 (secure 0 0 0) 0x7f7244046020 con 0x7f724c10e040 2024-03-21T14:01:59.317 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:01:59.315+0000 7f723b7fe640 1 --2- 172.21.15.97:0/2104641486 >> v2:172.21.15.97:6800/12494470 conn(0x7f723406e270 0x7f7234070730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T14:01:59.317 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:01:59.315+0000 7f723b7fe640 1 -- 172.21.15.97:0/2104641486 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(797..797 src has 254..797) v4 ==== 8202+0+0 (secure 0 0 0) 0x7f72440aff90 con 0x7f724c10e040 2024-03-21T14:01:59.317 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:01:59.315+0000 7f724a59c640 1 --2- 172.21.15.97:0/2104641486 >> v2:172.21.15.97:6800/12494470 conn(0x7f723406e270 0x7f7234070730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T14:01:59.317 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:01:59.315+0000 7f723b7fe640 1 -- 172.21.15.97:0/2104641486 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=798}) v3 -- 0x7f723407a060 con 0x7f724c10e040 2024-03-21T14:01:59.317 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:01:59.316+0000 7f724a59c640 1 --2- 172.21.15.97:0/2104641486 >> v2:172.21.15.97:6800/12494470 conn(0x7f723406e270 0x7f7234070730 secure :-1 s=READY pgs=4064 cs=0 l=1 rev1=1 crypto rx=0x7f723c0099a0 tx=0x7f723c009500 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T14:01:59.320 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:01:59.320+0000 7f723b7fe640 1 -- 172.21.15.97:0/2104641486 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7244080a70 con 0x7f724c10e040 2024-03-21T14:01:59.498 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:01:59.497+0000 7f7250c2e640 1 -- 172.21.15.97:0/2104641486 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7f724c19ab80 con 0x7f724c10e040 2024-03-21T14:02:00.267 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:00.267+0000 7f723b7fe640 1 -- 172.21.15.97:0/2104641486 <== mon.2 v2:172.21.15.97:3301/0 7 ==== osd_map(798..798 src has 254..798) v4 ==== 628+0+0 (secure 0 0 0) 0x7f72440e4740 con 0x7f724c10e040 2024-03-21T14:02:00.267 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:00.267+0000 7f723b7fe640 1 -- 172.21.15.97:0/2104641486 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=799}) v3 -- 0x7f723407aba0 con 0x7f724c10e040 2024-03-21T14:02:00.271 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:00.271+0000 7f723b7fe640 1 -- 172.21.15.97:0/2104641486 <== mon.2 v2:172.21.15.97:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool b5a5e36b-8518-4a8b-80b9-4898af5bd2dd v798) v1 ==== 217+0+0 (secure 0 0 0) 0x7f72440852a0 con 0x7f724c10e040 2024-03-21T14:02:00.381 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:00.380+0000 7f7250c2e640 1 -- 172.21.15.97:0/2104641486 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7f724c1a66e0 con 0x7f724c10e040 2024-03-21T14:02:00.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:00 smithi097 ceph-mon[39416]: pgmap v1783: 188 pgs: 188 active+clean; 859 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T14:02:00.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:00 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2104641486' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-21T14:02:00.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:00 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-21T14:02:00.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:00 smithi097 ceph-mon[30040]: pgmap v1783: 188 pgs: 188 active+clean; 859 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T14:02:00.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:00 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2104641486' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-21T14:02:00.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:00 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-21T14:02:00.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:00 smithi144 ceph-mon[33633]: pgmap v1783: 188 pgs: 188 active+clean; 859 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T14:02:00.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:00 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2104641486' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-21T14:02:00.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:00 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-21T14:02:01.274 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.273+0000 7f723b7fe640 1 -- 172.21.15.97:0/2104641486 <== mon.2 v2:172.21.15.97:3301/0 9 ==== osd_map(799..799 src has 254..799) v4 ==== 628+0+0 (secure 0 0 0) 0x7f72440adc60 con 0x7f724c10e040 2024-03-21T14:02:01.274 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.274+0000 7f723b7fe640 1 -- 172.21.15.97:0/2104641486 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=800}) v3 -- 0x7f723407afd0 con 0x7f724c10e040 2024-03-21T14:02:01.278 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.278+0000 7f723b7fe640 1 -- 172.21.15.97:0/2104641486 <== mon.2 v2:172.21.15.97:3301/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool b5a5e36b-8518-4a8b-80b9-4898af5bd2dd v799) v1 ==== 217+0+0 (secure 0 0 0) 0x7f724407aa70 con 0x7f724c10e040 2024-03-21T14:02:01.278 INFO:tasks.workunit.client.0.smithi097.stderr:set-quota max_bytes = 0 for pool b5a5e36b-8518-4a8b-80b9-4898af5bd2dd 2024-03-21T14:02:01.283 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.282+0000 7f7250c2e640 1 -- 172.21.15.97:0/2104641486 >> v2:172.21.15.97:6800/12494470 conn(0x7f723406e270 msgr2=0x7f7234070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:02:01.283 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.282+0000 7f7250c2e640 1 --2- 172.21.15.97:0/2104641486 >> v2:172.21.15.97:6800/12494470 conn(0x7f723406e270 0x7f7234070730 secure :-1 s=READY pgs=4064 cs=0 l=1 rev1=1 crypto rx=0x7f723c0099a0 tx=0x7f723c009500 comp rx=0 tx=0).stop 2024-03-21T14:02:01.283 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.282+0000 7f7250c2e640 1 -- 172.21.15.97:0/2104641486 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f724c10e040 msgr2=0x7f724c199650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:02:01.283 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.282+0000 7f7250c2e640 1 --2- 172.21.15.97:0/2104641486 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f724c10e040 0x7f724c199650 secure :-1 s=READY pgs=3250 cs=0 l=1 rev1=1 crypto rx=0x7f7244005940 tx=0x7f724402c790 comp rx=0 tx=0).stop 2024-03-21T14:02:01.283 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.282+0000 7f7250c2e640 1 -- 172.21.15.97:0/2104641486 shutdown_connections 2024-03-21T14:02:01.283 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.282+0000 7f7250c2e640 1 --2- 172.21.15.97:0/2104641486 >> v2:172.21.15.97:6800/12494470 conn(0x7f723406e270 0x7f7234070730 unknown :-1 s=CLOSED pgs=4064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:02:01.283 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.282+0000 7f7250c2e640 1 --2- 172.21.15.97:0/2104641486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f724c100db0 0x7f724c19fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:02:01.283 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.282+0000 7f7250c2e640 1 --2- 172.21.15.97:0/2104641486 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f724c10e040 0x7f724c199650 unknown :-1 s=CLOSED pgs=3250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:02:01.283 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.282+0000 7f7250c2e640 1 --2- 172.21.15.97:0/2104641486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f724c100490 0x7f724c19f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:02:01.283 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.282+0000 7f7250c2e640 1 -- 172.21.15.97:0/2104641486 >> 172.21.15.97:0/2104641486 conn(0x7f724c0776c0 msgr2=0x7f724c0ff480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T14:02:01.283 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.282+0000 7f7250c2e640 1 -- 172.21.15.97:0/2104641486 shutdown_connections 2024-03-21T14:02:01.283 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.282+0000 7f7250c2e640 1 -- 172.21.15.97:0/2104641486 wait complete. 2024-03-21T14:02:01.293 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph osd pool set-quota b5a5e36b-8518-4a8b-80b9-4898af5bd2dd max_objects 0 2024-03-21T14:02:01.381 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.378+0000 7f0ac65ac640 1 -- 172.21.15.97:0/400182148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ac0100580 msgr2=0x7f0ac0100a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:02:01.381 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.378+0000 7f0ac65ac640 1 --2- 172.21.15.97:0/400182148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ac0100580 0x7f0ac0100a00 secure :-1 s=READY pgs=3030 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4009900 tx=0x7f0ab402bc30 comp rx=0 tx=0).stop 2024-03-21T14:02:01.381 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.378+0000 7f0ac65ac640 1 -- 172.21.15.97:0/400182148 shutdown_connections 2024-03-21T14:02:01.381 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.378+0000 7f0ac65ac640 1 --2- 172.21.15.97:0/400182148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ac0100580 0x7f0ac0100a00 unknown :-1 s=CLOSED pgs=3030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:02:01.381 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.378+0000 7f0ac65ac640 1 --2- 172.21.15.97:0/400182148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ac0100f40 0x7f0ac010d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:02:01.381 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.378+0000 7f0ac65ac640 1 --2- 172.21.15.97:0/400182148 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0ac0106640 0x7f0ac0106a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:02:01.381 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.378+0000 7f0ac65ac640 1 -- 172.21.15.97:0/400182148 >> 172.21.15.97:0/400182148 conn(0x7f0ac00776c0 msgr2=0x7f0ac0077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T14:02:01.381 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.378+0000 7f0ac65ac640 1 -- 172.21.15.97:0/400182148 shutdown_connections 2024-03-21T14:02:01.381 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.378+0000 7f0ac65ac640 1 -- 172.21.15.97:0/400182148 wait complete. 2024-03-21T14:02:01.381 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.379+0000 7f0ac65ac640 1 Processor -- start 2024-03-21T14:02:01.382 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.379+0000 7f0ac65ac640 1 -- start start 2024-03-21T14:02:01.382 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.379+0000 7f0ac65ac640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ac0100580 0x7f0ac01021f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T14:02:01.382 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.379+0000 7f0ac65ac640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ac0100f40 0x7f0ac0102730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T14:02:01.382 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.379+0000 7f0ac65ac640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0ac0106640 0x7f0ac0102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T14:02:01.382 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.379+0000 7f0ac65ac640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac010fff0 con 0x7f0ac0100580 2024-03-21T14:02:01.382 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.379+0000 7f0ac65ac640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac010fe70 con 0x7f0ac0100f40 2024-03-21T14:02:01.382 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.379+0000 7f0ac65ac640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac0110170 con 0x7f0ac0106640 2024-03-21T14:02:01.382 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.379+0000 7f0abffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ac0100580 0x7f0ac01021f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T14:02:01.382 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.379+0000 7f0abffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ac0100580 0x7f0ac01021f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43506/0 (socket says 172.21.15.97:43506) 2024-03-21T14:02:01.382 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.379+0000 7f0abffff640 1 -- 172.21.15.97:0/210912661 learned_addr learned my addr 172.21.15.97:0/210912661 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T14:02:01.382 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.379+0000 7f0ac4b49640 1 --2- 172.21.15.97:0/210912661 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0ac0106640 0x7f0ac0102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T14:02:01.383 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.379+0000 7f0abf7fe640 1 --2- 172.21.15.97:0/210912661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ac0100f40 0x7f0ac0102730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T14:02:01.383 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.379+0000 7f0abffff640 1 -- 172.21.15.97:0/210912661 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0ac0106640 msgr2=0x7f0ac0102c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:02:01.383 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.379+0000 7f0abffff640 1 --2- 172.21.15.97:0/210912661 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0ac0106640 0x7f0ac0102c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:02:01.383 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.379+0000 7f0abffff640 1 -- 172.21.15.97:0/210912661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ac0100f40 msgr2=0x7f0ac0102730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:02:01.383 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.379+0000 7f0abffff640 1 --2- 172.21.15.97:0/210912661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ac0100f40 0x7f0ac0102730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:02:01.383 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.379+0000 7f0abffff640 1 -- 172.21.15.97:0/210912661 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac01a5ac0 con 0x7f0ac0100580 2024-03-21T14:02:01.383 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.379+0000 7f0abffff640 1 --2- 172.21.15.97:0/210912661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ac0100580 0x7f0ac01021f0 secure :-1 s=READY pgs=3031 cs=0 l=1 rev1=1 crypto rx=0x7f0ab000ea60 tx=0x7f0ab000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T14:02:01.383 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.380+0000 7f0abd7fa640 1 -- 172.21.15.97:0/210912661 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab000cde0 con 0x7f0ac0100580 2024-03-21T14:02:01.383 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.380+0000 7f0abd7fa640 1 -- 172.21.15.97:0/210912661 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f0ab00035f0 con 0x7f0ac0100580 2024-03-21T14:02:01.383 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.380+0000 7f0ac65ac640 1 -- 172.21.15.97:0/210912661 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ac01a5db0 con 0x7f0ac0100580 2024-03-21T14:02:01.383 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.380+0000 7f0abd7fa640 1 -- 172.21.15.97:0/210912661 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab0010690 con 0x7f0ac0100580 2024-03-21T14:02:01.383 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.380+0000 7f0ac65ac640 1 -- 172.21.15.97:0/210912661 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ac01a6270 con 0x7f0ac0100580 2024-03-21T14:02:01.383 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.381+0000 7f0ac65ac640 1 -- 172.21.15.97:0/210912661 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ac0068580 con 0x7f0ac0100580 2024-03-21T14:02:01.384 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.381+0000 7f0abd7fa640 1 -- 172.21.15.97:0/210912661 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 22) v1 ==== 92152+0+0 (secure 0 0 0) 0x7f0ab0004000 con 0x7f0ac0100580 2024-03-21T14:02:01.384 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.381+0000 7f0abd7fa640 1 --2- 172.21.15.97:0/210912661 >> v2:172.21.15.97:6800/12494470 conn(0x7f0aa806e270 0x7f0aa8070730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T14:02:01.384 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.382+0000 7f0abf7fe640 1 --2- 172.21.15.97:0/210912661 >> v2:172.21.15.97:6800/12494470 conn(0x7f0aa806e270 0x7f0aa8070730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T14:02:01.385 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.382+0000 7f0abd7fa640 1 -- 172.21.15.97:0/210912661 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(799..799 src has 254..799) v4 ==== 8202+0+0 (secure 0 0 0) 0x7f0ab008eda0 con 0x7f0ac0100580 2024-03-21T14:02:01.385 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.382+0000 7f0abd7fa640 1 -- 172.21.15.97:0/210912661 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=800}) v3 -- 0x7f0aa8079fe0 con 0x7f0ac0100580 2024-03-21T14:02:01.385 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.382+0000 7f0abf7fe640 1 --2- 172.21.15.97:0/210912661 >> v2:172.21.15.97:6800/12494470 conn(0x7f0aa806e270 0x7f0aa8070730 secure :-1 s=READY pgs=4065 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4006010 tx=0x7f0ab4005e40 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T14:02:01.386 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.386+0000 7f0abd7fa640 1 -- 172.21.15.97:0/210912661 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0ab00042f0 con 0x7f0ac0100580 2024-03-21T14:02:01.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:01 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "0"}]': finished 2024-03-21T14:02:01.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:01 smithi097 ceph-mon[39416]: osdmap e798: 8 total, 8 up, 8 in 2024-03-21T14:02:01.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:01 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/2104641486' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-21T14:02:01.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:01 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-21T14:02:01.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:01 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "0"}]': finished 2024-03-21T14:02:01.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:01 smithi097 ceph-mon[30040]: osdmap e798: 8 total, 8 up, 8 in 2024-03-21T14:02:01.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:01 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/2104641486' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-21T14:02:01.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:01 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-21T14:02:01.564 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:01.564+0000 7f0ac65ac640 1 -- 172.21.15.97:0/210912661 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7f0ac01a6500 con 0x7f0ac0100580 2024-03-21T14:02:01.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:01 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "0"}]': finished 2024-03-21T14:02:01.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:01 smithi144 ceph-mon[33633]: osdmap e798: 8 total, 8 up, 8 in 2024-03-21T14:02:01.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:01 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/2104641486' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-21T14:02:01.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:01 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-21T14:02:02.275 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:02.274+0000 7f0abd7fa640 1 -- 172.21.15.97:0/210912661 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool b5a5e36b-8518-4a8b-80b9-4898af5bd2dd v800) v1 ==== 221+0+0 (secure 0 0 0) 0x7f0ab005f760 con 0x7f0ac0100580 2024-03-21T14:02:02.275 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:02.274+0000 7f0abd7fa640 1 -- 172.21.15.97:0/210912661 <== mon.0 v2:172.21.15.97:3300/0 8 ==== osd_map(800..800 src has 254..800) v4 ==== 628+0+0 (secure 0 0 0) 0x7f0ab0059760 con 0x7f0ac0100580 2024-03-21T14:02:02.275 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:02.274+0000 7f0abd7fa640 1 -- 172.21.15.97:0/210912661 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=801}) v3 -- 0x7f0aa807af80 con 0x7f0ac0100580 2024-03-21T14:02:02.284 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 14:02:01 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:14:02:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T14:02:02.389 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:02.388+0000 7f0ac65ac640 1 -- 172.21.15.97:0/210912661 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7f0ac01a6750 con 0x7f0ac0100580 2024-03-21T14:02:02.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:02 smithi097 ceph-mon[39416]: pgmap v1785: 188 pgs: 188 active+clean; 859 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T14:02:02.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:02 smithi097 ceph-mon[39416]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "0"}]': finished 2024-03-21T14:02:02.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:02 smithi097 ceph-mon[39416]: osdmap e799: 8 total, 8 up, 8 in 2024-03-21T14:02:02.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:02 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/210912661' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "0"}]: dispatch 2024-03-21T14:02:02.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:02 smithi097 ceph-mon[30040]: pgmap v1785: 188 pgs: 188 active+clean; 859 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T14:02:02.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:02 smithi097 ceph-mon[30040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "0"}]': finished 2024-03-21T14:02:02.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:02 smithi097 ceph-mon[30040]: osdmap e799: 8 total, 8 up, 8 in 2024-03-21T14:02:02.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:02 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/210912661' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "0"}]: dispatch 2024-03-21T14:02:02.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:02 smithi144 ceph-mon[33633]: pgmap v1785: 188 pgs: 188 active+clean; 859 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T14:02:02.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:02 smithi144 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_bytes", "val": "0"}]': finished 2024-03-21T14:02:02.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:02 smithi144 ceph-mon[33633]: osdmap e799: 8 total, 8 up, 8 in 2024-03-21T14:02:02.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:02 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/210912661' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "0"}]: dispatch 2024-03-21T14:02:03.290 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:03.289+0000 7f0abd7fa640 1 -- 172.21.15.97:0/210912661 <== mon.0 v2:172.21.15.97:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool b5a5e36b-8518-4a8b-80b9-4898af5bd2dd v801) v1 ==== 221+0+0 (secure 0 0 0) 0x7f0ab0063f90 con 0x7f0ac0100580 2024-03-21T14:02:03.290 INFO:tasks.workunit.client.0.smithi097.stderr:set-quota max_objects = 0 for pool b5a5e36b-8518-4a8b-80b9-4898af5bd2dd 2024-03-21T14:02:03.291 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:03.290+0000 7f0abd7fa640 1 -- 172.21.15.97:0/210912661 <== mon.0 v2:172.21.15.97:3300/0 10 ==== osd_map(801..801 src has 254..801) v4 ==== 628+0+0 (secure 0 0 0) 0x7f0ab008c9a0 con 0x7f0ac0100580 2024-03-21T14:02:03.291 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:03.290+0000 7f0abd7fa640 1 -- 172.21.15.97:0/210912661 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=802}) v3 -- 0x7f0aa807b190 con 0x7f0ac0100580 2024-03-21T14:02:03.293 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:03.293+0000 7f0ac65ac640 1 -- 172.21.15.97:0/210912661 >> v2:172.21.15.97:6800/12494470 conn(0x7f0aa806e270 msgr2=0x7f0aa8070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:02:03.293 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:03.293+0000 7f0ac65ac640 1 --2- 172.21.15.97:0/210912661 >> v2:172.21.15.97:6800/12494470 conn(0x7f0aa806e270 0x7f0aa8070730 secure :-1 s=READY pgs=4065 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4006010 tx=0x7f0ab4005e40 comp rx=0 tx=0).stop 2024-03-21T14:02:03.294 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:03.293+0000 7f0ac65ac640 1 -- 172.21.15.97:0/210912661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ac0100580 msgr2=0x7f0ac01021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:02:03.294 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:03.293+0000 7f0ac65ac640 1 --2- 172.21.15.97:0/210912661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ac0100580 0x7f0ac01021f0 secure :-1 s=READY pgs=3031 cs=0 l=1 rev1=1 crypto rx=0x7f0ab000ea60 tx=0x7f0ab000ef20 comp rx=0 tx=0).stop 2024-03-21T14:02:03.294 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:03.293+0000 7f0ac65ac640 1 -- 172.21.15.97:0/210912661 shutdown_connections 2024-03-21T14:02:03.294 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:03.293+0000 7f0ac65ac640 1 --2- 172.21.15.97:0/210912661 >> v2:172.21.15.97:6800/12494470 conn(0x7f0aa806e270 0x7f0aa8070730 unknown :-1 s=CLOSED pgs=4065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:02:03.294 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:03.293+0000 7f0ac65ac640 1 --2- 172.21.15.97:0/210912661 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0ac0106640 0x7f0ac0102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:02:03.294 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:03.293+0000 7f0ac65ac640 1 --2- 172.21.15.97:0/210912661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ac0100f40 0x7f0ac0102730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:02:03.294 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:03.293+0000 7f0ac65ac640 1 --2- 172.21.15.97:0/210912661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ac0100580 0x7f0ac01021f0 unknown :-1 s=CLOSED pgs=3031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:02:03.294 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:03.293+0000 7f0ac65ac640 1 -- 172.21.15.97:0/210912661 >> 172.21.15.97:0/210912661 conn(0x7f0ac00776c0 msgr2=0x7f0ac00689d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T14:02:03.294 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:03.293+0000 7f0ac65ac640 1 -- 172.21.15.97:0/210912661 shutdown_connections 2024-03-21T14:02:03.294 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:03.293+0000 7f0ac65ac640 1 -- 172.21.15.97:0/210912661 wait complete. 2024-03-21T14:02:03.306 INFO:tasks.workunit.client.0.smithi097.stderr:+ sleep 30 2024-03-21T14:02:03.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:03 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/210912661' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "0"}]': finished 2024-03-21T14:02:03.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:03 smithi097 ceph-mon[39416]: osdmap e800: 8 total, 8 up, 8 in 2024-03-21T14:02:03.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:03 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/210912661' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "0"}]: dispatch 2024-03-21T14:02:03.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:03 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/210912661' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "0"}]': finished 2024-03-21T14:02:03.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:03 smithi097 ceph-mon[30040]: osdmap e800: 8 total, 8 up, 8 in 2024-03-21T14:02:03.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:03 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/210912661' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "0"}]: dispatch 2024-03-21T14:02:03.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:03 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/210912661' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "0"}]': finished 2024-03-21T14:02:03.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:03 smithi144 ceph-mon[33633]: osdmap e800: 8 total, 8 up, 8 in 2024-03-21T14:02:03.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:03 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/210912661' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "0"}]: dispatch 2024-03-21T14:02:04.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:04 smithi097 ceph-mon[30040]: pgmap v1788: 188 pgs: 188 active+clean; 859 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:02:04.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:04 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/210912661' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "0"}]': finished 2024-03-21T14:02:04.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:04 smithi097 ceph-mon[30040]: osdmap e801: 8 total, 8 up, 8 in 2024-03-21T14:02:04.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:04 smithi097 ceph-mon[39416]: pgmap v1788: 188 pgs: 188 active+clean; 859 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:02:04.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:04 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/210912661' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "0"}]': finished 2024-03-21T14:02:04.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:04 smithi097 ceph-mon[39416]: osdmap e801: 8 total, 8 up, 8 in 2024-03-21T14:02:04.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:04 smithi144 ceph-mon[33633]: pgmap v1788: 188 pgs: 188 active+clean; 859 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:02:04.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:04 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/210912661' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "field": "max_objects", "val": "0"}]': finished 2024-03-21T14:02:04.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:04 smithi144 ceph-mon[33633]: osdmap e801: 8 total, 8 up, 8 in 2024-03-21T14:02:05.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 14:02:05 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T14:02:06.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:06 smithi097 ceph-mon[39416]: pgmap v1790: 188 pgs: 188 active+clean; 871 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.5 KiB/s wr, 1 op/s 2024-03-21T14:02:06.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:06 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T14:02:06.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:06 smithi097 ceph-mon[30040]: pgmap v1790: 188 pgs: 188 active+clean; 871 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.5 KiB/s wr, 1 op/s 2024-03-21T14:02:06.554 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:06 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T14:02:06.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:06 smithi144 ceph-mon[33633]: pgmap v1790: 188 pgs: 188 active+clean; 871 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.5 KiB/s wr, 1 op/s 2024-03-21T14:02:06.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:06 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T14:02:08.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:08 smithi097 ceph-mon[30040]: pgmap v1791: 188 pgs: 188 active+clean; 871 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 2.0 KiB/s wr, 0 op/s 2024-03-21T14:02:08.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:08 smithi097 ceph-mon[39416]: pgmap v1791: 188 pgs: 188 active+clean; 871 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 2.0 KiB/s wr, 0 op/s 2024-03-21T14:02:08.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:08 smithi144 ceph-mon[33633]: pgmap v1791: 188 pgs: 188 active+clean; 871 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 2.0 KiB/s wr, 0 op/s 2024-03-21T14:02:10.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:10 smithi097 ceph-mon[39416]: pgmap v1792: 188 pgs: 188 active+clean; 871 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 1 op/s 2024-03-21T14:02:10.553 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:10 smithi097 ceph-mon[30040]: pgmap v1792: 188 pgs: 188 active+clean; 871 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 1 op/s 2024-03-21T14:02:10.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:10 smithi144 ceph-mon[33633]: pgmap v1792: 188 pgs: 188 active+clean; 871 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 1 op/s 2024-03-21T14:02:12.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 14:02:11 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:14:02:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T14:02:12.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:12 smithi144 ceph-mon[33633]: pgmap v1793: 188 pgs: 188 active+clean; 871 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.4 KiB/s wr, 1 op/s 2024-03-21T14:02:12.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:12 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T14:02:12.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:12 smithi097 ceph-mon[39416]: pgmap v1793: 188 pgs: 188 active+clean; 871 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.4 KiB/s wr, 1 op/s 2024-03-21T14:02:12.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:12 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T14:02:12.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:12 smithi097 ceph-mon[30040]: pgmap v1793: 188 pgs: 188 active+clean; 871 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.4 KiB/s wr, 1 op/s 2024-03-21T14:02:12.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:12 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T14:02:14.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:14 smithi144 ceph-mon[33633]: pgmap v1794: 188 pgs: 188 active+clean; 871 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 1.2 KiB/s wr, 1 op/s 2024-03-21T14:02:14.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:14 smithi097 ceph-mon[39416]: pgmap v1794: 188 pgs: 188 active+clean; 871 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 1.2 KiB/s wr, 1 op/s 2024-03-21T14:02:14.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:14 smithi097 ceph-mon[30040]: pgmap v1794: 188 pgs: 188 active+clean; 871 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 1.2 KiB/s wr, 1 op/s 2024-03-21T14:02:15.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 14:02:15 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T14:02:16.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:16 smithi144 ceph-mon[33633]: pgmap v1795: 188 pgs: 188 active+clean; 871 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.0 KiB/s wr, 1 op/s 2024-03-21T14:02:16.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:16 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T14:02:16.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:16 smithi097 ceph-mon[39416]: pgmap v1795: 188 pgs: 188 active+clean; 871 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.0 KiB/s wr, 1 op/s 2024-03-21T14:02:16.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:16 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T14:02:16.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:16 smithi097 ceph-mon[30040]: pgmap v1795: 188 pgs: 188 active+clean; 871 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.0 KiB/s wr, 1 op/s 2024-03-21T14:02:16.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:16 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T14:02:18.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:18 smithi144 ceph-mon[33633]: pgmap v1796: 188 pgs: 188 active+clean; 871 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:02:18.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:18 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T14:02:18.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:18 smithi097 ceph-mon[39416]: pgmap v1796: 188 pgs: 188 active+clean; 871 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:02:18.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:18 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T14:02:18.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:18 smithi097 ceph-mon[30040]: pgmap v1796: 188 pgs: 188 active+clean; 871 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:02:18.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:18 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T14:02:19.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:19 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T14:02:19.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:19 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T14:02:19.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:19 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T14:02:19.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:19 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T14:02:19.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:19 smithi144 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T14:02:19.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:19 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T14:02:19.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:19 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T14:02:19.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:19 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T14:02:19.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:19 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T14:02:19.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:19 smithi097 ceph-mon[30040]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T14:02:19.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:19 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T14:02:19.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:19 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T14:02:19.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:19 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T14:02:19.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:19 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T14:02:19.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:19 smithi097 ceph-mon[39416]: from='mgr.24482 ' entity='mgr.y' 2024-03-21T14:02:20.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:20 smithi144 ceph-mon[33633]: pgmap v1797: 188 pgs: 188 active+clean; 871 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T14:02:20.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:20 smithi097 ceph-mon[39416]: pgmap v1797: 188 pgs: 188 active+clean; 871 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T14:02:20.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:20 smithi097 ceph-mon[30040]: pgmap v1797: 188 pgs: 188 active+clean; 871 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T14:02:22.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 14:02:21 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:14:02:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T14:02:22.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:22 smithi144 ceph-mon[33633]: pgmap v1798: 188 pgs: 188 active+clean; 871 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:02:22.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:22 smithi097 ceph-mon[39416]: pgmap v1798: 188 pgs: 188 active+clean; 871 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:02:22.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:22 smithi097 ceph-mon[30040]: pgmap v1798: 188 pgs: 188 active+clean; 871 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:02:24.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:24 smithi144 ceph-mon[33633]: pgmap v1799: 188 pgs: 188 active+clean; 871 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:02:24.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:24 smithi097 ceph-mon[39416]: pgmap v1799: 188 pgs: 188 active+clean; 871 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:02:24.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:24 smithi097 ceph-mon[30040]: pgmap v1799: 188 pgs: 188 active+clean; 871 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:02:25.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 14:02:25 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T14:02:26.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:26 smithi144 ceph-mon[33633]: pgmap v1800: 188 pgs: 188 active+clean; 871 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T14:02:26.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:26 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T14:02:26.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:26 smithi097 ceph-mon[39416]: pgmap v1800: 188 pgs: 188 active+clean; 871 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T14:02:26.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:26 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T14:02:26.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:26 smithi097 ceph-mon[30040]: pgmap v1800: 188 pgs: 188 active+clean; 871 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T14:02:26.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:26 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T14:02:27.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:27 smithi144 ceph-mon[33633]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T14:02:27.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:27 smithi097 ceph-mon[39416]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T14:02:27.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:27 smithi097 ceph-mon[30040]: from='mgr.24482 172.21.15.97:0/1825034220' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T14:02:28.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:28 smithi144 ceph-mon[33633]: pgmap v1801: 188 pgs: 188 active+clean; 871 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T14:02:28.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:28 smithi097 ceph-mon[39416]: pgmap v1801: 188 pgs: 188 active+clean; 871 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T14:02:28.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:28 smithi097 ceph-mon[30040]: pgmap v1801: 188 pgs: 188 active+clean; 871 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T14:02:30.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:30 smithi097 ceph-mon[39416]: pgmap v1802: 188 pgs: 188 active+clean; 871 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T14:02:30.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:30 smithi097 ceph-mon[30040]: pgmap v1802: 188 pgs: 188 active+clean; 871 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T14:02:30.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:30 smithi144 ceph-mon[33633]: pgmap v1802: 188 pgs: 188 active+clean; 871 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T14:02:32.303 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 14:02:31 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: ::ffff:172.21.15.144 - - [21/Mar/2024:14:02:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T14:02:32.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:32 smithi097 ceph-mon[39416]: pgmap v1803: 188 pgs: 188 active+clean; 871 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T14:02:32.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:32 smithi097 ceph-mon[30040]: pgmap v1803: 188 pgs: 188 active+clean; 871 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T14:02:32.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:32 smithi144 ceph-mon[33633]: pgmap v1803: 188 pgs: 188 active+clean; 871 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T14:02:33.308 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph osd pool delete b5a5e36b-8518-4a8b-80b9-4898af5bd2dd b5a5e36b-8518-4a8b-80b9-4898af5bd2dd --yes-i-really-really-mean-it 2024-03-21T14:02:33.406 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:33.403+0000 7f65442c9640 1 -- 172.21.15.97:0/27621570 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f653c100490 msgr2=0x7f653c100870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:02:33.406 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:33.403+0000 7f65442c9640 1 --2- 172.21.15.97:0/27621570 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f653c100490 0x7f653c100870 secure :-1 s=READY pgs=3251 cs=0 l=1 rev1=1 crypto rx=0x7f6538009ca0 tx=0x7f653802c0f0 comp rx=0 tx=0).stop 2024-03-21T14:02:33.406 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:33.403+0000 7f65442c9640 1 -- 172.21.15.97:0/27621570 shutdown_connections 2024-03-21T14:02:33.406 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:33.403+0000 7f65442c9640 1 --2- 172.21.15.97:0/27621570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f653c100db0 0x7f653c10db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:02:33.406 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:33.403+0000 7f65442c9640 1 --2- 172.21.15.97:0/27621570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f653c10e040 0x7f653c110430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:02:33.406 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:33.403+0000 7f65442c9640 1 --2- 172.21.15.97:0/27621570 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f653c100490 0x7f653c100870 unknown :-1 s=CLOSED pgs=3251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:02:33.406 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:33.403+0000 7f65442c9640 1 -- 172.21.15.97:0/27621570 >> 172.21.15.97:0/27621570 conn(0x7f653c0776c0 msgr2=0x7f653c077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T14:02:33.406 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:33.403+0000 7f65442c9640 1 -- 172.21.15.97:0/27621570 shutdown_connections 2024-03-21T14:02:33.406 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:33.403+0000 7f65442c9640 1 -- 172.21.15.97:0/27621570 wait complete. 2024-03-21T14:02:33.407 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:33.403+0000 7f65442c9640 1 Processor -- start 2024-03-21T14:02:33.407 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:33.403+0000 7f65442c9640 1 -- start start 2024-03-21T14:02:33.407 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:33.404+0000 7f65442c9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f653c100490 0x7f653c19f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T14:02:33.407 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:33.404+0000 7f65442c9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f653c100db0 0x7f653c19fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T14:02:33.407 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:33.404+0000 7f65442c9640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f653c10e040 0x7f653c199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T14:02:33.407 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:33.404+0000 7f65442c9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653c113380 con 0x7f653c100490 2024-03-21T14:02:33.407 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:33.404+0000 7f65442c9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653c113200 con 0x7f653c100db0 2024-03-21T14:02:33.407 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:33.404+0000 7f65442c9640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f653c113500 con 0x7f653c10e040 2024-03-21T14:02:33.407 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:33.404+0000 7f6542065640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f653c100490 0x7f653c19f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T14:02:33.407 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:33.404+0000 7f6542065640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f653c100490 0x7f653c19f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34230/0 (socket says 172.21.15.97:34230) 2024-03-21T14:02:33.407 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:33.404+0000 7f6542065640 1 -- 172.21.15.97:0/1023304906 learned_addr learned my addr 172.21.15.97:0/1023304906 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T14:02:33.407 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:33.404+0000 7f6542866640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f653c10e040 0x7f653c199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T14:02:33.407 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:33.404+0000 7f6541864640 1 --2- 172.21.15.97:0/1023304906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f653c100db0 0x7f653c19fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T14:02:33.407 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:33.404+0000 7f6542065640 1 -- 172.21.15.97:0/1023304906 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f653c10e040 msgr2=0x7f653c199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:02:33.407 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:33.404+0000 7f6542065640 1 --2- 172.21.15.97:0/1023304906 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f653c10e040 0x7f653c199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:02:33.408 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:33.404+0000 7f6542065640 1 -- 172.21.15.97:0/1023304906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f653c100db0 msgr2=0x7f653c19fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:02:33.408 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:33.404+0000 7f6542065640 1 --2- 172.21.15.97:0/1023304906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f653c100db0 0x7f653c19fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:02:33.408 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:33.404+0000 7f6542065640 1 -- 172.21.15.97:0/1023304906 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f653c199fd0 con 0x7f653c100490 2024-03-21T14:02:33.408 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:33.404+0000 7f6541864640 1 --2- 172.21.15.97:0/1023304906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f653c100db0 0x7f653c19fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T14:02:33.408 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:33.404+0000 7f6542065640 1 --2- 172.21.15.97:0/1023304906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f653c100490 0x7f653c19f650 secure :-1 s=READY pgs=3032 cs=0 l=1 rev1=1 crypto rx=0x7f6538005e30 tx=0x7f65380328f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T14:02:33.408 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:33.404+0000 7f65337fe640 1 -- 172.21.15.97:0/1023304906 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6538041020 con 0x7f653c100490 2024-03-21T14:02:33.408 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:33.404+0000 7f65442c9640 1 -- 172.21.15.97:0/1023304906 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f653c19a260 con 0x7f653c100490 2024-03-21T14:02:33.408 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:33.404+0000 7f65337fe640 1 -- 172.21.15.97:0/1023304906 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f6538003e20 con 0x7f653c100490 2024-03-21T14:02:33.408 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:33.404+0000 7f65337fe640 1 -- 172.21.15.97:0/1023304906 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6538039040 con 0x7f653c100490 2024-03-21T14:02:33.408 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:33.404+0000 7f65442c9640 1 -- 172.21.15.97:0/1023304906 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f653c1a65d0 con 0x7f653c100490 2024-03-21T14:02:33.408 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:33.405+0000 7f65442c9640 1 -- 172.21.15.97:0/1023304906 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f653c0691d0 con 0x7f653c100490 2024-03-21T14:02:33.409 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:33.406+0000 7f65337fe640 1 -- 172.21.15.97:0/1023304906 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 22) v1 ==== 92152+0+0 (secure 0 0 0) 0x7f6538044020 con 0x7f653c100490 2024-03-21T14:02:33.409 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:33.406+0000 7f65337fe640 1 --2- 172.21.15.97:0/1023304906 >> v2:172.21.15.97:6800/12494470 conn(0x7f652806e270 0x7f6528070730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T14:02:33.409 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:33.406+0000 7f65337fe640 1 -- 172.21.15.97:0/1023304906 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(801..801 src has 254..801) v4 ==== 8202+0+0 (secure 0 0 0) 0x7f65380b0d20 con 0x7f653c100490 2024-03-21T14:02:33.409 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:33.406+0000 7f6541864640 1 --2- 172.21.15.97:0/1023304906 >> v2:172.21.15.97:6800/12494470 conn(0x7f652806e270 0x7f6528070730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T14:02:33.409 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:33.407+0000 7f65337fe640 1 -- 172.21.15.97:0/1023304906 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=802}) v3 -- 0x7f652807a060 con 0x7f653c100490 2024-03-21T14:02:33.410 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:33.407+0000 7f6541864640 1 --2- 172.21.15.97:0/1023304906 >> v2:172.21.15.97:6800/12494470 conn(0x7f652806e270 0x7f6528070730 secure :-1 s=READY pgs=4066 cs=0 l=1 rev1=1 crypto rx=0x7f653c19af60 tx=0x7f652c005f90 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T14:02:33.411 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:33.411+0000 7f65337fe640 1 -- 172.21.15.97:0/1023304906 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f653804b020 con 0x7f653c100490 2024-03-21T14:02:33.589 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:33.589+0000 7f65442c9640 1 -- 172.21.15.97:0/1023304906 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "pool2": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f653c19ac40 con 0x7f653c100490 2024-03-21T14:02:34.377 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.376+0000 7f65337fe640 1 -- 172.21.15.97:0/1023304906 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "pool2": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "yes_i_really_really_mean_it": true}]=0 pool 'b5a5e36b-8518-4a8b-80b9-4898af5bd2dd' removed v802) v1 ==== 248+0+0 (secure 0 0 0) 0x7f653807dd10 con 0x7f653c100490 2024-03-21T14:02:34.377 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.376+0000 7f65337fe640 1 -- 172.21.15.97:0/1023304906 <== mon.0 v2:172.21.15.97:3300/0 8 ==== osd_map(802..802 src has 254..802) v4 ==== 296+0+0 (secure 0 0 0) 0x7f6538009890 con 0x7f653c100490 2024-03-21T14:02:34.377 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.377+0000 7f65337fe640 1 -- 172.21.15.97:0/1023304906 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=803}) v3 -- 0x7f652807ad10 con 0x7f653c100490 2024-03-21T14:02:34.494 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.493+0000 7f65442c9640 1 -- 172.21.15.97:0/1023304906 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "pool2": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f653c19a5c0 con 0x7f653c100490 2024-03-21T14:02:34.494 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.493+0000 7f65337fe640 1 -- 172.21.15.97:0/1023304906 <== mon.0 v2:172.21.15.97:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "pool2": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "yes_i_really_really_mean_it": true}]=0 pool 'b5a5e36b-8518-4a8b-80b9-4898af5bd2dd' does not exist v802) v1 ==== 255+0+0 (secure 0 0 0) 0x7f6538085f90 con 0x7f653c100490 2024-03-21T14:02:34.495 INFO:tasks.workunit.client.0.smithi097.stderr:pool 'b5a5e36b-8518-4a8b-80b9-4898af5bd2dd' does not exist 2024-03-21T14:02:34.496 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.495+0000 7f65442c9640 1 -- 172.21.15.97:0/1023304906 >> v2:172.21.15.97:6800/12494470 conn(0x7f652806e270 msgr2=0x7f6528070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:02:34.496 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.495+0000 7f65442c9640 1 --2- 172.21.15.97:0/1023304906 >> v2:172.21.15.97:6800/12494470 conn(0x7f652806e270 0x7f6528070730 secure :-1 s=READY pgs=4066 cs=0 l=1 rev1=1 crypto rx=0x7f653c19af60 tx=0x7f652c005f90 comp rx=0 tx=0).stop 2024-03-21T14:02:34.496 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.495+0000 7f65442c9640 1 -- 172.21.15.97:0/1023304906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f653c100490 msgr2=0x7f653c19f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:02:34.496 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.495+0000 7f65442c9640 1 --2- 172.21.15.97:0/1023304906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f653c100490 0x7f653c19f650 secure :-1 s=READY pgs=3032 cs=0 l=1 rev1=1 crypto rx=0x7f6538005e30 tx=0x7f65380328f0 comp rx=0 tx=0).stop 2024-03-21T14:02:34.496 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.495+0000 7f65442c9640 1 -- 172.21.15.97:0/1023304906 shutdown_connections 2024-03-21T14:02:34.496 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.495+0000 7f65442c9640 1 --2- 172.21.15.97:0/1023304906 >> v2:172.21.15.97:6800/12494470 conn(0x7f652806e270 0x7f6528070730 unknown :-1 s=CLOSED pgs=4066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:02:34.496 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.495+0000 7f65442c9640 1 --2- 172.21.15.97:0/1023304906 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f653c10e040 0x7f653c199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:02:34.496 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.495+0000 7f65442c9640 1 --2- 172.21.15.97:0/1023304906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f653c100db0 0x7f653c19fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:02:34.496 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.495+0000 7f65442c9640 1 --2- 172.21.15.97:0/1023304906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f653c100490 0x7f653c19f650 unknown :-1 s=CLOSED pgs=3032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:02:34.497 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.495+0000 7f65442c9640 1 -- 172.21.15.97:0/1023304906 >> 172.21.15.97:0/1023304906 conn(0x7f653c0776c0 msgr2=0x7f653c0ff480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T14:02:34.497 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.495+0000 7f65442c9640 1 -- 172.21.15.97:0/1023304906 shutdown_connections 2024-03-21T14:02:34.497 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.495+0000 7f65442c9640 1 -- 172.21.15.97:0/1023304906 wait complete. 2024-03-21T14:02:34.505 INFO:tasks.workunit.client.0.smithi097.stderr:+ ceph osd pool delete d7ed4d26-9388-40f6-9c6c-02152ee33b37 d7ed4d26-9388-40f6-9c6c-02152ee33b37 --yes-i-really-really-mean-it 2024-03-21T14:02:34.589 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.587+0000 7fe2497d0640 1 -- 172.21.15.97:0/2198424866 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe244103230 msgr2=0x7fe24410b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:02:34.590 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.587+0000 7fe2497d0640 1 --2- 172.21.15.97:0/2198424866 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe244103230 0x7fe24410b710 secure :-1 s=READY pgs=3033 cs=0 l=1 rev1=1 crypto rx=0x7fe238009900 tx=0x7fe23802bcd0 comp rx=0 tx=0).stop 2024-03-21T14:02:34.590 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.587+0000 7fe2497d0640 1 -- 172.21.15.97:0/2198424866 shutdown_connections 2024-03-21T14:02:34.590 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.587+0000 7fe2497d0640 1 --2- 172.21.15.97:0/2198424866 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe244103230 0x7fe24410b710 unknown :-1 s=CLOSED pgs=3033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:02:34.590 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.587+0000 7fe2497d0640 1 --2- 172.21.15.97:0/2198424866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe24410bce0 0x7fe24410e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:02:34.590 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.587+0000 7fe2497d0640 1 --2- 172.21.15.97:0/2198424866 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe244102910 0x7fe244102cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:02:34.590 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.587+0000 7fe2497d0640 1 -- 172.21.15.97:0/2198424866 >> 172.21.15.97:0/2198424866 conn(0x7fe244077690 msgr2=0x7fe244077aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T14:02:34.590 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.587+0000 7fe2497d0640 1 -- 172.21.15.97:0/2198424866 shutdown_connections 2024-03-21T14:02:34.590 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.587+0000 7fe2497d0640 1 -- 172.21.15.97:0/2198424866 wait complete. 2024-03-21T14:02:34.590 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.587+0000 7fe2497d0640 1 Processor -- start 2024-03-21T14:02:34.590 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.588+0000 7fe2497d0640 1 -- start start 2024-03-21T14:02:34.590 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.588+0000 7fe2497d0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe244102910 0x7fe244100560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T14:02:34.590 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.588+0000 7fe2497d0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe244103230 0x7fe244100aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T14:02:34.590 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.588+0000 7fe2497d0640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe24410bce0 0x7fe244100fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T14:02:34.591 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.588+0000 7fe2497d0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe244111180 con 0x7fe244103230 2024-03-21T14:02:34.591 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.588+0000 7fe2497d0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe244111000 con 0x7fe244102910 2024-03-21T14:02:34.591 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.588+0000 7fe2497d0640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe244111300 con 0x7fe24410bce0 2024-03-21T14:02:34.591 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.588+0000 7fe2427fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe244103230 0x7fe244100aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T14:02:34.591 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.588+0000 7fe2437fe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe24410bce0 0x7fe244100fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T14:02:34.591 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.588+0000 7fe2427fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe244103230 0x7fe244100aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34248/0 (socket says 172.21.15.97:34248) 2024-03-21T14:02:34.591 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.588+0000 7fe2427fc640 1 -- 172.21.15.97:0/4045524637 learned_addr learned my addr 172.21.15.97:0/4045524637 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-21T14:02:34.591 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.588+0000 7fe242ffd640 1 --2- 172.21.15.97:0/4045524637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe244102910 0x7fe244100560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T14:02:34.591 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.588+0000 7fe2427fc640 1 -- 172.21.15.97:0/4045524637 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe24410bce0 msgr2=0x7fe244100fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:02:34.591 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.588+0000 7fe2427fc640 1 --2- 172.21.15.97:0/4045524637 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe24410bce0 0x7fe244100fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:02:34.591 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.588+0000 7fe2427fc640 1 -- 172.21.15.97:0/4045524637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe244102910 msgr2=0x7fe244100560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:02:34.591 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.588+0000 7fe2427fc640 1 --2- 172.21.15.97:0/4045524637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe244102910 0x7fe244100560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:02:34.591 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.588+0000 7fe2427fc640 1 -- 172.21.15.97:0/4045524637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2441a5bd0 con 0x7fe244103230 2024-03-21T14:02:34.591 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.588+0000 7fe2437fe640 1 --2- 172.21.15.97:0/4045524637 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe24410bce0 0x7fe244100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T14:02:34.591 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.588+0000 7fe2427fc640 1 --2- 172.21.15.97:0/4045524637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe244103230 0x7fe244100aa0 secure :-1 s=READY pgs=3034 cs=0 l=1 rev1=1 crypto rx=0x7fe238005940 tx=0x7fe23802c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T14:02:34.592 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.588+0000 7fe22bfff640 1 -- 172.21.15.97:0/4045524637 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe238043020 con 0x7fe244103230 2024-03-21T14:02:34.592 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.589+0000 7fe22bfff640 1 -- 172.21.15.97:0/4045524637 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fe2380043d0 con 0x7fe244103230 2024-03-21T14:02:34.592 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.589+0000 7fe2497d0640 1 -- 172.21.15.97:0/4045524637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2441a5e60 con 0x7fe244103230 2024-03-21T14:02:34.592 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.589+0000 7fe22bfff640 1 -- 172.21.15.97:0/4045524637 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe238003ae0 con 0x7fe244103230 2024-03-21T14:02:34.592 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.589+0000 7fe2497d0640 1 -- 172.21.15.97:0/4045524637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2441a6320 con 0x7fe244103230 2024-03-21T14:02:34.593 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.590+0000 7fe2497d0640 1 -- 172.21.15.97:0/4045524637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2440689d0 con 0x7fe244103230 2024-03-21T14:02:34.593 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.590+0000 7fe22bfff640 1 -- 172.21.15.97:0/4045524637 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 22) v1 ==== 92152+0+0 (secure 0 0 0) 0x7fe238046020 con 0x7fe244103230 2024-03-21T14:02:34.593 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.590+0000 7fe22bfff640 1 --2- 172.21.15.97:0/4045524637 >> v2:172.21.15.97:6800/12494470 conn(0x7fe22c06e270 0x7fe22c070730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T14:02:34.593 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.590+0000 7fe22bfff640 1 -- 172.21.15.97:0/4045524637 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(802..802 src has 254..802) v4 ==== 7814+0+0 (secure 0 0 0) 0x7fe2380b0c40 con 0x7fe244103230 2024-03-21T14:02:34.593 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.590+0000 7fe242ffd640 1 --2- 172.21.15.97:0/4045524637 >> v2:172.21.15.97:6800/12494470 conn(0x7fe22c06e270 0x7fe22c070730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T14:02:34.593 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.591+0000 7fe22bfff640 1 -- 172.21.15.97:0/4045524637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=803}) v3 -- 0x7fe22c0797c0 con 0x7fe244103230 2024-03-21T14:02:34.593 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.591+0000 7fe242ffd640 1 --2- 172.21.15.97:0/4045524637 >> v2:172.21.15.97:6800/12494470 conn(0x7fe22c06e270 0x7fe22c070730 secure :-1 s=READY pgs=4067 cs=0 l=1 rev1=1 crypto rx=0x7fe2340095f0 tx=0x7fe234005f90 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T14:02:34.595 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.595+0000 7fe22bfff640 1 -- 172.21.15.97:0/4045524637 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe23802ce20 con 0x7fe244103230 2024-03-21T14:02:34.770 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:34.770+0000 7fe2497d0640 1 -- 172.21.15.97:0/4045524637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "pool2": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe2441a64f0 con 0x7fe244103230 2024-03-21T14:02:34.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:34 smithi097 ceph-mon[39416]: pgmap v1804: 188 pgs: 188 active+clean; 871 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:02:34.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:34 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1023304906' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "pool2": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T14:02:34.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:34 smithi097 ceph-mon[30040]: pgmap v1804: 188 pgs: 188 active+clean; 871 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:02:34.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:34 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1023304906' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "pool2": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T14:02:34.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:34 smithi144 ceph-mon[33633]: pgmap v1804: 188 pgs: 188 active+clean; 871 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T14:02:34.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:34 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1023304906' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "pool2": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T14:02:35.395 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:35.394+0000 7fe22bfff640 1 -- 172.21.15.97:0/4045524637 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "pool2": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "yes_i_really_really_mean_it": true}]=0 pool 'd7ed4d26-9388-40f6-9c6c-02152ee33b37' removed v803) v1 ==== 248+0+0 (secure 0 0 0) 0x7fe23804d020 con 0x7fe244103230 2024-03-21T14:02:35.395 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:35.394+0000 7fe22bfff640 1 -- 172.21.15.97:0/4045524637 <== mon.0 v2:172.21.15.97:3300/0 8 ==== osd_map(803..803 src has 254..803) v4 ==== 296+0+0 (secure 0 0 0) 0x7fe2380aea40 con 0x7fe244103230 2024-03-21T14:02:35.395 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:35.395+0000 7fe22bfff640 1 -- 172.21.15.97:0/4045524637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=804}) v3 -- 0x7fe22c07a440 con 0x7fe244103230 2024-03-21T14:02:35.508 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:35.508+0000 7fe2497d0640 1 -- 172.21.15.97:0/4045524637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "pool2": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe2441a6770 con 0x7fe244103230 2024-03-21T14:02:35.509 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:35.508+0000 7fe22bfff640 1 -- 172.21.15.97:0/4045524637 <== mon.0 v2:172.21.15.97:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "pool2": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "yes_i_really_really_mean_it": true}]=0 pool 'd7ed4d26-9388-40f6-9c6c-02152ee33b37' does not exist v803) v1 ==== 255+0+0 (secure 0 0 0) 0x7fe23807dd80 con 0x7fe244103230 2024-03-21T14:02:35.509 INFO:tasks.workunit.client.0.smithi097.stderr:pool 'd7ed4d26-9388-40f6-9c6c-02152ee33b37' does not exist 2024-03-21T14:02:35.514 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:35.510+0000 7fe2497d0640 1 -- 172.21.15.97:0/4045524637 >> v2:172.21.15.97:6800/12494470 conn(0x7fe22c06e270 msgr2=0x7fe22c070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:02:35.514 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:35.510+0000 7fe2497d0640 1 --2- 172.21.15.97:0/4045524637 >> v2:172.21.15.97:6800/12494470 conn(0x7fe22c06e270 0x7fe22c070730 secure :-1 s=READY pgs=4067 cs=0 l=1 rev1=1 crypto rx=0x7fe2340095f0 tx=0x7fe234005f90 comp rx=0 tx=0).stop 2024-03-21T14:02:35.514 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:35.510+0000 7fe2497d0640 1 -- 172.21.15.97:0/4045524637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe244103230 msgr2=0x7fe244100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T14:02:35.514 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:35.510+0000 7fe2497d0640 1 --2- 172.21.15.97:0/4045524637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe244103230 0x7fe244100aa0 secure :-1 s=READY pgs=3034 cs=0 l=1 rev1=1 crypto rx=0x7fe238005940 tx=0x7fe23802c870 comp rx=0 tx=0).stop 2024-03-21T14:02:35.514 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:35.510+0000 7fe2497d0640 1 -- 172.21.15.97:0/4045524637 shutdown_connections 2024-03-21T14:02:35.514 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:35.510+0000 7fe2497d0640 1 --2- 172.21.15.97:0/4045524637 >> v2:172.21.15.97:6800/12494470 conn(0x7fe22c06e270 0x7fe22c070730 unknown :-1 s=CLOSED pgs=4067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:02:35.514 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:35.510+0000 7fe2497d0640 1 --2- 172.21.15.97:0/4045524637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe244103230 0x7fe244100aa0 unknown :-1 s=CLOSED pgs=3034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:02:35.514 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:35.510+0000 7fe2497d0640 1 --2- 172.21.15.97:0/4045524637 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe24410bce0 0x7fe244100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:02:35.514 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:35.510+0000 7fe2497d0640 1 --2- 172.21.15.97:0/4045524637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe244102910 0x7fe244100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T14:02:35.514 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:35.510+0000 7fe2497d0640 1 -- 172.21.15.97:0/4045524637 >> 172.21.15.97:0/4045524637 conn(0x7fe244077690 msgr2=0x7fe2440fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T14:02:35.514 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:35.511+0000 7fe2497d0640 1 -- 172.21.15.97:0/4045524637 shutdown_connections 2024-03-21T14:02:35.514 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-21T14:02:35.511+0000 7fe2497d0640 1 -- 172.21.15.97:0/4045524637 wait complete. 2024-03-21T14:02:35.520 INFO:tasks.workunit.client.0.smithi097.stderr:+ echo OK 2024-03-21T14:02:35.521 INFO:tasks.workunit.client.0.smithi097.stdout:OK 2024-03-21T14:02:35.521 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-21T14:02:35.521 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-21T14:02:35.590 INFO:tasks.workunit:Stopping ['rados/test.sh', 'rados/test_pool_quota.sh'] on client.0... 2024-03-21T14:02:35.590 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-21T14:02:35.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:35 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1023304906' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "pool2": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "yes_i_really_really_mean_it": true}]': finished 2024-03-21T14:02:35.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:35 smithi144 ceph-mon[33633]: osdmap e802: 8 total, 8 up, 8 in 2024-03-21T14:02:35.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:35 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/1023304906' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "pool2": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T14:02:35.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:35 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4045524637' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "pool2": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T14:02:35.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 14:02:35 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug there is no tcmu-runner data available 2024-03-21T14:02:35.654 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:35 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1023304906' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "pool2": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "yes_i_really_really_mean_it": true}]': finished 2024-03-21T14:02:35.654 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:35 smithi097 ceph-mon[39416]: osdmap e802: 8 total, 8 up, 8 in 2024-03-21T14:02:35.654 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:35 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/1023304906' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "pool2": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T14:02:35.654 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:35 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4045524637' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "pool2": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T14:02:35.655 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:35 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1023304906' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "pool2": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "yes_i_really_really_mean_it": true}]': finished 2024-03-21T14:02:35.655 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:35 smithi097 ceph-mon[30040]: osdmap e802: 8 total, 8 up, 8 in 2024-03-21T14:02:35.655 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:35 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/1023304906' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "pool2": "b5a5e36b-8518-4a8b-80b9-4898af5bd2dd", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T14:02:35.655 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:35 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4045524637' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "pool2": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T14:02:35.894 DEBUG:teuthology.parallel:result is None 2024-03-21T14:02:35.895 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-21T14:02:35.961 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-21T14:02:35.962 DEBUG:teuthology.orchestra.run.smithi097:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-03-21T14:02:36.029 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-03-21T14:02:36.029 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-21T14:02:36.038 INFO:tasks.cephadm:Teardown begin 2024-03-21T14:02:36.039 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-21T14:02:36.103 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-21T14:02:36.139 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-21T14:02:36.139 DEBUG:teuthology.orchestra.run.smithi097:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-21T14:02:36.158 DEBUG:teuthology.orchestra.run.smithi144:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-21T14:02:36.196 INFO:tasks.cephadm:Stopping all daemons... 2024-03-21T14:02:36.196 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-03-21T14:02:36.196 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@mon.a 2024-03-21T14:02:36.511 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:36 smithi097 ceph-mon[39416]: pgmap v1806: 176 pgs: 176 active+clean; 698 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T14:02:36.511 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:36 smithi097 ceph-mon[39416]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T14:02:36.511 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:36 smithi097 ceph-mon[39416]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-21T14:02:36.511 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:36 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4045524637' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "pool2": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "yes_i_really_really_mean_it": true}]': finished 2024-03-21T14:02:36.511 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:36 smithi097 ceph-mon[39416]: osdmap e803: 8 total, 8 up, 8 in 2024-03-21T14:02:36.511 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:36 smithi097 ceph-mon[39416]: from='client.? 172.21.15.97:0/4045524637' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "pool2": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T14:02:36.511 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:36 smithi097 systemd[1]: Stopping Ceph mon.a for a5dd1f32-e785-11ee-95cd-87774f69a715... 2024-03-21T14:02:36.511 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:36 smithi097 ceph-mon[30040]: pgmap v1806: 176 pgs: 176 active+clean; 698 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T14:02:36.511 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:36 smithi097 ceph-mon[30040]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T14:02:36.511 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:36 smithi097 ceph-mon[30040]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-21T14:02:36.511 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:36 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4045524637' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "pool2": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "yes_i_really_really_mean_it": true}]': finished 2024-03-21T14:02:36.511 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:36 smithi097 ceph-mon[30040]: osdmap e803: 8 total, 8 up, 8 in 2024-03-21T14:02:36.512 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:36 smithi097 ceph-mon[30040]: from='client.? 172.21.15.97:0/4045524637' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "pool2": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T14:02:36.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:36 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mon-a[30017]: 2024-03-21T14:02:36.509+0000 7f9d802bf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T14:02:36.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:36 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mon-a[30017]: 2024-03-21T14:02:36.509+0000 7f9d802bf700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-03-21T14:02:36.803 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:36 smithi097 podman[157548]: 2024-03-21 14:02:36.561809841 +0000 UTC m=+0.182638719 container died d6b35fa2e2973371234c332ac19ae07729d357ab56ff0c4b075b0092a84f4b5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mon-a, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T14:02:36.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:36 smithi144 ceph-mon[33633]: pgmap v1806: 176 pgs: 176 active+clean; 698 KiB data, 937 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T14:02:36.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:36 smithi144 ceph-mon[33633]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T14:02:36.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:36 smithi144 ceph-mon[33633]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-21T14:02:36.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:36 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4045524637' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "pool2": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "yes_i_really_really_mean_it": true}]': finished 2024-03-21T14:02:36.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:36 smithi144 ceph-mon[33633]: osdmap e803: 8 total, 8 up, 8 in 2024-03-21T14:02:36.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:36 smithi144 ceph-mon[33633]: from='client.? 172.21.15.97:0/4045524637' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "pool2": "d7ed4d26-9388-40f6-9c6c-02152ee33b37", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T14:02:37.201 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:36 smithi097 podman[157548]: 2024-03-21 14:02:36.999873107 +0000 UTC m=+0.620701940 container cleanup d6b35fa2e2973371234c332ac19ae07729d357ab56ff0c4b075b0092a84f4b5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mon-a, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T14:02:37.201 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 21 14:02:37 smithi097 bash[157548]: ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mon-a 2024-03-21T14:02:37.423 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@mon.a.service' 2024-03-21T14:02:37.463 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T14:02:37.463 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-03-21T14:02:37.463 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-03-21T14:02:37.463 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@mon.c 2024-03-21T14:02:37.781 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:37 smithi097 systemd[1]: Stopping Ceph mon.c for a5dd1f32-e785-11ee-95cd-87774f69a715... 2024-03-21T14:02:38.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:37 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mon-c[39393]: 2024-03-21T14:02:37.780+0000 7facd6c3a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T14:02:38.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:37 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mon-c[39393]: 2024-03-21T14:02:37.780+0000 7facd6c3a700 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-03-21T14:02:38.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:37 smithi097 podman[157681]: 2024-03-21 14:02:37.934899819 +0000 UTC m=+0.249233629 container died e696409da010072945642e8c0109c725327d14b9c04a17faa462c3069fe12552 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mon-c, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_BRANCH=HEAD) 2024-03-21T14:02:38.054 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 14:02:37 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: [21/Mar/2024:14:02:37] ENGINE Bus STOPPING 2024-03-21T14:02:38.054 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 14:02:37 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: [21/Mar/2024:14:02:37] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-03-21T14:02:38.054 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 14:02:37 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: [21/Mar/2024:14:02:37] ENGINE Bus STOPPED 2024-03-21T14:02:38.054 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 14:02:37 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: [21/Mar/2024:14:02:37] ENGINE Bus STARTING 2024-03-21T14:02:38.054 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 14:02:37 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: [21/Mar/2024:14:02:37] ENGINE Serving on http://:::9283 2024-03-21T14:02:38.054 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 14:02:37 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: [21/Mar/2024:14:02:37] ENGINE Bus STARTED 2024-03-21T14:02:38.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:38 smithi097 podman[157681]: 2024-03-21 14:02:38.312778915 +0000 UTC m=+0.627112722 container cleanup e696409da010072945642e8c0109c725327d14b9c04a17faa462c3069fe12552 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T14:02:38.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Mar 21 14:02:38 smithi097 bash[157681]: ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mon-c 2024-03-21T14:02:38.779 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@mon.c.service' 2024-03-21T14:02:38.815 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T14:02:38.815 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-03-21T14:02:38.815 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-03-21T14:02:38.815 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@mon.b 2024-03-21T14:02:39.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:38 smithi144 systemd[1]: Stopping Ceph mon.b for a5dd1f32-e785-11ee-95cd-87774f69a715... 2024-03-21T14:02:39.558 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:39 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mon-b[33610]: 2024-03-21T14:02:39.125+0000 7f8705ac2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T14:02:39.558 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:39 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mon-b[33610]: 2024-03-21T14:02:39.125+0000 7f8705ac2700 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-03-21T14:02:39.558 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:39 smithi144 podman[78414]: 2024-03-21 14:02:39.269376831 +0000 UTC m=+0.252092465 container died 97da8a80ab456ec9d311a685feccc89d5866182051c55185cd5302459da56b00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mon-b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True) 2024-03-21T14:02:39.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:39 smithi144 podman[78414]: 2024-03-21 14:02:39.749581565 +0000 UTC m=+0.732297248 container cleanup 97da8a80ab456ec9d311a685feccc89d5866182051c55185cd5302459da56b00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-21T14:02:39.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 21 14:02:39 smithi144 bash[78414]: ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mon-b 2024-03-21T14:02:40.119 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@mon.b.service' 2024-03-21T14:02:40.181 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T14:02:40.181 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-03-21T14:02:40.181 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-03-21T14:02:40.181 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@mgr.y 2024-03-21T14:02:40.478 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 14:02:40 smithi097 systemd[1]: Stopping Ceph mgr.y for a5dd1f32-e785-11ee-95cd-87774f69a715... 2024-03-21T14:02:40.803 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 14:02:40 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T14:02:40.477+0000 7f763c42e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T14:02:40.803 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 14:02:40 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y[30298]: 2024-03-21T14:02:40.477+0000 7f763c42e700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-03-21T14:02:40.803 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 14:02:40 smithi097 podman[157823]: 2024-03-21 14:02:40.510186385 +0000 UTC m=+0.137297107 container died 6c7ef8cfbe19127a76518a69d0acabf8a5ce304c0734df85b6a001986eba5de1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T14:02:41.143 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 14:02:40 smithi097 podman[157823]: 2024-03-21 14:02:40.892689146 +0000 UTC m=+0.519799857 container cleanup 6c7ef8cfbe19127a76518a69d0acabf8a5ce304c0734df85b6a001986eba5de1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-21T14:02:41.144 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Mar 21 14:02:40 smithi097 bash[157823]: ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-y 2024-03-21T14:02:41.319 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@mgr.y.service' 2024-03-21T14:02:41.355 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T14:02:41.356 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-03-21T14:02:41.356 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-03-21T14:02:41.356 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@mgr.x 2024-03-21T14:02:41.676 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 14:02:41 smithi144 systemd[1]: Stopping Ceph mgr.x for a5dd1f32-e785-11ee-95cd-87774f69a715... 2024-03-21T14:02:41.963 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 14:02:41 smithi144 podman[78547]: 2024-03-21 14:02:41.677506019 +0000 UTC m=+0.124123272 container died 7b6751b4e086b5441292b2e07303cdd91e646bf423bed6b6d60e487ac1a0b70f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-21T14:02:42.313 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 14:02:42 smithi144 podman[78547]: 2024-03-21 14:02:42.037327087 +0000 UTC m=+0.483944342 container cleanup 7b6751b4e086b5441292b2e07303cdd91e646bf423bed6b6d60e487ac1a0b70f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-21T14:02:42.313 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Mar 21 14:02:42 smithi144 bash[78547]: ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-mgr-x 2024-03-21T14:02:42.449 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@mgr.x.service' 2024-03-21T14:02:42.486 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T14:02:42.487 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-03-21T14:02:42.487 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-21T14:02:42.487 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@osd.0 2024-03-21T14:02:42.803 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 21 14:02:42 smithi097 systemd[1]: Stopping Ceph osd.0 for a5dd1f32-e785-11ee-95cd-87774f69a715... 2024-03-21T14:02:43.062 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 21 14:02:43 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-0[43810]: 2024-03-21T14:02:43.007+0000 7f4675d2f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T14:02:43.062 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 21 14:02:43 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-0[43810]: 2024-03-21T14:02:43.007+0000 7f4675d2f700 -1 osd.0 803 *** Got signal Terminated *** 2024-03-21T14:02:43.062 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 21 14:02:43 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-0[43810]: 2024-03-21T14:02:43.007+0000 7f4675d2f700 -1 osd.0 803 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T14:02:48.798 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 21 14:02:48 smithi097 podman[157955]: 2024-03-21 14:02:48.536986466 +0000 UTC m=+5.862744023 container died d36ca525ca7b66c918d4be07fcc03bceaa3b9b0d8a90d631f7faff86250c571f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-03-21T14:02:49.053 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 21 14:02:48 smithi097 podman[157955]: 2024-03-21 14:02:48.897027237 +0000 UTC m=+6.222784786 container cleanup d36ca525ca7b66c918d4be07fcc03bceaa3b9b0d8a90d631f7faff86250c571f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-0, org.label-schema.build-date=20240319, ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-21T14:02:49.053 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 21 14:02:48 smithi097 bash[157955]: ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-0 2024-03-21T14:02:49.515 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 21 14:02:49 smithi097 podman[157990]: 2024-03-21 14:02:49.165119764 +0000 UTC m=+0.630494971 container remove d36ca525ca7b66c918d4be07fcc03bceaa3b9b0d8a90d631f7faff86250c571f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T14:02:49.803 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 21 14:02:49 smithi097 podman[158088]: 2024-03-21 14:02:49.419810067 +0000 UTC m=+0.018843181 image pull 19e12293512ff7978c8adb5f72a829060dd47f07bfd6903b6b3178d2ebc29e4e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93 2024-03-21T14:02:49.803 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 21 14:02:49 smithi097 podman[158088]: 2024-03-21 14:02:49.532651706 +0000 UTC m=+0.131684806 container create 9ffa015e8b27c7aefb5b6c7d14d94ca882480ad7a86bba0005c11c697fb87c6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T14:02:49.803 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 21 14:02:49 smithi097 podman[158088]: 2024-03-21 14:02:49.751697634 +0000 UTC m=+0.350730749 container init 9ffa015e8b27c7aefb5b6c7d14d94ca882480ad7a86bba0005c11c697fb87c6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.build-date=20240319, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-21T14:02:49.804 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 21 14:02:49 smithi097 podman[158088]: 2024-03-21 14:02:49.757107284 +0000 UTC m=+0.356140386 container start 9ffa015e8b27c7aefb5b6c7d14d94ca882480ad7a86bba0005c11c697fb87c6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240319) 2024-03-21T14:02:50.303 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 21 14:02:49 smithi097 podman[158088]: 2024-03-21 14:02:49.834783854 +0000 UTC m=+0.433816962 container attach 9ffa015e8b27c7aefb5b6c7d14d94ca882480ad7a86bba0005c11c697fb87c6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-21T14:02:50.303 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 21 14:02:50 smithi097 podman[158088]: 2024-03-21 14:02:50.045829146 +0000 UTC m=+0.644862239 container died 9ffa015e8b27c7aefb5b6c7d14d94ca882480ad7a86bba0005c11c697fb87c6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-0-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240319, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-21T14:02:50.711 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@osd.0.service' 2024-03-21T14:02:50.752 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T14:02:50.753 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-21T14:02:50.753 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-21T14:02:50.753 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@osd.1 2024-03-21T14:02:51.053 INFO:journalctl@ceph.osd.1.smithi097.stdout:Mar 21 14:02:50 smithi097 systemd[1]: Stopping Ceph osd.1 for a5dd1f32-e785-11ee-95cd-87774f69a715... 2024-03-21T14:02:51.553 INFO:journalctl@ceph.osd.1.smithi097.stdout:Mar 21 14:02:51 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-1[49765]: 2024-03-21T14:02:51.085+0000 7fc97776a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T14:02:51.553 INFO:journalctl@ceph.osd.1.smithi097.stdout:Mar 21 14:02:51 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-1[49765]: 2024-03-21T14:02:51.085+0000 7fc97776a700 -1 osd.1 803 *** Got signal Terminated *** 2024-03-21T14:02:51.553 INFO:journalctl@ceph.osd.1.smithi097.stdout:Mar 21 14:02:51 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-1[49765]: 2024-03-21T14:02:51.085+0000 7fc97776a700 -1 osd.1 803 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T14:02:56.804 INFO:journalctl@ceph.osd.1.smithi097.stdout:Mar 21 14:02:56 smithi097 podman[158222]: 2024-03-21 14:02:56.547973829 +0000 UTC m=+5.573164256 container died 0e19eb0d7a9ef75cf729835fecf5f3282527eded1b2a985f4755adb623627d5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-1, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe) 2024-03-21T14:02:57.200 INFO:journalctl@ceph.osd.1.smithi097.stdout:Mar 21 14:02:56 smithi097 podman[158222]: 2024-03-21 14:02:56.908027609 +0000 UTC m=+5.933218036 container cleanup 0e19eb0d7a9ef75cf729835fecf5f3282527eded1b2a985f4755adb623627d5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-1, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T14:02:57.200 INFO:journalctl@ceph.osd.1.smithi097.stdout:Mar 21 14:02:56 smithi097 bash[158222]: ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-1 2024-03-21T14:02:57.529 INFO:journalctl@ceph.osd.1.smithi097.stdout:Mar 21 14:02:57 smithi097 podman[158244]: 2024-03-21 14:02:57.200553241 +0000 UTC m=+0.653721599 container remove 0e19eb0d7a9ef75cf729835fecf5f3282527eded1b2a985f4755adb623627d5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-1, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T14:02:57.803 INFO:journalctl@ceph.osd.1.smithi097.stdout:Mar 21 14:02:57 smithi097 podman[158344]: 2024-03-21 14:02:57.434203367 +0000 UTC m=+0.018612382 image pull 19e12293512ff7978c8adb5f72a829060dd47f07bfd6903b6b3178d2ebc29e4e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93 2024-03-21T14:02:57.803 INFO:journalctl@ceph.osd.1.smithi097.stdout:Mar 21 14:02:57 smithi097 podman[158344]: 2024-03-21 14:02:57.535053487 +0000 UTC m=+0.119462545 container create 2fd835d64bfab9fe3f5b28af0e57d7bb44f6ad98310aa2ee5d655b88116653ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe) 2024-03-21T14:02:57.803 INFO:journalctl@ceph.osd.1.smithi097.stdout:Mar 21 14:02:57 smithi097 podman[158344]: 2024-03-21 14:02:57.737340368 +0000 UTC m=+0.321749384 container init 2fd835d64bfab9fe3f5b28af0e57d7bb44f6ad98310aa2ee5d655b88116653ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-21T14:02:57.804 INFO:journalctl@ceph.osd.1.smithi097.stdout:Mar 21 14:02:57 smithi097 podman[158344]: 2024-03-21 14:02:57.74307034 +0000 UTC m=+0.327479348 container start 2fd835d64bfab9fe3f5b28af0e57d7bb44f6ad98310aa2ee5d655b88116653ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-1-deactivate, ceph=True, GIT_CLEAN=True, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-21T14:02:58.304 INFO:journalctl@ceph.osd.1.smithi097.stdout:Mar 21 14:02:57 smithi097 podman[158344]: 2024-03-21 14:02:57.837027351 +0000 UTC m=+0.421436355 container attach 2fd835d64bfab9fe3f5b28af0e57d7bb44f6ad98310aa2ee5d655b88116653ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T14:02:58.304 INFO:journalctl@ceph.osd.1.smithi097.stdout:Mar 21 14:02:58 smithi097 podman[158344]: 2024-03-21 14:02:58.034405763 +0000 UTC m=+0.618814761 container died 2fd835d64bfab9fe3f5b28af0e57d7bb44f6ad98310aa2ee5d655b88116653ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-1-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T14:02:58.583 INFO:journalctl@ceph.osd.1.smithi097.stdout:Mar 21 14:02:58 smithi097 podman[158390]: 2024-03-21 14:02:58.581897312 +0000 UTC m=+0.542191450 container remove 2fd835d64bfab9fe3f5b28af0e57d7bb44f6ad98310aa2ee5d655b88116653ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD) 2024-03-21T14:02:58.607 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@osd.1.service' 2024-03-21T14:02:58.647 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T14:02:58.648 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-21T14:02:58.648 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-21T14:02:58.648 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@osd.2 2024-03-21T14:02:58.979 INFO:journalctl@ceph.osd.2.smithi097.stdout:Mar 21 14:02:58 smithi097 systemd[1]: Stopping Ceph osd.2 for a5dd1f32-e785-11ee-95cd-87774f69a715... 2024-03-21T14:02:59.303 INFO:journalctl@ceph.osd.2.smithi097.stdout:Mar 21 14:02:58 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-2[56207]: 2024-03-21T14:02:58.978+0000 7fa8985fe700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T14:02:59.303 INFO:journalctl@ceph.osd.2.smithi097.stdout:Mar 21 14:02:58 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-2[56207]: 2024-03-21T14:02:58.978+0000 7fa8985fe700 -1 osd.2 803 *** Got signal Terminated *** 2024-03-21T14:02:59.303 INFO:journalctl@ceph.osd.2.smithi097.stdout:Mar 21 14:02:58 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-2[56207]: 2024-03-21T14:02:58.978+0000 7fa8985fe700 -1 osd.2 803 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T14:03:04.652 INFO:journalctl@ceph.osd.2.smithi097.stdout:Mar 21 14:03:04 smithi097 podman[158474]: 2024-03-21 14:03:04.376056306 +0000 UTC m=+5.506006076 container died 7412cfcf7f1134e2d0ec3b631b18b5ada5567251ffa981f5df815d607590947c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-2, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True) 2024-03-21T14:03:05.018 INFO:journalctl@ceph.osd.2.smithi097.stdout:Mar 21 14:03:04 smithi097 podman[158474]: 2024-03-21 14:03:04.751188524 +0000 UTC m=+5.881138302 container cleanup 7412cfcf7f1134e2d0ec3b631b18b5ada5567251ffa981f5df815d607590947c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240319, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-03-21T14:03:05.018 INFO:journalctl@ceph.osd.2.smithi097.stdout:Mar 21 14:03:04 smithi097 bash[158474]: ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-2 2024-03-21T14:03:05.303 INFO:journalctl@ceph.osd.2.smithi097.stdout:Mar 21 14:03:05 smithi097 podman[158496]: 2024-03-21 14:03:05.018943389 +0000 UTC m=+0.635894220 container remove 7412cfcf7f1134e2d0ec3b631b18b5ada5567251ffa981f5df815d607590947c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-03-21T14:03:05.663 INFO:journalctl@ceph.osd.2.smithi097.stdout:Mar 21 14:03:05 smithi097 podman[158598]: 2024-03-21 14:03:05.353117027 +0000 UTC m=+0.098213152 container create aaa26604f07a67d8b9205b064a0e68059529159449b066dfde39542d839b5778 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-2-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD) 2024-03-21T14:03:05.663 INFO:journalctl@ceph.osd.2.smithi097.stdout:Mar 21 14:03:05 smithi097 podman[158598]: 2024-03-21 14:03:05.272178002 +0000 UTC m=+0.017274146 image pull 19e12293512ff7978c8adb5f72a829060dd47f07bfd6903b6b3178d2ebc29e4e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93 2024-03-21T14:03:05.663 INFO:journalctl@ceph.osd.2.smithi097.stdout:Mar 21 14:03:05 smithi097 podman[158598]: 2024-03-21 14:03:05.580373732 +0000 UTC m=+0.325469864 container init aaa26604f07a67d8b9205b064a0e68059529159449b066dfde39542d839b5778 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-2-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T14:03:05.663 INFO:journalctl@ceph.osd.2.smithi097.stdout:Mar 21 14:03:05 smithi097 podman[158598]: 2024-03-21 14:03:05.585503875 +0000 UTC m=+0.330600001 container start aaa26604f07a67d8b9205b064a0e68059529159449b066dfde39542d839b5778 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-2-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319) 2024-03-21T14:03:05.663 INFO:journalctl@ceph.osd.2.smithi097.stdout:Mar 21 14:03:05 smithi097 podman[158598]: 2024-03-21 14:03:05.663598592 +0000 UTC m=+0.408694734 container attach aaa26604f07a67d8b9205b064a0e68059529159449b066dfde39542d839b5778 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-2-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2) 2024-03-21T14:03:06.053 INFO:journalctl@ceph.osd.2.smithi097.stdout:Mar 21 14:03:05 smithi097 podman[158598]: 2024-03-21 14:03:05.874277671 +0000 UTC m=+0.619373797 container died aaa26604f07a67d8b9205b064a0e68059529159449b066dfde39542d839b5778 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-2-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-21T14:03:06.414 INFO:journalctl@ceph.osd.2.smithi097.stdout:Mar 21 14:03:06 smithi097 podman[158644]: 2024-03-21 14:03:06.391683641 +0000 UTC m=+0.512248499 container remove aaa26604f07a67d8b9205b064a0e68059529159449b066dfde39542d839b5778 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-2-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2) 2024-03-21T14:03:06.419 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@osd.2.service' 2024-03-21T14:03:06.458 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T14:03:06.458 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-21T14:03:06.458 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-21T14:03:06.458 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@osd.3 2024-03-21T14:03:06.789 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 21 14:03:06 smithi097 systemd[1]: Stopping Ceph osd.3 for a5dd1f32-e785-11ee-95cd-87774f69a715... 2024-03-21T14:03:07.053 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 21 14:03:06 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-3[62397]: 2024-03-21T14:03:06.788+0000 7f7ded795700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T14:03:07.053 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 21 14:03:06 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-3[62397]: 2024-03-21T14:03:06.788+0000 7f7ded795700 -1 osd.3 803 *** Got signal Terminated *** 2024-03-21T14:03:07.053 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 21 14:03:06 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-3[62397]: 2024-03-21T14:03:06.788+0000 7f7ded795700 -1 osd.3 803 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T14:03:09.622 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 21 14:03:09 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-5[45082]: 2024-03-21T14:03:09.172+0000 7f9752c0b700 -1 osd.5 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:44.059320+0000 front 2024-03-21T14:02:44.059308+0000 (oldest deadline 2024-03-21T14:03:08.659602+0000) 2024-03-21T14:03:09.622 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:09 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:09.421+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:46.959880+0000 front 2024-03-21T14:02:46.959716+0000 (oldest deadline 2024-03-21T14:03:08.659523+0000) 2024-03-21T14:03:10.553 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 21 14:03:10 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-3[62397]: 2024-03-21T14:03:10.272+0000 7f7de5d81700 -1 osd.3 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:43.531153+0000 front 2024-03-21T14:02:43.531270+0000 (oldest deadline 2024-03-21T14:03:09.431257+0000) 2024-03-21T14:03:10.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:10 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:10.441+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:46.959880+0000 front 2024-03-21T14:02:46.959716+0000 (oldest deadline 2024-03-21T14:03:08.659523+0000) 2024-03-21T14:03:10.621 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 21 14:03:10 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-5[45082]: 2024-03-21T14:03:10.195+0000 7f9752c0b700 -1 osd.5 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:44.059320+0000 front 2024-03-21T14:02:44.059308+0000 (oldest deadline 2024-03-21T14:03:08.659602+0000) 2024-03-21T14:03:11.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:11 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:11.402+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:46.959880+0000 front 2024-03-21T14:02:46.959716+0000 (oldest deadline 2024-03-21T14:03:08.659523+0000) 2024-03-21T14:03:11.622 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 21 14:03:11 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-5[45082]: 2024-03-21T14:03:11.205+0000 7f9752c0b700 -1 osd.5 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:44.059320+0000 front 2024-03-21T14:02:44.059308+0000 (oldest deadline 2024-03-21T14:03:08.659602+0000) 2024-03-21T14:03:11.803 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 21 14:03:11 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-3[62397]: 2024-03-21T14:03:11.312+0000 7f7de5d81700 -1 osd.3 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:43.531153+0000 front 2024-03-21T14:02:43.531270+0000 (oldest deadline 2024-03-21T14:03:09.431257+0000) 2024-03-21T14:03:12.445 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 21 14:03:12 smithi097 podman[158727]: 2024-03-21 14:03:12.17374939 +0000 UTC m=+5.499906797 container died 7e5a685bb21ba46f182aad70e844e181739349ff28c4383c87f4740972e6f0a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-3, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-21T14:03:12.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:12 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:12.374+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:46.959880+0000 front 2024-03-21T14:02:46.959716+0000 (oldest deadline 2024-03-21T14:03:08.659523+0000) 2024-03-21T14:03:12.621 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 21 14:03:12 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-5[45082]: 2024-03-21T14:03:12.244+0000 7f9752c0b700 -1 osd.5 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:44.059320+0000 front 2024-03-21T14:02:44.059308+0000 (oldest deadline 2024-03-21T14:03:08.659602+0000) 2024-03-21T14:03:12.794 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 21 14:03:12 smithi097 podman[158727]: 2024-03-21 14:03:12.535789821 +0000 UTC m=+5.861947229 container cleanup 7e5a685bb21ba46f182aad70e844e181739349ff28c4383c87f4740972e6f0a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, io.buildah.version=1.33.5) 2024-03-21T14:03:12.795 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 21 14:03:12 smithi097 bash[158727]: ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-3 2024-03-21T14:03:13.053 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 21 14:03:12 smithi097 podman[158751]: 2024-03-21 14:03:12.795172345 +0000 UTC m=+0.620655866 container remove 7e5a685bb21ba46f182aad70e844e181739349ff28c4383c87f4740972e6f0a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-3, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-21T14:03:13.121 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:12 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:12.754+0000 7f88f7dd7700 -1 osd.6 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:47.905585+0000 front 2024-03-21T14:02:47.905677+0000 (oldest deadline 2024-03-21T14:03:12.605669+0000) 2024-03-21T14:03:13.473 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 21 14:03:13 smithi097 podman[158846]: 2024-03-21 14:03:13.053643275 +0000 UTC m=+0.017923378 image pull 19e12293512ff7978c8adb5f72a829060dd47f07bfd6903b6b3178d2ebc29e4e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93 2024-03-21T14:03:13.473 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 21 14:03:13 smithi097 podman[158846]: 2024-03-21 14:03:13.171010744 +0000 UTC m=+0.135290841 container create 4961a7b9998146dd85d06bb1e600c831e93ecc49ddab59f1a5393198d10ef39b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T14:03:13.473 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 21 14:03:13 smithi097 podman[158846]: 2024-03-21 14:03:13.390032035 +0000 UTC m=+0.354312142 container init 4961a7b9998146dd85d06bb1e600c831e93ecc49ddab59f1a5393198d10ef39b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-3-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-21T14:03:13.473 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 21 14:03:13 smithi097 podman[158846]: 2024-03-21 14:03:13.395472588 +0000 UTC m=+0.359752686 container start 4961a7b9998146dd85d06bb1e600c831e93ecc49ddab59f1a5393198d10ef39b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-21T14:03:13.474 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 21 14:03:13 smithi097 podman[158846]: 2024-03-21 14:03:13.473183702 +0000 UTC m=+0.437463822 container attach 4961a7b9998146dd85d06bb1e600c831e93ecc49ddab59f1a5393198d10ef39b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-21T14:03:13.621 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 21 14:03:13 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-5[45082]: 2024-03-21T14:03:13.281+0000 7f9752c0b700 -1 osd.5 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:44.059320+0000 front 2024-03-21T14:02:44.059308+0000 (oldest deadline 2024-03-21T14:03:08.659602+0000) 2024-03-21T14:03:13.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:13 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:13.351+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:46.959880+0000 front 2024-03-21T14:02:46.959716+0000 (oldest deadline 2024-03-21T14:03:08.659523+0000) 2024-03-21T14:03:13.803 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 21 14:03:13 smithi097 podman[158846]: 2024-03-21 14:03:13.683739273 +0000 UTC m=+0.648019374 container died 4961a7b9998146dd85d06bb1e600c831e93ecc49ddab59f1a5393198d10ef39b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-3-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-21T14:03:14.121 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:13 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:13.742+0000 7f88f7dd7700 -1 osd.6 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:47.905585+0000 front 2024-03-21T14:02:47.905677+0000 (oldest deadline 2024-03-21T14:03:12.605669+0000) 2024-03-21T14:03:14.271 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@osd.3.service' 2024-03-21T14:03:14.311 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T14:03:14.312 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-21T14:03:14.312 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-21T14:03:14.312 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@osd.4 2024-03-21T14:03:14.616 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 21 14:03:14 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-5[45082]: 2024-03-21T14:03:14.239+0000 7f9752c0b700 -1 osd.5 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:44.059320+0000 front 2024-03-21T14:02:44.059308+0000 (oldest deadline 2024-03-21T14:03:08.659602+0000) 2024-03-21T14:03:14.616 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:14 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:14.330+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:46.959880+0000 front 2024-03-21T14:02:46.959716+0000 (oldest deadline 2024-03-21T14:03:08.659523+0000) 2024-03-21T14:03:14.616 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 21 14:03:14 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-4[39152]: 2024-03-21T14:03:14.173+0000 7fa4b6832700 -1 osd.4 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:47.619979+0000 front 2024-03-21T14:02:47.620039+0000 (oldest deadline 2024-03-21T14:03:14.020168+0000) 2024-03-21T14:03:14.616 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 21 14:03:14 smithi144 systemd[1]: Stopping Ceph osd.4 for a5dd1f32-e785-11ee-95cd-87774f69a715... 2024-03-21T14:03:14.871 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:14 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:14.730+0000 7f88f7dd7700 -1 osd.6 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:47.905585+0000 front 2024-03-21T14:02:47.905677+0000 (oldest deadline 2024-03-21T14:03:12.605669+0000) 2024-03-21T14:03:14.871 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 21 14:03:14 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-4[39152]: 2024-03-21T14:03:14.615+0000 7fa4be246700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T14:03:14.871 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 21 14:03:14 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-4[39152]: 2024-03-21T14:03:14.615+0000 7fa4be246700 -1 osd.4 803 *** Got signal Terminated *** 2024-03-21T14:03:14.871 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 21 14:03:14 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-4[39152]: 2024-03-21T14:03:14.615+0000 7fa4be246700 -1 osd.4 803 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T14:03:15.621 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 21 14:03:15 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-4[39152]: 2024-03-21T14:03:15.165+0000 7fa4b6832700 -1 osd.4 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:47.619979+0000 front 2024-03-21T14:02:47.620039+0000 (oldest deadline 2024-03-21T14:03:14.020168+0000) 2024-03-21T14:03:15.621 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 21 14:03:15 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-5[45082]: 2024-03-21T14:03:15.249+0000 7f9752c0b700 -1 osd.5 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:44.059320+0000 front 2024-03-21T14:02:44.059308+0000 (oldest deadline 2024-03-21T14:03:08.659602+0000) 2024-03-21T14:03:15.622 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:15 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:15.289+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:46.959880+0000 front 2024-03-21T14:02:46.959716+0000 (oldest deadline 2024-03-21T14:03:08.659523+0000) 2024-03-21T14:03:16.121 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:15 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:15.713+0000 7f88f7dd7700 -1 osd.6 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:47.905585+0000 front 2024-03-21T14:02:47.905677+0000 (oldest deadline 2024-03-21T14:03:12.605669+0000) 2024-03-21T14:03:16.621 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 21 14:03:16 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-4[39152]: 2024-03-21T14:03:16.171+0000 7fa4b6832700 -1 osd.4 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:47.619979+0000 front 2024-03-21T14:02:47.620039+0000 (oldest deadline 2024-03-21T14:03:14.020168+0000) 2024-03-21T14:03:16.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:16 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:16.319+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:46.959880+0000 front 2024-03-21T14:02:46.959716+0000 (oldest deadline 2024-03-21T14:03:08.659523+0000) 2024-03-21T14:03:16.622 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 21 14:03:16 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-5[45082]: 2024-03-21T14:03:16.205+0000 7f9752c0b700 -1 osd.5 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:44.059320+0000 front 2024-03-21T14:02:44.059308+0000 (oldest deadline 2024-03-21T14:03:08.659602+0000) 2024-03-21T14:03:17.121 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:16 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:16.707+0000 7f88f7dd7700 -1 osd.6 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:47.905585+0000 front 2024-03-21T14:02:47.905677+0000 (oldest deadline 2024-03-21T14:03:12.605669+0000) 2024-03-21T14:03:17.600 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 21 14:03:17 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-4[39152]: 2024-03-21T14:03:17.205+0000 7fa4b6832700 -1 osd.4 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:47.619979+0000 front 2024-03-21T14:02:47.620039+0000 (oldest deadline 2024-03-21T14:03:14.020168+0000) 2024-03-21T14:03:17.600 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 21 14:03:17 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-5[45082]: 2024-03-21T14:03:17.169+0000 7f9752c0b700 -1 osd.5 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:44.059320+0000 front 2024-03-21T14:02:44.059308+0000 (oldest deadline 2024-03-21T14:03:08.659602+0000) 2024-03-21T14:03:17.601 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:17 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:17.276+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:46.959880+0000 front 2024-03-21T14:02:46.959716+0000 (oldest deadline 2024-03-21T14:03:08.659523+0000) 2024-03-21T14:03:17.601 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:17 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:17.276+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:52.160479+0000 front 2024-03-21T14:02:52.160586+0000 (oldest deadline 2024-03-21T14:03:16.860486+0000) 2024-03-21T14:03:17.871 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:17 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:17.754+0000 7f88f7dd7700 -1 osd.6 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:47.905585+0000 front 2024-03-21T14:02:47.905677+0000 (oldest deadline 2024-03-21T14:03:12.605669+0000) 2024-03-21T14:03:17.871 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 14:03:17 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[67795]: ts=2024-03-21T14:03:17.600Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph-exporter msg="Unable to refresh target groups" err="Get \"http://172.21.15.97:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.97:8765: connect: connection refused" 2024-03-21T14:03:17.872 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 14:03:17 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[67795]: ts=2024-03-21T14:03:17.603Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=node msg="Unable to refresh target groups" err="Get \"http://172.21.15.97:8765/sd/prometheus/sd-config?service=node-exporter\": dial tcp 172.21.15.97:8765: connect: connection refused" 2024-03-21T14:03:17.872 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 14:03:17 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[67795]: ts=2024-03-21T14:03:17.604Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=nvmeof msg="Unable to refresh target groups" err="Get \"http://172.21.15.97:8765/sd/prometheus/sd-config?service=nvmeof\": dial tcp 172.21.15.97:8765: connect: connection refused" 2024-03-21T14:03:17.872 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 14:03:17 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[67795]: ts=2024-03-21T14:03:17.604Z caller=refresh.go:99 level=error component="discovery manager notify" discovery=http config=config-0 msg="Unable to refresh target groups" err="Get \"http://172.21.15.97:8765/sd/prometheus/sd-config?service=alertmanager\": dial tcp 172.21.15.97:8765: connect: connection refused" 2024-03-21T14:03:17.872 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 14:03:17 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[67795]: ts=2024-03-21T14:03:17.604Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph msg="Unable to refresh target groups" err="Get \"http://172.21.15.97:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.97:8765: connect: connection refused" 2024-03-21T14:03:18.621 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 21 14:03:18 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-4[39152]: 2024-03-21T14:03:18.224+0000 7fa4b6832700 -1 osd.4 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:47.619979+0000 front 2024-03-21T14:02:47.620039+0000 (oldest deadline 2024-03-21T14:03:14.020168+0000) 2024-03-21T14:03:18.621 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 21 14:03:18 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-4[39152]: 2024-03-21T14:03:18.224+0000 7fa4b6832700 -1 osd.4 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:55.121344+0000 front 2024-03-21T14:02:55.121308+0000 (oldest deadline 2024-03-21T14:03:18.021095+0000) 2024-03-21T14:03:18.622 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:18 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:18.313+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:46.959880+0000 front 2024-03-21T14:02:46.959716+0000 (oldest deadline 2024-03-21T14:03:08.659523+0000) 2024-03-21T14:03:18.622 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:18 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:18.313+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:52.160479+0000 front 2024-03-21T14:02:52.160586+0000 (oldest deadline 2024-03-21T14:03:16.860486+0000) 2024-03-21T14:03:18.622 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 21 14:03:18 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-5[45082]: 2024-03-21T14:03:18.202+0000 7f9752c0b700 -1 osd.5 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:44.059320+0000 front 2024-03-21T14:02:44.059308+0000 (oldest deadline 2024-03-21T14:03:08.659602+0000) 2024-03-21T14:03:19.121 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:18 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:18.711+0000 7f88f7dd7700 -1 osd.6 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:47.905585+0000 front 2024-03-21T14:02:47.905677+0000 (oldest deadline 2024-03-21T14:03:12.605669+0000) 2024-03-21T14:03:19.121 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:18 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:18.711+0000 7f88f7dd7700 -1 osd.6 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:53.706532+0000 front 2024-03-21T14:02:53.706622+0000 (oldest deadline 2024-03-21T14:03:18.406602+0000) 2024-03-21T14:03:19.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:19 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:19.303+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:46.959880+0000 front 2024-03-21T14:02:46.959716+0000 (oldest deadline 2024-03-21T14:03:08.659523+0000) 2024-03-21T14:03:19.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:19 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:19.303+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:52.160479+0000 front 2024-03-21T14:02:52.160586+0000 (oldest deadline 2024-03-21T14:03:16.860486+0000) 2024-03-21T14:03:19.621 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 21 14:03:19 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-4[39152]: 2024-03-21T14:03:19.241+0000 7fa4b6832700 -1 osd.4 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:47.619979+0000 front 2024-03-21T14:02:47.620039+0000 (oldest deadline 2024-03-21T14:03:14.020168+0000) 2024-03-21T14:03:19.622 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 21 14:03:19 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-4[39152]: 2024-03-21T14:03:19.241+0000 7fa4b6832700 -1 osd.4 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:55.121344+0000 front 2024-03-21T14:02:55.121308+0000 (oldest deadline 2024-03-21T14:03:18.021095+0000) 2024-03-21T14:03:19.622 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 21 14:03:19 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-5[45082]: 2024-03-21T14:03:19.198+0000 7f9752c0b700 -1 osd.5 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:44.059320+0000 front 2024-03-21T14:02:44.059308+0000 (oldest deadline 2024-03-21T14:03:08.659602+0000) 2024-03-21T14:03:20.039 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:19 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:19.667+0000 7f88f7dd7700 -1 osd.6 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:47.905585+0000 front 2024-03-21T14:02:47.905677+0000 (oldest deadline 2024-03-21T14:03:12.605669+0000) 2024-03-21T14:03:20.039 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:19 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:19.667+0000 7f88f7dd7700 -1 osd.6 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:53.706532+0000 front 2024-03-21T14:02:53.706622+0000 (oldest deadline 2024-03-21T14:03:18.406602+0000) 2024-03-21T14:03:20.322 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 21 14:03:20 smithi144 podman[78692]: 2024-03-21 14:03:20.039798139 +0000 UTC m=+5.529606232 container died 1b36d276fe17cc2529fe0fd28026655fce155b0c8911c2606bfa2a974de7082a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-4, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, ceph=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-21T14:03:20.323 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 21 14:03:20 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-5[45082]: 2024-03-21T14:03:20.247+0000 7f9752c0b700 -1 osd.5 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:44.059320+0000 front 2024-03-21T14:02:44.059308+0000 (oldest deadline 2024-03-21T14:03:08.659602+0000) 2024-03-21T14:03:20.323 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:20 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:20.272+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:46.959880+0000 front 2024-03-21T14:02:46.959716+0000 (oldest deadline 2024-03-21T14:03:08.659523+0000) 2024-03-21T14:03:20.323 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:20 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:20.272+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:52.160479+0000 front 2024-03-21T14:02:52.160586+0000 (oldest deadline 2024-03-21T14:03:16.860486+0000) 2024-03-21T14:03:20.621 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 21 14:03:20 smithi144 podman[78692]: 2024-03-21 14:03:20.413286611 +0000 UTC m=+5.903094703 container cleanup 1b36d276fe17cc2529fe0fd28026655fce155b0c8911c2606bfa2a974de7082a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-4, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.vendor=CentOS, ceph=True) 2024-03-21T14:03:20.621 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 21 14:03:20 smithi144 bash[78692]: ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-4 2024-03-21T14:03:21.031 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 21 14:03:20 smithi144 podman[78714]: 2024-03-21 14:03:20.697805378 +0000 UTC m=+0.649412051 container remove 1b36d276fe17cc2529fe0fd28026655fce155b0c8911c2606bfa2a974de7082a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-4, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-21T14:03:21.031 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:20 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:20.662+0000 7f88f7dd7700 -1 osd.6 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:47.905585+0000 front 2024-03-21T14:02:47.905677+0000 (oldest deadline 2024-03-21T14:03:12.605669+0000) 2024-03-21T14:03:21.031 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:20 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:20.662+0000 7f88f7dd7700 -1 osd.6 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:53.706532+0000 front 2024-03-21T14:02:53.706622+0000 (oldest deadline 2024-03-21T14:03:18.406602+0000) 2024-03-21T14:03:21.283 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 21 14:03:21 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-5[45082]: 2024-03-21T14:03:21.209+0000 7f9752c0b700 -1 osd.5 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:44.059320+0000 front 2024-03-21T14:02:44.059308+0000 (oldest deadline 2024-03-21T14:03:08.659602+0000) 2024-03-21T14:03:21.283 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 21 14:03:21 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-5[45082]: 2024-03-21T14:03:21.209+0000 7f9752c0b700 -1 osd.5 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:54.461046+0000 front 2024-03-21T14:02:54.461003+0000 (oldest deadline 2024-03-21T14:03:20.261381+0000) 2024-03-21T14:03:21.283 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 21 14:03:21 smithi144 podman[78813]: 2024-03-21 14:03:21.031839564 +0000 UTC m=+0.114759919 container create e7187bd7f7be3698ed93d18d4f3c91c38d1c9b53ea580f7fbdd04bb9c9cbc228 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-4-deactivate, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T14:03:21.283 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 21 14:03:21 smithi144 podman[78813]: 2024-03-21 14:03:20.93540775 +0000 UTC m=+0.018328063 image pull 19e12293512ff7978c8adb5f72a829060dd47f07bfd6903b6b3178d2ebc29e4e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93 2024-03-21T14:03:21.283 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 21 14:03:21 smithi144 podman[78813]: 2024-03-21 14:03:21.276003729 +0000 UTC m=+0.358924049 container init e7187bd7f7be3698ed93d18d4f3c91c38d1c9b53ea580f7fbdd04bb9c9cbc228 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T14:03:21.283 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 21 14:03:21 smithi144 podman[78813]: 2024-03-21 14:03:21.281600982 +0000 UTC m=+0.364521295 container start e7187bd7f7be3698ed93d18d4f3c91c38d1c9b53ea580f7fbdd04bb9c9cbc228 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-4-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-21T14:03:21.574 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:21 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:21.281+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:46.959880+0000 front 2024-03-21T14:02:46.959716+0000 (oldest deadline 2024-03-21T14:03:08.659523+0000) 2024-03-21T14:03:21.574 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:21 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:21.281+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:52.160479+0000 front 2024-03-21T14:02:52.160586+0000 (oldest deadline 2024-03-21T14:03:16.860486+0000) 2024-03-21T14:03:21.575 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 21 14:03:21 smithi144 podman[78813]: 2024-03-21 14:03:21.384551702 +0000 UTC m=+0.467472033 container attach e7187bd7f7be3698ed93d18d4f3c91c38d1c9b53ea580f7fbdd04bb9c9cbc228 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2) 2024-03-21T14:03:21.871 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 21 14:03:21 smithi144 podman[78813]: 2024-03-21 14:03:21.574958485 +0000 UTC m=+0.657878802 container died e7187bd7f7be3698ed93d18d4f3c91c38d1c9b53ea580f7fbdd04bb9c9cbc228 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T14:03:21.872 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:21 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:21.672+0000 7f88f7dd7700 -1 osd.6 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:47.905585+0000 front 2024-03-21T14:02:47.905677+0000 (oldest deadline 2024-03-21T14:03:12.605669+0000) 2024-03-21T14:03:21.872 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:21 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:21.672+0000 7f88f7dd7700 -1 osd.6 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:53.706532+0000 front 2024-03-21T14:02:53.706622+0000 (oldest deadline 2024-03-21T14:03:18.406602+0000) 2024-03-21T14:03:22.174 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 21 14:03:22 smithi144 podman[78860]: 2024-03-21 14:03:22.171418918 +0000 UTC m=+0.591357319 container remove e7187bd7f7be3698ed93d18d4f3c91c38d1c9b53ea580f7fbdd04bb9c9cbc228 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-03-21T14:03:22.188 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@osd.4.service' 2024-03-21T14:03:22.230 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T14:03:22.231 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-21T14:03:22.231 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-21T14:03:22.231 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@osd.5 2024-03-21T14:03:22.552 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:22 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:22.262+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:46.959880+0000 front 2024-03-21T14:02:46.959716+0000 (oldest deadline 2024-03-21T14:03:08.659523+0000) 2024-03-21T14:03:22.552 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:22 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:22.262+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:52.160479+0000 front 2024-03-21T14:02:52.160586+0000 (oldest deadline 2024-03-21T14:03:16.860486+0000) 2024-03-21T14:03:22.552 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 21 14:03:22 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-5[45082]: 2024-03-21T14:03:22.228+0000 7f9752c0b700 -1 osd.5 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:44.059320+0000 front 2024-03-21T14:02:44.059308+0000 (oldest deadline 2024-03-21T14:03:08.659602+0000) 2024-03-21T14:03:22.552 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 21 14:03:22 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-5[45082]: 2024-03-21T14:03:22.228+0000 7f9752c0b700 -1 osd.5 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:54.461046+0000 front 2024-03-21T14:02:54.461003+0000 (oldest deadline 2024-03-21T14:03:20.261381+0000) 2024-03-21T14:03:22.552 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 21 14:03:22 smithi144 systemd[1]: Stopping Ceph osd.5 for a5dd1f32-e785-11ee-95cd-87774f69a715... 2024-03-21T14:03:22.552 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 21 14:03:22 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-5[45082]: 2024-03-21T14:03:22.550+0000 7f975a61f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T14:03:22.552 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 21 14:03:22 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-5[45082]: 2024-03-21T14:03:22.550+0000 7f975a61f700 -1 osd.5 803 *** Got signal Terminated *** 2024-03-21T14:03:22.553 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 21 14:03:22 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-5[45082]: 2024-03-21T14:03:22.550+0000 7f975a61f700 -1 osd.5 803 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T14:03:22.871 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:22 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:22.686+0000 7f88f7dd7700 -1 osd.6 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:47.905585+0000 front 2024-03-21T14:02:47.905677+0000 (oldest deadline 2024-03-21T14:03:12.605669+0000) 2024-03-21T14:03:22.871 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:22 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:22.686+0000 7f88f7dd7700 -1 osd.6 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:53.706532+0000 front 2024-03-21T14:02:53.706622+0000 (oldest deadline 2024-03-21T14:03:18.406602+0000) 2024-03-21T14:03:23.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:23 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:23.310+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:46.959880+0000 front 2024-03-21T14:02:46.959716+0000 (oldest deadline 2024-03-21T14:03:08.659523+0000) 2024-03-21T14:03:23.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:23 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:23.310+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:52.160479+0000 front 2024-03-21T14:02:52.160586+0000 (oldest deadline 2024-03-21T14:03:16.860486+0000) 2024-03-21T14:03:23.621 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 21 14:03:23 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-5[45082]: 2024-03-21T14:03:23.228+0000 7f9752c0b700 -1 osd.5 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:44.059320+0000 front 2024-03-21T14:02:44.059308+0000 (oldest deadline 2024-03-21T14:03:08.659602+0000) 2024-03-21T14:03:23.622 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 21 14:03:23 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-5[45082]: 2024-03-21T14:03:23.228+0000 7f9752c0b700 -1 osd.5 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:54.461046+0000 front 2024-03-21T14:02:54.461003+0000 (oldest deadline 2024-03-21T14:03:20.261381+0000) 2024-03-21T14:03:24.121 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:23 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:23.711+0000 7f88f7dd7700 -1 osd.6 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:47.905585+0000 front 2024-03-21T14:02:47.905677+0000 (oldest deadline 2024-03-21T14:03:12.605669+0000) 2024-03-21T14:03:24.121 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:23 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:23.711+0000 7f88f7dd7700 -1 osd.6 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:53.706532+0000 front 2024-03-21T14:02:53.706622+0000 (oldest deadline 2024-03-21T14:03:18.406602+0000) 2024-03-21T14:03:24.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:24 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:24.275+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:46.959880+0000 front 2024-03-21T14:02:46.959716+0000 (oldest deadline 2024-03-21T14:03:08.659523+0000) 2024-03-21T14:03:24.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:24 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:24.275+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:52.160479+0000 front 2024-03-21T14:02:52.160586+0000 (oldest deadline 2024-03-21T14:03:16.860486+0000) 2024-03-21T14:03:24.622 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 21 14:03:24 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-5[45082]: 2024-03-21T14:03:24.269+0000 7f9752c0b700 -1 osd.5 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:44.059320+0000 front 2024-03-21T14:02:44.059308+0000 (oldest deadline 2024-03-21T14:03:08.659602+0000) 2024-03-21T14:03:24.622 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 21 14:03:24 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-5[45082]: 2024-03-21T14:03:24.269+0000 7f9752c0b700 -1 osd.5 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:54.461046+0000 front 2024-03-21T14:02:54.461003+0000 (oldest deadline 2024-03-21T14:03:20.261381+0000) 2024-03-21T14:03:25.121 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:24 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:24.704+0000 7f88f7dd7700 -1 osd.6 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:47.905585+0000 front 2024-03-21T14:02:47.905677+0000 (oldest deadline 2024-03-21T14:03:12.605669+0000) 2024-03-21T14:03:25.121 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:24 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:24.704+0000 7f88f7dd7700 -1 osd.6 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:53.706532+0000 front 2024-03-21T14:02:53.706622+0000 (oldest deadline 2024-03-21T14:03:18.406602+0000) 2024-03-21T14:03:25.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:25 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:25.245+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:46.959880+0000 front 2024-03-21T14:02:46.959716+0000 (oldest deadline 2024-03-21T14:03:08.659523+0000) 2024-03-21T14:03:25.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:25 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:25.245+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:52.160479+0000 front 2024-03-21T14:02:52.160586+0000 (oldest deadline 2024-03-21T14:03:16.860486+0000) 2024-03-21T14:03:25.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:25 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:25.245+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6811 osd.2 since back 2024-03-21T14:03:00.361535+0000 front 2024-03-21T14:03:00.361519+0000 (oldest deadline 2024-03-21T14:03:24.461482+0000) 2024-03-21T14:03:25.622 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 21 14:03:25 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-5[45082]: 2024-03-21T14:03:25.248+0000 7f9752c0b700 -1 osd.5 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:44.059320+0000 front 2024-03-21T14:02:44.059308+0000 (oldest deadline 2024-03-21T14:03:08.659602+0000) 2024-03-21T14:03:25.622 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 21 14:03:25 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-5[45082]: 2024-03-21T14:03:25.248+0000 7f9752c0b700 -1 osd.5 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:54.461046+0000 front 2024-03-21T14:02:54.461003+0000 (oldest deadline 2024-03-21T14:03:20.261381+0000) 2024-03-21T14:03:26.121 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:25 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:25.752+0000 7f88f7dd7700 -1 osd.6 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:47.905585+0000 front 2024-03-21T14:02:47.905677+0000 (oldest deadline 2024-03-21T14:03:12.605669+0000) 2024-03-21T14:03:26.121 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:25 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:25.752+0000 7f88f7dd7700 -1 osd.6 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:53.706532+0000 front 2024-03-21T14:02:53.706622+0000 (oldest deadline 2024-03-21T14:03:18.406602+0000) 2024-03-21T14:03:26.121 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:25 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:25.752+0000 7f88f7dd7700 -1 osd.6 803 heartbeat_check: no reply from 172.21.15.97:6811 osd.2 since back 2024-03-21T14:03:00.707717+0000 front 2024-03-21T14:03:00.707759+0000 (oldest deadline 2024-03-21T14:03:25.407724+0000) 2024-03-21T14:03:26.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:26 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:26.214+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:46.959880+0000 front 2024-03-21T14:02:46.959716+0000 (oldest deadline 2024-03-21T14:03:08.659523+0000) 2024-03-21T14:03:26.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:26 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:26.214+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:52.160479+0000 front 2024-03-21T14:02:52.160586+0000 (oldest deadline 2024-03-21T14:03:16.860486+0000) 2024-03-21T14:03:26.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:26 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:26.214+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6811 osd.2 since back 2024-03-21T14:03:00.361535+0000 front 2024-03-21T14:03:00.361519+0000 (oldest deadline 2024-03-21T14:03:24.461482+0000) 2024-03-21T14:03:26.621 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 21 14:03:26 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-5[45082]: 2024-03-21T14:03:26.208+0000 7f9752c0b700 -1 osd.5 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:44.059320+0000 front 2024-03-21T14:02:44.059308+0000 (oldest deadline 2024-03-21T14:03:08.659602+0000) 2024-03-21T14:03:26.622 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 21 14:03:26 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-5[45082]: 2024-03-21T14:03:26.208+0000 7f9752c0b700 -1 osd.5 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:54.461046+0000 front 2024-03-21T14:02:54.461003+0000 (oldest deadline 2024-03-21T14:03:20.261381+0000) 2024-03-21T14:03:26.622 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 21 14:03:26 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-5[45082]: 2024-03-21T14:03:26.208+0000 7f9752c0b700 -1 osd.5 803 heartbeat_check: no reply from 172.21.15.97:6811 osd.2 since back 2024-03-21T14:03:00.261968+0000 front 2024-03-21T14:03:00.261948+0000 (oldest deadline 2024-03-21T14:03:25.561870+0000) 2024-03-21T14:03:27.121 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:26 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:26.727+0000 7f88f7dd7700 -1 osd.6 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:47.905585+0000 front 2024-03-21T14:02:47.905677+0000 (oldest deadline 2024-03-21T14:03:12.605669+0000) 2024-03-21T14:03:27.121 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:26 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:26.727+0000 7f88f7dd7700 -1 osd.6 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:53.706532+0000 front 2024-03-21T14:02:53.706622+0000 (oldest deadline 2024-03-21T14:03:18.406602+0000) 2024-03-21T14:03:27.121 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:26 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:26.727+0000 7f88f7dd7700 -1 osd.6 803 heartbeat_check: no reply from 172.21.15.97:6811 osd.2 since back 2024-03-21T14:03:00.707717+0000 front 2024-03-21T14:03:00.707759+0000 (oldest deadline 2024-03-21T14:03:25.407724+0000) 2024-03-21T14:03:27.621 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 21 14:03:27 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-5[45082]: 2024-03-21T14:03:27.240+0000 7f9752c0b700 -1 osd.5 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:44.059320+0000 front 2024-03-21T14:02:44.059308+0000 (oldest deadline 2024-03-21T14:03:08.659602+0000) 2024-03-21T14:03:27.621 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 21 14:03:27 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-5[45082]: 2024-03-21T14:03:27.240+0000 7f9752c0b700 -1 osd.5 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:54.461046+0000 front 2024-03-21T14:02:54.461003+0000 (oldest deadline 2024-03-21T14:03:20.261381+0000) 2024-03-21T14:03:27.621 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 21 14:03:27 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-5[45082]: 2024-03-21T14:03:27.240+0000 7f9752c0b700 -1 osd.5 803 heartbeat_check: no reply from 172.21.15.97:6811 osd.2 since back 2024-03-21T14:03:00.261968+0000 front 2024-03-21T14:03:00.261948+0000 (oldest deadline 2024-03-21T14:03:25.561870+0000) 2024-03-21T14:03:27.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:27 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:27.231+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:46.959880+0000 front 2024-03-21T14:02:46.959716+0000 (oldest deadline 2024-03-21T14:03:08.659523+0000) 2024-03-21T14:03:27.622 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:27 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:27.231+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:52.160479+0000 front 2024-03-21T14:02:52.160586+0000 (oldest deadline 2024-03-21T14:03:16.860486+0000) 2024-03-21T14:03:27.622 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:27 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:27.231+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6811 osd.2 since back 2024-03-21T14:03:00.361535+0000 front 2024-03-21T14:03:00.361519+0000 (oldest deadline 2024-03-21T14:03:24.461482+0000) 2024-03-21T14:03:28.121 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 21 14:03:27 smithi144 podman[78945]: 2024-03-21 14:03:27.965747955 +0000 UTC m=+5.505183268 container died a19a82f62ec8cb64509ccf69698589c1df131721b3e043081ea2cbad7ab75f9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T14:03:28.121 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:27 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:27.737+0000 7f88f7dd7700 -1 osd.6 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:47.905585+0000 front 2024-03-21T14:02:47.905677+0000 (oldest deadline 2024-03-21T14:03:12.605669+0000) 2024-03-21T14:03:28.121 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:27 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:27.737+0000 7f88f7dd7700 -1 osd.6 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:53.706532+0000 front 2024-03-21T14:02:53.706622+0000 (oldest deadline 2024-03-21T14:03:18.406602+0000) 2024-03-21T14:03:28.122 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:27 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:27.737+0000 7f88f7dd7700 -1 osd.6 803 heartbeat_check: no reply from 172.21.15.97:6811 osd.2 since back 2024-03-21T14:03:00.707717+0000 front 2024-03-21T14:03:00.707759+0000 (oldest deadline 2024-03-21T14:03:25.407724+0000) 2024-03-21T14:03:28.583 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 21 14:03:28 smithi144 podman[78945]: 2024-03-21 14:03:28.323870483 +0000 UTC m=+5.863305858 container cleanup a19a82f62ec8cb64509ccf69698589c1df131721b3e043081ea2cbad7ab75f9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-5, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240319) 2024-03-21T14:03:28.583 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 21 14:03:28 smithi144 bash[78945]: ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-5 2024-03-21T14:03:28.583 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:28 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:28.225+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:46.959880+0000 front 2024-03-21T14:02:46.959716+0000 (oldest deadline 2024-03-21T14:03:08.659523+0000) 2024-03-21T14:03:28.583 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:28 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:28.225+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:52.160479+0000 front 2024-03-21T14:02:52.160586+0000 (oldest deadline 2024-03-21T14:03:16.860486+0000) 2024-03-21T14:03:28.583 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:28 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:28.225+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6811 osd.2 since back 2024-03-21T14:03:00.361535+0000 front 2024-03-21T14:03:00.361519+0000 (oldest deadline 2024-03-21T14:03:24.461482+0000) 2024-03-21T14:03:28.871 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 21 14:03:28 smithi144 podman[78968]: 2024-03-21 14:03:28.583302526 +0000 UTC m=+0.614513987 container remove a19a82f62ec8cb64509ccf69698589c1df131721b3e043081ea2cbad7ab75f9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True) 2024-03-21T14:03:28.871 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:28 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:28.694+0000 7f88f7dd7700 -1 osd.6 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:47.905585+0000 front 2024-03-21T14:02:47.905677+0000 (oldest deadline 2024-03-21T14:03:12.605669+0000) 2024-03-21T14:03:28.871 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:28 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:28.694+0000 7f88f7dd7700 -1 osd.6 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:53.706532+0000 front 2024-03-21T14:02:53.706622+0000 (oldest deadline 2024-03-21T14:03:18.406602+0000) 2024-03-21T14:03:28.872 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:28 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:28.694+0000 7f88f7dd7700 -1 osd.6 803 heartbeat_check: no reply from 172.21.15.97:6811 osd.2 since back 2024-03-21T14:03:00.707717+0000 front 2024-03-21T14:03:00.707759+0000 (oldest deadline 2024-03-21T14:03:25.407724+0000) 2024-03-21T14:03:29.233 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 21 14:03:28 smithi144 podman[79067]: 2024-03-21 14:03:28.854010843 +0000 UTC m=+0.018652621 image pull 19e12293512ff7978c8adb5f72a829060dd47f07bfd6903b6b3178d2ebc29e4e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93 2024-03-21T14:03:29.234 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 21 14:03:28 smithi144 podman[79067]: 2024-03-21 14:03:28.976002265 +0000 UTC m=+0.140644035 container create 1d06c2346d733ef40c33d02db9b09b78cbaf5fe0f8a3a541c5f06cd4127c0906 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-5-deactivate, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T14:03:29.234 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 21 14:03:29 smithi144 podman[79067]: 2024-03-21 14:03:29.161525353 +0000 UTC m=+0.326167123 container init 1d06c2346d733ef40c33d02db9b09b78cbaf5fe0f8a3a541c5f06cd4127c0906 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T14:03:29.234 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 21 14:03:29 smithi144 podman[79067]: 2024-03-21 14:03:29.166976003 +0000 UTC m=+0.331617774 container start 1d06c2346d733ef40c33d02db9b09b78cbaf5fe0f8a3a541c5f06cd4127c0906 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-21T14:03:29.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:29 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:29.233+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:46.959880+0000 front 2024-03-21T14:02:46.959716+0000 (oldest deadline 2024-03-21T14:03:08.659523+0000) 2024-03-21T14:03:29.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:29 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:29.233+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:52.160479+0000 front 2024-03-21T14:02:52.160586+0000 (oldest deadline 2024-03-21T14:03:16.860486+0000) 2024-03-21T14:03:29.622 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:29 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:29.233+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6811 osd.2 since back 2024-03-21T14:03:00.361535+0000 front 2024-03-21T14:03:00.361519+0000 (oldest deadline 2024-03-21T14:03:24.461482+0000) 2024-03-21T14:03:29.622 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 21 14:03:29 smithi144 podman[79067]: 2024-03-21 14:03:29.236228443 +0000 UTC m=+0.400870220 container attach 1d06c2346d733ef40c33d02db9b09b78cbaf5fe0f8a3a541c5f06cd4127c0906 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-5-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T14:03:29.622 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 21 14:03:29 smithi144 podman[79067]: 2024-03-21 14:03:29.458550169 +0000 UTC m=+0.623191942 container died 1d06c2346d733ef40c33d02db9b09b78cbaf5fe0f8a3a541c5f06cd4127c0906 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-5-deactivate, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, io.buildah.version=1.33.5) 2024-03-21T14:03:29.989 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:29 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:29.705+0000 7f88f7dd7700 -1 osd.6 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:47.905585+0000 front 2024-03-21T14:02:47.905677+0000 (oldest deadline 2024-03-21T14:03:12.605669+0000) 2024-03-21T14:03:29.989 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:29 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:29.705+0000 7f88f7dd7700 -1 osd.6 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:53.706532+0000 front 2024-03-21T14:02:53.706622+0000 (oldest deadline 2024-03-21T14:03:18.406602+0000) 2024-03-21T14:03:29.990 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:29 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:29.705+0000 7f88f7dd7700 -1 osd.6 803 heartbeat_check: no reply from 172.21.15.97:6811 osd.2 since back 2024-03-21T14:03:00.707717+0000 front 2024-03-21T14:03:00.707759+0000 (oldest deadline 2024-03-21T14:03:25.407724+0000) 2024-03-21T14:03:30.009 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@osd.5.service' 2024-03-21T14:03:30.052 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T14:03:30.052 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-21T14:03:30.052 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-03-21T14:03:30.052 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@osd.6 2024-03-21T14:03:30.243 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:30 smithi144 systemd[1]: Stopping Ceph osd.6 for a5dd1f32-e785-11ee-95cd-87774f69a715... 2024-03-21T14:03:30.621 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:30 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:30.378+0000 7f88ff7eb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T14:03:30.621 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:30 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:30.378+0000 7f88ff7eb700 -1 osd.6 803 *** Got signal Terminated *** 2024-03-21T14:03:30.621 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:30 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:30.378+0000 7f88ff7eb700 -1 osd.6 803 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T14:03:30.622 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:30 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:30.242+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:46.959880+0000 front 2024-03-21T14:02:46.959716+0000 (oldest deadline 2024-03-21T14:03:08.659523+0000) 2024-03-21T14:03:30.622 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:30 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:30.242+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:52.160479+0000 front 2024-03-21T14:02:52.160586+0000 (oldest deadline 2024-03-21T14:03:16.860486+0000) 2024-03-21T14:03:30.622 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:30 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:30.242+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6811 osd.2 since back 2024-03-21T14:03:00.361535+0000 front 2024-03-21T14:03:00.361519+0000 (oldest deadline 2024-03-21T14:03:24.461482+0000) 2024-03-21T14:03:30.622 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:30 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:30.242+0000 7fc76ae92700 -1 osd.7 803 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14544.0:2193 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e803) 2024-03-21T14:03:31.121 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:30 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:30.715+0000 7f88f7dd7700 -1 osd.6 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:47.905585+0000 front 2024-03-21T14:02:47.905677+0000 (oldest deadline 2024-03-21T14:03:12.605669+0000) 2024-03-21T14:03:31.121 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:30 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:30.715+0000 7f88f7dd7700 -1 osd.6 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:53.706532+0000 front 2024-03-21T14:02:53.706622+0000 (oldest deadline 2024-03-21T14:03:18.406602+0000) 2024-03-21T14:03:31.121 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:30 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:30.715+0000 7f88f7dd7700 -1 osd.6 803 heartbeat_check: no reply from 172.21.15.97:6811 osd.2 since back 2024-03-21T14:03:00.707717+0000 front 2024-03-21T14:03:00.707759+0000 (oldest deadline 2024-03-21T14:03:25.407724+0000) 2024-03-21T14:03:31.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:31 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:31.222+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:46.959880+0000 front 2024-03-21T14:02:46.959716+0000 (oldest deadline 2024-03-21T14:03:08.659523+0000) 2024-03-21T14:03:31.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:31 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:31.222+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:52.160479+0000 front 2024-03-21T14:02:52.160586+0000 (oldest deadline 2024-03-21T14:03:16.860486+0000) 2024-03-21T14:03:31.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:31 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:31.222+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6811 osd.2 since back 2024-03-21T14:03:00.361535+0000 front 2024-03-21T14:03:00.361519+0000 (oldest deadline 2024-03-21T14:03:24.461482+0000) 2024-03-21T14:03:31.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:31 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:31.222+0000 7fc76ae92700 -1 osd.7 803 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14544.0:2193 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e803) 2024-03-21T14:03:32.121 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:31 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:31.736+0000 7f88f7dd7700 -1 osd.6 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:47.905585+0000 front 2024-03-21T14:02:47.905677+0000 (oldest deadline 2024-03-21T14:03:12.605669+0000) 2024-03-21T14:03:32.121 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:31 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:31.736+0000 7f88f7dd7700 -1 osd.6 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:53.706532+0000 front 2024-03-21T14:02:53.706622+0000 (oldest deadline 2024-03-21T14:03:18.406602+0000) 2024-03-21T14:03:32.121 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:31 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:31.736+0000 7f88f7dd7700 -1 osd.6 803 heartbeat_check: no reply from 172.21.15.97:6811 osd.2 since back 2024-03-21T14:03:00.707717+0000 front 2024-03-21T14:03:00.707759+0000 (oldest deadline 2024-03-21T14:03:25.407724+0000) 2024-03-21T14:03:32.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:32 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:32.226+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:46.959880+0000 front 2024-03-21T14:02:46.959716+0000 (oldest deadline 2024-03-21T14:03:08.659523+0000) 2024-03-21T14:03:32.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:32 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:32.226+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:52.160479+0000 front 2024-03-21T14:02:52.160586+0000 (oldest deadline 2024-03-21T14:03:16.860486+0000) 2024-03-21T14:03:32.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:32 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:32.226+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6811 osd.2 since back 2024-03-21T14:03:00.361535+0000 front 2024-03-21T14:03:00.361519+0000 (oldest deadline 2024-03-21T14:03:24.461482+0000) 2024-03-21T14:03:32.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:32 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:32.226+0000 7fc76ae92700 -1 osd.7 803 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14544.0:2193 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e803) 2024-03-21T14:03:33.121 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:32 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:32.744+0000 7f88f7dd7700 -1 osd.6 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:47.905585+0000 front 2024-03-21T14:02:47.905677+0000 (oldest deadline 2024-03-21T14:03:12.605669+0000) 2024-03-21T14:03:33.121 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:32 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:32.744+0000 7f88f7dd7700 -1 osd.6 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:53.706532+0000 front 2024-03-21T14:02:53.706622+0000 (oldest deadline 2024-03-21T14:03:18.406602+0000) 2024-03-21T14:03:33.121 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:32 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:32.744+0000 7f88f7dd7700 -1 osd.6 803 heartbeat_check: no reply from 172.21.15.97:6811 osd.2 since back 2024-03-21T14:03:00.707717+0000 front 2024-03-21T14:03:00.707759+0000 (oldest deadline 2024-03-21T14:03:25.407724+0000) 2024-03-21T14:03:33.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:33 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:33.206+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:46.959880+0000 front 2024-03-21T14:02:46.959716+0000 (oldest deadline 2024-03-21T14:03:08.659523+0000) 2024-03-21T14:03:33.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:33 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:33.206+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:52.160479+0000 front 2024-03-21T14:02:52.160586+0000 (oldest deadline 2024-03-21T14:03:16.860486+0000) 2024-03-21T14:03:33.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:33 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:33.206+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6811 osd.2 since back 2024-03-21T14:03:00.361535+0000 front 2024-03-21T14:03:00.361519+0000 (oldest deadline 2024-03-21T14:03:24.461482+0000) 2024-03-21T14:03:33.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:33 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:33.206+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6815 osd.3 since back 2024-03-21T14:03:10.362549+0000 front 2024-03-21T14:03:10.362535+0000 (oldest deadline 2024-03-21T14:03:32.662502+0000) 2024-03-21T14:03:33.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:33 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:33.206+0000 7fc76ae92700 -1 osd.7 803 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14544.0:2193 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e803) 2024-03-21T14:03:34.121 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:33 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:33.753+0000 7f88f7dd7700 -1 osd.6 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:47.905585+0000 front 2024-03-21T14:02:47.905677+0000 (oldest deadline 2024-03-21T14:03:12.605669+0000) 2024-03-21T14:03:34.121 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:33 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:33.753+0000 7f88f7dd7700 -1 osd.6 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:53.706532+0000 front 2024-03-21T14:02:53.706622+0000 (oldest deadline 2024-03-21T14:03:18.406602+0000) 2024-03-21T14:03:34.121 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:33 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:33.753+0000 7f88f7dd7700 -1 osd.6 803 heartbeat_check: no reply from 172.21.15.97:6811 osd.2 since back 2024-03-21T14:03:00.707717+0000 front 2024-03-21T14:03:00.707759+0000 (oldest deadline 2024-03-21T14:03:25.407724+0000) 2024-03-21T14:03:34.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:34 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:34.178+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:46.959880+0000 front 2024-03-21T14:02:46.959716+0000 (oldest deadline 2024-03-21T14:03:08.659523+0000) 2024-03-21T14:03:34.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:34 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:34.178+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:52.160479+0000 front 2024-03-21T14:02:52.160586+0000 (oldest deadline 2024-03-21T14:03:16.860486+0000) 2024-03-21T14:03:34.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:34 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:34.178+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6811 osd.2 since back 2024-03-21T14:03:00.361535+0000 front 2024-03-21T14:03:00.361519+0000 (oldest deadline 2024-03-21T14:03:24.461482+0000) 2024-03-21T14:03:34.622 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:34 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:34.178+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6815 osd.3 since back 2024-03-21T14:03:10.362549+0000 front 2024-03-21T14:03:10.362535+0000 (oldest deadline 2024-03-21T14:03:32.662502+0000) 2024-03-21T14:03:34.622 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:34 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:34.178+0000 7fc76ae92700 -1 osd.7 803 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14544.0:2193 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e803) 2024-03-21T14:03:35.121 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:34 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:34.720+0000 7f88f7dd7700 -1 osd.6 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:47.905585+0000 front 2024-03-21T14:02:47.905677+0000 (oldest deadline 2024-03-21T14:03:12.605669+0000) 2024-03-21T14:03:35.121 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:34 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:34.720+0000 7f88f7dd7700 -1 osd.6 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:53.706532+0000 front 2024-03-21T14:02:53.706622+0000 (oldest deadline 2024-03-21T14:03:18.406602+0000) 2024-03-21T14:03:35.121 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:34 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6[51033]: 2024-03-21T14:03:34.720+0000 7f88f7dd7700 -1 osd.6 803 heartbeat_check: no reply from 172.21.15.97:6811 osd.2 since back 2024-03-21T14:03:00.707717+0000 front 2024-03-21T14:03:00.707759+0000 (oldest deadline 2024-03-21T14:03:25.407724+0000) 2024-03-21T14:03:35.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:35 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:35.187+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:46.959880+0000 front 2024-03-21T14:02:46.959716+0000 (oldest deadline 2024-03-21T14:03:08.659523+0000) 2024-03-21T14:03:35.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:35 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:35.187+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:52.160479+0000 front 2024-03-21T14:02:52.160586+0000 (oldest deadline 2024-03-21T14:03:16.860486+0000) 2024-03-21T14:03:35.622 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:35 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:35.187+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6811 osd.2 since back 2024-03-21T14:03:00.361535+0000 front 2024-03-21T14:03:00.361519+0000 (oldest deadline 2024-03-21T14:03:24.461482+0000) 2024-03-21T14:03:35.622 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:35 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:35.187+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6815 osd.3 since back 2024-03-21T14:03:10.362549+0000 front 2024-03-21T14:03:10.362535+0000 (oldest deadline 2024-03-21T14:03:32.662502+0000) 2024-03-21T14:03:35.622 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:35 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:35.187+0000 7fc76ae92700 -1 osd.7 803 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14544.0:2193 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e803) 2024-03-21T14:03:36.121 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:35 smithi144 podman[79196]: 2024-03-21 14:03:35.845347617 +0000 UTC m=+5.567256405 container died 0877bd59a7d18a90459877698d6ea5b92089aebe1e2b0e0146c8848e086b94c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-21T14:03:36.476 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:36 smithi144 podman[79196]: 2024-03-21 14:03:36.217222228 +0000 UTC m=+5.939131021 container cleanup 0877bd59a7d18a90459877698d6ea5b92089aebe1e2b0e0146c8848e086b94c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T14:03:36.476 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:36 smithi144 bash[79196]: ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6 2024-03-21T14:03:36.476 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:36 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:36.197+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:46.959880+0000 front 2024-03-21T14:02:46.959716+0000 (oldest deadline 2024-03-21T14:03:08.659523+0000) 2024-03-21T14:03:36.476 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:36 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:36.197+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:52.160479+0000 front 2024-03-21T14:02:52.160586+0000 (oldest deadline 2024-03-21T14:03:16.860486+0000) 2024-03-21T14:03:36.476 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:36 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:36.197+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6811 osd.2 since back 2024-03-21T14:03:00.361535+0000 front 2024-03-21T14:03:00.361519+0000 (oldest deadline 2024-03-21T14:03:24.461482+0000) 2024-03-21T14:03:36.476 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:36 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:36.197+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6815 osd.3 since back 2024-03-21T14:03:10.362549+0000 front 2024-03-21T14:03:10.362535+0000 (oldest deadline 2024-03-21T14:03:32.662502+0000) 2024-03-21T14:03:36.476 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:36 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:36.197+0000 7fc76ae92700 -1 osd.7 803 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14544.0:2193 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e803) 2024-03-21T14:03:36.835 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:36 smithi144 podman[79218]: 2024-03-21 14:03:36.476466927 +0000 UTC m=+0.623922755 container remove 0877bd59a7d18a90459877698d6ea5b92089aebe1e2b0e0146c8848e086b94c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0) 2024-03-21T14:03:37.121 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:36 smithi144 podman[79317]: 2024-03-21 14:03:36.835805606 +0000 UTC m=+0.113198560 container create b5b87f6292d2fa9239a9594c3b39c65d530ae1ccbb7d7fb1b688d03ccc2fed15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6-deactivate, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD) 2024-03-21T14:03:37.121 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:36 smithi144 podman[79317]: 2024-03-21 14:03:36.740620317 +0000 UTC m=+0.018013248 image pull 19e12293512ff7978c8adb5f72a829060dd47f07bfd6903b6b3178d2ebc29e4e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93 2024-03-21T14:03:37.121 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:37 smithi144 podman[79317]: 2024-03-21 14:03:37.055052389 +0000 UTC m=+0.332445313 container init b5b87f6292d2fa9239a9594c3b39c65d530ae1ccbb7d7fb1b688d03ccc2fed15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T14:03:37.121 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:37 smithi144 podman[79317]: 2024-03-21 14:03:37.060869954 +0000 UTC m=+0.338262869 container start b5b87f6292d2fa9239a9594c3b39c65d530ae1ccbb7d7fb1b688d03ccc2fed15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-21T14:03:37.121 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:37 smithi144 podman[79317]: 2024-03-21 14:03:37.121053535 +0000 UTC m=+0.398446446 container attach b5b87f6292d2fa9239a9594c3b39c65d530ae1ccbb7d7fb1b688d03ccc2fed15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T14:03:37.371 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:37 smithi144 podman[79317]: 2024-03-21 14:03:37.350983696 +0000 UTC m=+0.628376597 container died b5b87f6292d2fa9239a9594c3b39c65d530ae1ccbb7d7fb1b688d03ccc2fed15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-03-21T14:03:37.371 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:37 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:37.200+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:46.959880+0000 front 2024-03-21T14:02:46.959716+0000 (oldest deadline 2024-03-21T14:03:08.659523+0000) 2024-03-21T14:03:37.371 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:37 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:37.200+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:52.160479+0000 front 2024-03-21T14:02:52.160586+0000 (oldest deadline 2024-03-21T14:03:16.860486+0000) 2024-03-21T14:03:37.371 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:37 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:37.200+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6811 osd.2 since back 2024-03-21T14:03:00.361535+0000 front 2024-03-21T14:03:00.361519+0000 (oldest deadline 2024-03-21T14:03:24.461482+0000) 2024-03-21T14:03:37.371 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:37 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:37.200+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6815 osd.3 since back 2024-03-21T14:03:10.362549+0000 front 2024-03-21T14:03:10.362535+0000 (oldest deadline 2024-03-21T14:03:32.662502+0000) 2024-03-21T14:03:37.372 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:37 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:37.200+0000 7fc76ae92700 -1 osd.7 803 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14544.0:2193 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e803) 2024-03-21T14:03:37.910 INFO:journalctl@ceph.osd.6.smithi144.stdout:Mar 21 14:03:37 smithi144 podman[79363]: 2024-03-21 14:03:37.891113497 +0000 UTC m=+0.534891995 container remove b5b87f6292d2fa9239a9594c3b39c65d530ae1ccbb7d7fb1b688d03ccc2fed15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-6-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-21T14:03:37.916 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@osd.6.service' 2024-03-21T14:03:37.957 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T14:03:37.957 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-03-21T14:03:37.957 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-03-21T14:03:37.957 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@osd.7 2024-03-21T14:03:38.170 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:38 smithi144 systemd[1]: Stopping Ceph osd.7 for a5dd1f32-e785-11ee-95cd-87774f69a715... 2024-03-21T14:03:38.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:38 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:38.170+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:46.959880+0000 front 2024-03-21T14:02:46.959716+0000 (oldest deadline 2024-03-21T14:03:08.659523+0000) 2024-03-21T14:03:38.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:38 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:38.170+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:52.160479+0000 front 2024-03-21T14:02:52.160586+0000 (oldest deadline 2024-03-21T14:03:16.860486+0000) 2024-03-21T14:03:38.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:38 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:38.170+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6811 osd.2 since back 2024-03-21T14:03:00.361535+0000 front 2024-03-21T14:03:00.361519+0000 (oldest deadline 2024-03-21T14:03:24.461482+0000) 2024-03-21T14:03:38.622 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:38 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:38.170+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6815 osd.3 since back 2024-03-21T14:03:10.362549+0000 front 2024-03-21T14:03:10.362535+0000 (oldest deadline 2024-03-21T14:03:32.662502+0000) 2024-03-21T14:03:38.622 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:38 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:38.170+0000 7fc76ae92700 -1 osd.7 803 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14544.0:2193 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e803) 2024-03-21T14:03:38.622 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:38 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:38.279+0000 7fc7728a6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T14:03:38.622 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:38 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:38.279+0000 7fc7728a6700 -1 osd.7 803 *** Got signal Terminated *** 2024-03-21T14:03:38.622 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:38 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:38.279+0000 7fc7728a6700 -1 osd.7 803 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T14:03:39.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:39 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:39.206+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:46.959880+0000 front 2024-03-21T14:02:46.959716+0000 (oldest deadline 2024-03-21T14:03:08.659523+0000) 2024-03-21T14:03:39.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:39 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:39.206+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:52.160479+0000 front 2024-03-21T14:02:52.160586+0000 (oldest deadline 2024-03-21T14:03:16.860486+0000) 2024-03-21T14:03:39.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:39 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:39.206+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6811 osd.2 since back 2024-03-21T14:03:00.361535+0000 front 2024-03-21T14:03:00.361519+0000 (oldest deadline 2024-03-21T14:03:24.461482+0000) 2024-03-21T14:03:39.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:39 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:39.206+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6815 osd.3 since back 2024-03-21T14:03:10.362549+0000 front 2024-03-21T14:03:10.362535+0000 (oldest deadline 2024-03-21T14:03:32.662502+0000) 2024-03-21T14:03:39.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:39 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:39.206+0000 7fc76ae92700 -1 osd.7 803 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14544.0:2193 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e803) 2024-03-21T14:03:40.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:40 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:40.243+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:46.959880+0000 front 2024-03-21T14:02:46.959716+0000 (oldest deadline 2024-03-21T14:03:08.659523+0000) 2024-03-21T14:03:40.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:40 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:40.243+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:52.160479+0000 front 2024-03-21T14:02:52.160586+0000 (oldest deadline 2024-03-21T14:03:16.860486+0000) 2024-03-21T14:03:40.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:40 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:40.243+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6811 osd.2 since back 2024-03-21T14:03:00.361535+0000 front 2024-03-21T14:03:00.361519+0000 (oldest deadline 2024-03-21T14:03:24.461482+0000) 2024-03-21T14:03:40.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:40 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:40.243+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6815 osd.3 since back 2024-03-21T14:03:10.362549+0000 front 2024-03-21T14:03:10.362535+0000 (oldest deadline 2024-03-21T14:03:32.662502+0000) 2024-03-21T14:03:40.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:40 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:40.243+0000 7fc76ae92700 -1 osd.7 803 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14544.0:2193 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e803) 2024-03-21T14:03:41.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:41 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:41.233+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:46.959880+0000 front 2024-03-21T14:02:46.959716+0000 (oldest deadline 2024-03-21T14:03:08.659523+0000) 2024-03-21T14:03:41.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:41 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:41.233+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:52.160479+0000 front 2024-03-21T14:02:52.160586+0000 (oldest deadline 2024-03-21T14:03:16.860486+0000) 2024-03-21T14:03:41.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:41 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:41.233+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6811 osd.2 since back 2024-03-21T14:03:00.361535+0000 front 2024-03-21T14:03:00.361519+0000 (oldest deadline 2024-03-21T14:03:24.461482+0000) 2024-03-21T14:03:41.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:41 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:41.233+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6815 osd.3 since back 2024-03-21T14:03:10.362549+0000 front 2024-03-21T14:03:10.362535+0000 (oldest deadline 2024-03-21T14:03:32.662502+0000) 2024-03-21T14:03:41.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:41 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:41.233+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.144:6802 osd.4 since back 2024-03-21T14:03:16.063917+0000 front 2024-03-21T14:03:16.063953+0000 (oldest deadline 2024-03-21T14:03:40.763891+0000) 2024-03-21T14:03:41.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:41 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:41.233+0000 7fc76ae92700 -1 osd.7 803 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14544.0:2193 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e803) 2024-03-21T14:03:42.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:42 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:42.186+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:46.959880+0000 front 2024-03-21T14:02:46.959716+0000 (oldest deadline 2024-03-21T14:03:08.659523+0000) 2024-03-21T14:03:42.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:42 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:42.186+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:52.160479+0000 front 2024-03-21T14:02:52.160586+0000 (oldest deadline 2024-03-21T14:03:16.860486+0000) 2024-03-21T14:03:42.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:42 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:42.186+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6811 osd.2 since back 2024-03-21T14:03:00.361535+0000 front 2024-03-21T14:03:00.361519+0000 (oldest deadline 2024-03-21T14:03:24.461482+0000) 2024-03-21T14:03:42.622 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:42 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:42.186+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6815 osd.3 since back 2024-03-21T14:03:10.362549+0000 front 2024-03-21T14:03:10.362535+0000 (oldest deadline 2024-03-21T14:03:32.662502+0000) 2024-03-21T14:03:42.622 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:42 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:42.186+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.144:6802 osd.4 since back 2024-03-21T14:03:16.063917+0000 front 2024-03-21T14:03:16.063953+0000 (oldest deadline 2024-03-21T14:03:40.763891+0000) 2024-03-21T14:03:42.622 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:42 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:42.186+0000 7fc76ae92700 -1 osd.7 803 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14544.0:2193 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e803) 2024-03-21T14:03:43.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:43 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:43.211+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6803 osd.0 since back 2024-03-21T14:02:46.959880+0000 front 2024-03-21T14:02:46.959716+0000 (oldest deadline 2024-03-21T14:03:08.659523+0000) 2024-03-21T14:03:43.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:43 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:43.211+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6807 osd.1 since back 2024-03-21T14:02:52.160479+0000 front 2024-03-21T14:02:52.160586+0000 (oldest deadline 2024-03-21T14:03:16.860486+0000) 2024-03-21T14:03:43.622 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:43 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:43.211+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6811 osd.2 since back 2024-03-21T14:03:00.361535+0000 front 2024-03-21T14:03:00.361519+0000 (oldest deadline 2024-03-21T14:03:24.461482+0000) 2024-03-21T14:03:43.622 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:43 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:43.211+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.97:6815 osd.3 since back 2024-03-21T14:03:10.362549+0000 front 2024-03-21T14:03:10.362535+0000 (oldest deadline 2024-03-21T14:03:32.662502+0000) 2024-03-21T14:03:43.622 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:43 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:43.211+0000 7fc76ae92700 -1 osd.7 803 heartbeat_check: no reply from 172.21.15.144:6802 osd.4 since back 2024-03-21T14:03:16.063917+0000 front 2024-03-21T14:03:16.063953+0000 (oldest deadline 2024-03-21T14:03:40.763891+0000) 2024-03-21T14:03:43.622 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:43 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7[57296]: 2024-03-21T14:03:43.211+0000 7fc76ae92700 -1 osd.7 803 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14544.0:2193 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e803) 2024-03-21T14:03:44.044 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:43 smithi144 podman[79451]: 2024-03-21 14:03:43.723875026 +0000 UTC m=+5.547851812 container died 9b34bb9c0c6391fb34462e935acebc28a79916a94d023f3523f6266c6be4cf87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-21T14:03:44.327 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:44 smithi144 podman[79451]: 2024-03-21 14:03:44.14323657 +0000 UTC m=+5.967213355 container cleanup 9b34bb9c0c6391fb34462e935acebc28a79916a94d023f3523f6266c6be4cf87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5) 2024-03-21T14:03:44.327 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:44 smithi144 bash[79451]: ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7 2024-03-21T14:03:44.621 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:44 smithi144 podman[79474]: 2024-03-21 14:03:44.327519799 +0000 UTC m=+0.602423271 container remove 9b34bb9c0c6391fb34462e935acebc28a79916a94d023f3523f6266c6be4cf87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe) 2024-03-21T14:03:45.055 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:44 smithi144 podman[79571]: 2024-03-21 14:03:44.761624151 +0000 UTC m=+0.114641477 container create 8efd23bea397980b9443193f9c6929f2bcd19729777e63271ae330d8bffdb9ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T14:03:45.055 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:44 smithi144 podman[79571]: 2024-03-21 14:03:44.665381402 +0000 UTC m=+0.018398758 image pull 19e12293512ff7978c8adb5f72a829060dd47f07bfd6903b6b3178d2ebc29e4e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93 2024-03-21T14:03:45.055 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:44 smithi144 podman[79571]: 2024-03-21 14:03:44.972527778 +0000 UTC m=+0.325545122 container init 8efd23bea397980b9443193f9c6929f2bcd19729777e63271ae330d8bffdb9ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7-deactivate, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=) 2024-03-21T14:03:45.055 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:44 smithi144 podman[79571]: 2024-03-21 14:03:44.978390329 +0000 UTC m=+0.331407662 container start 8efd23bea397980b9443193f9c6929f2bcd19729777e63271ae330d8bffdb9ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T14:03:45.055 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:45 smithi144 podman[79571]: 2024-03-21 14:03:45.055704743 +0000 UTC m=+0.408722091 container attach 8efd23bea397980b9443193f9c6929f2bcd19729777e63271ae330d8bffdb9ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, io.buildah.version=1.33.5, org.label-schema.build-date=20240319) 2024-03-21T14:03:45.371 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 21 14:03:45 smithi144 podman[79571]: 2024-03-21 14:03:45.264914154 +0000 UTC m=+0.617931481 container died 8efd23bea397980b9443193f9c6929f2bcd19729777e63271ae330d8bffdb9ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-osd-7-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, io.buildah.version=1.33.5) 2024-03-21T14:03:45.770 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@osd.7.service' 2024-03-21T14:03:45.812 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T14:03:45.812 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-03-21T14:03:45.812 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-03-21T14:03:45.812 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@rgw.foo.a 2024-03-21T14:03:46.179 INFO:journalctl@ceph.rgw.foo.a.smithi097.stdout:Mar 21 14:03:45 smithi097 systemd[1]: Stopping Ceph rgw.foo.a for a5dd1f32-e785-11ee-95cd-87774f69a715... 2024-03-21T14:03:46.553 INFO:journalctl@ceph.rgw.foo.a.smithi097.stdout:Mar 21 14:03:46 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-rgw-foo-a[67804]: 2024-03-21T14:03:46.178+0000 7f3c8beca700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T14:03:46.553 INFO:journalctl@ceph.rgw.foo.a.smithi097.stdout:Mar 21 14:03:46 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-rgw-foo-a[67804]: 2024-03-21T14:03:46.178+0000 7f3c9a74fac0 -1 shutting down 2024-03-21T14:03:56.485 INFO:journalctl@ceph.rgw.foo.a.smithi097.stdout:Mar 21 14:03:56 smithi097 bash[158988]: time="2024-03-21T14:03:56Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-03-21T14:03:56.485 INFO:journalctl@ceph.rgw.foo.a.smithi097.stdout:Mar 21 14:03:56 smithi097 podman[158988]: 2024-03-21 14:03:56.200565637 +0000 UTC m=+10.189807173 container died 71d31bf20f2cc96ab4f9586c9dfe1dfff3e0815f758cc381757454753be55fdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-rgw-foo-a, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-21T14:03:56.803 INFO:journalctl@ceph.rgw.foo.a.smithi097.stdout:Mar 21 14:03:56 smithi097 podman[158988]: 2024-03-21 14:03:56.575200521 +0000 UTC m=+10.564442104 container cleanup 71d31bf20f2cc96ab4f9586c9dfe1dfff3e0815f758cc381757454753be55fdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-rgw-foo-a, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe) 2024-03-21T14:03:56.803 INFO:journalctl@ceph.rgw.foo.a.smithi097.stdout:Mar 21 14:03:56 smithi097 bash[158988]: ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-rgw-foo-a 2024-03-21T14:03:56.989 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@rgw.foo.a.service' 2024-03-21T14:03:57.027 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T14:03:57.027 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-03-21T14:03:57.027 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-03-21T14:03:57.027 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@prometheus.a 2024-03-21T14:03:57.312 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 14:03:57 smithi144 systemd[1]: Stopping Ceph prometheus.a for a5dd1f32-e785-11ee-95cd-87774f69a715... 2024-03-21T14:03:57.574 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 14:03:57 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[67795]: ts=2024-03-21T14:03:57.312Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-03-21T14:03:57.575 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 14:03:57 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[67795]: ts=2024-03-21T14:03:57.312Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-03-21T14:03:57.575 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 14:03:57 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[67795]: ts=2024-03-21T14:03:57.312Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-03-21T14:03:57.575 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 14:03:57 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[67795]: ts=2024-03-21T14:03:57.312Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-03-21T14:03:57.575 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 14:03:57 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[67795]: ts=2024-03-21T14:03:57.312Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-03-21T14:03:57.575 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 14:03:57 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[67795]: ts=2024-03-21T14:03:57.312Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-03-21T14:03:57.575 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 14:03:57 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[67795]: ts=2024-03-21T14:03:57.312Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-03-21T14:03:57.575 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 14:03:57 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[67795]: ts=2024-03-21T14:03:57.312Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-03-21T14:03:57.575 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 14:03:57 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[67795]: ts=2024-03-21T14:03:57.312Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-03-21T14:03:57.575 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 14:03:57 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[67795]: ts=2024-03-21T14:03:57.313Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-03-21T14:03:57.575 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 14:03:57 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[67795]: ts=2024-03-21T14:03:57.313Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-03-21T14:03:57.575 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 14:03:57 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a[67795]: ts=2024-03-21T14:03:57.313Z caller=main.go:1147 level=info msg="See you next time!" 2024-03-21T14:03:57.575 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 14:03:57 smithi144 podman[79701]: 2024-03-21 14:03:57.323138035 +0000 UTC m=+0.103402586 container died 1a9823cc5c75409c5d0b588f283632a600da23b051281af5aba6ca8c5998dd84 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-21T14:03:57.871 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 14:03:57 smithi144 podman[79701]: 2024-03-21 14:03:57.593736417 +0000 UTC m=+0.374000964 container cleanup 1a9823cc5c75409c5d0b588f283632a600da23b051281af5aba6ca8c5998dd84 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-21T14:03:57.871 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 14:03:57 smithi144 bash[79701]: ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a 2024-03-21T14:03:57.871 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Mar 21 14:03:57 smithi144 podman[79721]: 2024-03-21 14:03:57.79409061 +0000 UTC m=+0.471890628 container remove 1a9823cc5c75409c5d0b588f283632a600da23b051281af5aba6ca8c5998dd84 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-21T14:03:57.933 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@prometheus.a.service' 2024-03-21T14:03:57.971 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T14:03:57.971 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-03-21T14:03:57.972 DEBUG:teuthology.orchestra.run.smithi097:> sudo cephadm rm-cluster --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-21T14:03:58.151 INFO:teuthology.orchestra.run.smithi097.stdout:Deleting cluster with fsid: a5dd1f32-e785-11ee-95cd-87774f69a715 2024-03-21T14:03:59.983 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 14:03:59 smithi097 systemd[1]: Stopping Ceph alertmanager.a for a5dd1f32-e785-11ee-95cd-87774f69a715... 2024-03-21T14:04:00.298 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 14:03:59 smithi097 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-alertmanager-a[78832]: ts=2024-03-21T14:03:59.983Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-03-21T14:04:00.298 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 14:04:00 smithi097 podman[159512]: 2024-03-21 14:04:00.034498124 +0000 UTC m=+0.168599153 container died e5a584b4baa96dfbaa4832d154f4578ecb535cf27e1a397735fef7cdfb2e573a (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-21T14:04:00.553 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 14:04:00 smithi097 podman[159512]: 2024-03-21 14:04:00.396811195 +0000 UTC m=+0.530912222 container cleanup e5a584b4baa96dfbaa4832d154f4578ecb535cf27e1a397735fef7cdfb2e573a (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-21T14:04:00.553 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 14:04:00 smithi097 bash[159512]: ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-alertmanager-a 2024-03-21T14:04:01.007 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 14:04:00 smithi097 podman[159531]: 2024-03-21 14:04:00.647858241 +0000 UTC m=+0.609898274 container remove e5a584b4baa96dfbaa4832d154f4578ecb535cf27e1a397735fef7cdfb2e573a (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-21T14:04:01.007 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 14:04:00 smithi097 podman[159531]: 2024-03-21 14:04:00.723467385 +0000 UTC m=+0.685507427 volume remove c9e3d9c764aa2dcec73073d6fb6b5396b6c8829a571d2af65b31b2234cf35685 2024-03-21T14:04:01.008 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 14:04:00 smithi097 systemd[1]: ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@alertmanager.a.service: Deactivated successfully. 2024-03-21T14:04:01.008 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 14:04:00 smithi097 systemd[1]: Stopped Ceph alertmanager.a for a5dd1f32-e785-11ee-95cd-87774f69a715. 2024-03-21T14:04:01.008 INFO:journalctl@ceph.alertmanager.a.smithi097.stdout:Mar 21 14:04:00 smithi097 systemd[1]: ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@alertmanager.a.service: Consumed 2.487s CPU time. 2024-03-21T14:04:01.264 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 14:04:01 smithi097 systemd[1]: Stopping Ceph node-exporter.a for a5dd1f32-e785-11ee-95cd-87774f69a715... 2024-03-21T14:04:01.553 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 14:04:01 smithi097 podman[159649]: 2024-03-21 14:04:01.26448759 +0000 UTC m=+0.120000354 container died d296a13aefa1785643bc968f14c17d81489509ee19be68945462d80fe93ba404 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-03-21T14:04:01.963 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 14:04:01 smithi097 podman[159649]: 2024-03-21 14:04:01.609230906 +0000 UTC m=+0.464743671 container cleanup d296a13aefa1785643bc968f14c17d81489509ee19be68945462d80fe93ba404 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-03-21T14:04:01.963 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 14:04:01 smithi097 bash[159649]: ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a 2024-03-21T14:04:01.963 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 14:04:01 smithi097 podman[159668]: 2024-03-21 14:04:01.76840784 +0000 UTC m=+0.508796578 container remove d296a13aefa1785643bc968f14c17d81489509ee19be68945462d80fe93ba404 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-03-21T14:04:01.963 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 14:04:01 smithi097 systemd[1]: ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-03-21T14:04:02.303 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 14:04:01 smithi097 systemd[1]: ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@node-exporter.a.service: Failed with result 'exit-code'. 2024-03-21T14:04:02.303 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 14:04:01 smithi097 systemd[1]: Stopped Ceph node-exporter.a for a5dd1f32-e785-11ee-95cd-87774f69a715. 2024-03-21T14:04:02.303 INFO:journalctl@ceph.node-exporter.a.smithi097.stdout:Mar 21 14:04:01 smithi097 systemd[1]: ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@node-exporter.a.service: Consumed 5.254s CPU time. 2024-03-21T14:04:02.596 DEBUG:teuthology.orchestra.run.smithi144:> sudo cephadm rm-cluster --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-21T14:04:02.787 INFO:teuthology.orchestra.run.smithi144.stdout:Deleting cluster with fsid: a5dd1f32-e785-11ee-95cd-87774f69a715 2024-03-21T14:04:04.121 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 14:04:04 smithi144 systemd[1]: Stopping Ceph iscsi.iscsi.a for a5dd1f32-e785-11ee-95cd-87774f69a715... 2024-03-21T14:04:04.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 14:04:04 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a[62199]: debug Shutdown received 2024-03-21T14:04:14.580 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 14:04:14 smithi144 bash[80134]: time="2024-03-21T14:04:14Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a in 10 seconds, resorting to SIGKILL" 2024-03-21T14:04:14.581 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 14:04:14 smithi144 podman[80134]: 2024-03-21 14:04:14.29387147 +0000 UTC m=+10.125585179 container died 260ec876767b01441664452e8f814301a4c0a529cd08ce2ff765a2760a4baa43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-03-21T14:04:14.871 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 14:04:14 smithi144 podman[80134]: 2024-03-21 14:04:14.670633443 +0000 UTC m=+10.502347151 container cleanup 260ec876767b01441664452e8f814301a4c0a529cd08ce2ff765a2760a4baa43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-21T14:04:14.871 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 14:04:14 smithi144 bash[80134]: ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a 2024-03-21T14:04:15.242 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 14:04:14 smithi144 podman[80159]: 2024-03-21 14:04:14.921800006 +0000 UTC m=+0.629003161 container remove 260ec876767b01441664452e8f814301a4c0a529cd08ce2ff765a2760a4baa43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-iscsi-iscsi-a, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T14:04:15.242 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 14:04:14 smithi144 systemd[1]: ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-03-21T14:04:15.243 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 14:04:15 smithi144 systemd[1]: ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-03-21T14:04:15.243 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 14:04:15 smithi144 systemd[1]: Stopped Ceph iscsi.iscsi.a for a5dd1f32-e785-11ee-95cd-87774f69a715. 2024-03-21T14:04:15.243 INFO:journalctl@ceph.iscsi.iscsi.a.smithi144.stdout:Mar 21 14:04:15 smithi144 systemd[1]: ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@iscsi.iscsi.a.service: Consumed 4.449s CPU time. 2024-03-21T14:04:16.621 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 14:04:16 smithi144 systemd[1]: Stopping Ceph grafana.a for a5dd1f32-e785-11ee-95cd-87774f69a715... 2024-03-21T14:04:16.938 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 14:04:16 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=server t=2024-03-21T14:04:16.68106909Z level=info msg="Shutdown started" reason="System signal: terminated" 2024-03-21T14:04:16.938 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 14:04:16 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=ticker t=2024-03-21T14:04:16.681494581Z level=info msg=stopped last_tick=2024-03-21T14:04:10Z 2024-03-21T14:04:16.938 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 14:04:16 smithi144 ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a[66277]: logger=tracing t=2024-03-21T14:04:16.68150698Z level=info msg="Closing tracing" 2024-03-21T14:04:16.938 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 14:04:16 smithi144 podman[80434]: 2024-03-21 14:04:16.691656023 +0000 UTC m=+0.116437216 container died a01d0c4fc05f5eb5e1a0aaee458e5ec42cbb796716f0f95f7d9d06977dc543cb (image=quay.io/ceph/grafana:9.4.12, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a, maintainer=Grafana Labs ) 2024-03-21T14:04:17.203 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 14:04:17 smithi144 podman[80434]: 2024-03-21 14:04:17.019655328 +0000 UTC m=+0.444436565 container cleanup a01d0c4fc05f5eb5e1a0aaee458e5ec42cbb796716f0f95f7d9d06977dc543cb (image=quay.io/ceph/grafana:9.4.12, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a, maintainer=Grafana Labs ) 2024-03-21T14:04:17.203 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 14:04:17 smithi144 bash[80434]: ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a 2024-03-21T14:04:17.621 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 14:04:17 smithi144 podman[80453]: 2024-03-21 14:04:17.203855618 +0000 UTC m=+0.511725281 container remove a01d0c4fc05f5eb5e1a0aaee458e5ec42cbb796716f0f95f7d9d06977dc543cb (image=quay.io/ceph/grafana:9.4.12, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-grafana-a, maintainer=Grafana Labs ) 2024-03-21T14:04:17.621 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 14:04:17 smithi144 systemd[1]: ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@grafana.a.service: Deactivated successfully. 2024-03-21T14:04:17.621 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 14:04:17 smithi144 systemd[1]: Stopped Ceph grafana.a for a5dd1f32-e785-11ee-95cd-87774f69a715. 2024-03-21T14:04:17.621 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Mar 21 14:04:17 smithi144 systemd[1]: ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@grafana.a.service: Consumed 24.249s CPU time. 2024-03-21T14:04:17.887 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 14:04:17 smithi144 systemd[1]: Stopping Ceph node-exporter.b for a5dd1f32-e785-11ee-95cd-87774f69a715... 2024-03-21T14:04:18.181 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 14:04:17 smithi144 podman[80572]: 2024-03-21 14:04:17.887510537 +0000 UTC m=+0.123338141 container died 4c4e84e2fcaed915f9b00d05fd0021adb18fb96101c3cbc0473757d4594b379f (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-03-21T14:04:18.181 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 14:04:18 smithi144 podman[80572]: 2024-03-21 14:04:18.181447123 +0000 UTC m=+0.417274730 container cleanup 4c4e84e2fcaed915f9b00d05fd0021adb18fb96101c3cbc0473757d4594b379f (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-03-21T14:04:18.510 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 14:04:18 smithi144 bash[80572]: ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b 2024-03-21T14:04:18.510 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 14:04:18 smithi144 podman[80590]: 2024-03-21 14:04:18.39927552 +0000 UTC m=+0.514290444 container remove 4c4e84e2fcaed915f9b00d05fd0021adb18fb96101c3cbc0473757d4594b379f (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-a5dd1f32-e785-11ee-95cd-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-03-21T14:04:18.510 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 14:04:18 smithi144 systemd[1]: ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-03-21T14:04:18.871 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 14:04:18 smithi144 systemd[1]: ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@node-exporter.b.service: Failed with result 'exit-code'. 2024-03-21T14:04:18.871 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 14:04:18 smithi144 systemd[1]: Stopped Ceph node-exporter.b for a5dd1f32-e785-11ee-95cd-87774f69a715. 2024-03-21T14:04:18.871 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Mar 21 14:04:18 smithi144 systemd[1]: ceph-a5dd1f32-e785-11ee-95cd-87774f69a715@node-exporter.b.service: Consumed 5.178s CPU time. 2024-03-21T14:04:19.148 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-21T14:04:19.183 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-21T14:04:19.213 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-21T14:04:19.215 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/gabrioux-2024-03-21_12:04:17-orch:cephadm-wip-guits-testing-2024-03-21-0818-distro-default-smithi/7614989/remote/smithi097/crash 2024-03-21T14:04:19.216 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/crash -- . 2024-03-21T14:04:19.270 INFO:teuthology.orchestra.run.smithi097.stderr:tar: /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-21T14:04:19.270 INFO:teuthology.orchestra.run.smithi097.stderr:tar: Error is not recoverable: exiting now 2024-03-21T14:04:19.272 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/gabrioux-2024-03-21_12:04:17-orch:cephadm-wip-guits-testing-2024-03-21-0818-distro-default-smithi/7614989/remote/smithi144/crash 2024-03-21T14:04:19.273 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/crash -- . 2024-03-21T14:04:19.309 INFO:teuthology.orchestra.run.smithi144.stderr:tar: /var/lib/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-21T14:04:19.309 INFO:teuthology.orchestra.run.smithi144.stderr:tar: Error is not recoverable: exiting now 2024-03-21T14:04:19.310 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-21T14:04:19.311 DEBUG:teuthology.orchestra.run.smithi097:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-03-21T14:04:19.344 INFO:teuthology.orchestra.run.smithi097.stdout:2024-03-21T13:29:31.728276+0000 mon.a (mon.0) 1076 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-03-21T14:04:19.344 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-03-21T14:04:19.344 DEBUG:teuthology.orchestra.run.smithi097:> sudo egrep '\[SEC\]' /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-03-21T14:04:19.416 DEBUG:teuthology.orchestra.run.smithi097:> sudo egrep '\[ERR\]' /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-03-21T14:04:19.488 DEBUG:teuthology.orchestra.run.smithi097:> sudo egrep '\[WRN\]' /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-03-21T14:04:19.559 INFO:teuthology.orchestra.run.smithi097.stdout:2024-03-21T13:29:31.728276+0000 mon.a (mon.0) 1076 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-03-21T14:04:19.560 INFO:tasks.cephadm:Compressing logs... 2024-03-21T14:04:19.560 DEBUG:teuthology.orchestra.run.smithi097:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-21T14:04:19.602 DEBUG:teuthology.orchestra.run.smithi144:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-21T14:04:19.628 INFO:teuthology.orchestra.run.smithi097.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-21T14:04:19.628 INFO:teuthology.orchestra.run.smithi097.stderr:: No such file or directory 2024-03-21T14:04:19.629 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph-mon.a.log 2024-03-21T14:04:19.629 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph.audit.log 2024-03-21T14:04:19.629 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph-client.rgw.foo.a.log 2024-03-21T14:04:19.629 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph-mon.a.log: /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph-osd.1.log 2024-03-21T14:04:19.629 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph-osd.3.log 2024-03-21T14:04:19.629 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph-client.rgw.foo.a.log: /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph-mgr.y.log 2024-03-21T14:04:19.629 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph-osd.0.log 2024-03-21T14:04:19.635 INFO:teuthology.orchestra.run.smithi144.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-21T14:04:19.635 INFO:teuthology.orchestra.run.smithi144.stderr:: No such file or directory 2024-03-21T14:04:19.635 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph-osd.6.log 2024-03-21T14:04:19.635 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph.audit.log 2024-03-21T14:04:19.635 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph-osd.4.log 2024-03-21T14:04:19.636 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph-osd.7.log 2024-03-21T14:04:19.636 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph-osd.5.log 2024-03-21T14:04:19.636 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph-osd.4.log: /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph-mgr.x.log 2024-03-21T14:04:19.636 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/tcmu-runner.log 2024-03-21T14:04:19.636 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph.cephadm.log 2024-03-21T14:04:19.636 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/tcmu-runner.log: 67.3% -- replaced with /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/tcmu-runner.log.gz 2024-03-21T14:04:19.636 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph.log 2024-03-21T14:04:19.636 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph.cephadm.log: 81.7% -- replaced with /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph.cephadm.log.gz 2024-03-21T14:04:19.636 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph-volume.log 2024-03-21T14:04:19.637 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph-mgr.y.log: /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph-osd.3.log: 93.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-21T14:04:19.638 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph.cephadm.log 2024-03-21T14:04:19.642 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph.log: 88.0% -- replaced with /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph.log.gz 2024-03-21T14:04:19.642 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph-mon.b.log 2024-03-21T14:04:19.642 INFO:teuthology.orchestra.run.smithi144.stderr: 92.4% -- replaced with /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph.audit.log.gz 2024-03-21T14:04:19.643 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph.log 2024-03-21T14:04:19.645 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph-volume.log: /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph-mon.b.log: 91.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-21T14:04:19.645 INFO:teuthology.orchestra.run.smithi144.stderr: 92.5% -- replaced with /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph-mgr.x.log.gz 2024-03-21T14:04:19.647 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph.cephadm.log: 95.3% -- replaced with /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph.audit.log.gz 2024-03-21T14:04:19.647 INFO:teuthology.orchestra.run.smithi097.stderr: 89.1% -- replaced with /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph.cephadm.log.gz 2024-03-21T14:04:19.653 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph-osd.2.log 2024-03-21T14:04:19.656 INFO:teuthology.orchestra.run.smithi144.stderr: 91.0% -- replaced with /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph-volume.log.gz 2024-03-21T14:04:19.662 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph-volume.log 2024-03-21T14:04:19.663 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph-osd.2.log: 93.3% -- replaced with /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph.log.gz 2024-03-21T14:04:19.664 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph-mon.c.log 2024-03-21T14:04:19.678 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph-volume.log: /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph-mon.c.log: 91.0% -- replaced with /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph-volume.log.gz 2024-03-21T14:04:19.692 INFO:teuthology.orchestra.run.smithi097.stderr: 94.1% -- replaced with /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph-client.rgw.foo.a.log.gz 2024-03-21T14:04:20.195 INFO:teuthology.orchestra.run.smithi097.stderr: 90.8% -- replaced with /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph-mgr.y.log.gz 2024-03-21T14:04:20.621 INFO:teuthology.orchestra.run.smithi144.stderr: 93.0% -- replaced with /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph-mon.b.log.gz 2024-03-21T14:04:21.168 INFO:teuthology.orchestra.run.smithi097.stderr: 92.4% -- replaced with /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph-mon.c.log.gz 2024-03-21T14:04:22.475 INFO:teuthology.orchestra.run.smithi097.stderr: 92.0% -- replaced with /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph-mon.a.log.gz 2024-03-21T14:04:27.948 INFO:teuthology.orchestra.run.smithi144.stderr: 93.9% -- replaced with /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph-osd.6.log.gz 2024-03-21T14:04:28.017 INFO:teuthology.orchestra.run.smithi097.stderr: 94.0% -- replaced with /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph-osd.0.log.gz 2024-03-21T14:04:28.242 INFO:teuthology.orchestra.run.smithi144.stderr: 93.9% -- replaced with /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph-osd.5.log.gz 2024-03-21T14:04:28.418 INFO:teuthology.orchestra.run.smithi144.stderr: 93.9% -- replaced with /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph-osd.7.log.gz 2024-03-21T14:04:28.483 INFO:teuthology.orchestra.run.smithi144.stderr: 94.0% -- replaced with /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph-osd.4.log.gz 2024-03-21T14:04:28.485 INFO:teuthology.orchestra.run.smithi144.stderr: 2024-03-21T14:04:28.485 INFO:teuthology.orchestra.run.smithi144.stderr:real 0m8.868s 2024-03-21T14:04:28.485 INFO:teuthology.orchestra.run.smithi144.stderr:user 0m34.304s 2024-03-21T14:04:28.486 INFO:teuthology.orchestra.run.smithi144.stderr:sys 0m1.267s 2024-03-21T14:04:28.514 INFO:teuthology.orchestra.run.smithi097.stderr: 94.0% -- replaced with /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph-osd.1.log.gz 2024-03-21T14:04:28.565 INFO:teuthology.orchestra.run.smithi097.stderr: 93.9% -- replaced with /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph-osd.2.log.gz 2024-03-21T14:04:29.608 INFO:teuthology.orchestra.run.smithi097.stderr: 94.0% -- replaced with /var/log/ceph/a5dd1f32-e785-11ee-95cd-87774f69a715/ceph-osd.3.log.gz 2024-03-21T14:04:29.611 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-03-21T14:04:29.611 INFO:teuthology.orchestra.run.smithi097.stderr:real 0m9.997s 2024-03-21T14:04:29.611 INFO:teuthology.orchestra.run.smithi097.stderr:user 0m39.658s 2024-03-21T14:04:29.611 INFO:teuthology.orchestra.run.smithi097.stderr:sys 0m1.416s 2024-03-21T14:04:29.611 INFO:tasks.cephadm:Archiving logs... 2024-03-21T14:04:29.611 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-03-21_12:04:17-orch:cephadm-wip-guits-testing-2024-03-21-0818-distro-default-smithi/7614989/remote/smithi097/log 2024-03-21T14:04:29.612 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-21T14:04:34.985 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-03-21_12:04:17-orch:cephadm-wip-guits-testing-2024-03-21-0818-distro-default-smithi/7614989/remote/smithi144/log 2024-03-21T14:04:34.986 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-21T14:04:39.906 INFO:tasks.cephadm:Removing cluster... 2024-03-21T14:04:39.906 DEBUG:teuthology.orchestra.run.smithi097:> sudo cephadm rm-cluster --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 --force 2024-03-21T14:04:40.099 INFO:teuthology.orchestra.run.smithi097.stdout:Deleting cluster with fsid: a5dd1f32-e785-11ee-95cd-87774f69a715 2024-03-21T14:04:40.396 DEBUG:teuthology.orchestra.run.smithi144:> sudo cephadm rm-cluster --fsid a5dd1f32-e785-11ee-95cd-87774f69a715 --force 2024-03-21T14:04:40.577 INFO:teuthology.orchestra.run.smithi144.stdout:Deleting cluster with fsid: a5dd1f32-e785-11ee-95cd-87774f69a715 2024-03-21T14:04:40.873 INFO:tasks.cephadm:Teardown complete 2024-03-21T14:04:40.874 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-21T14:04:40.886 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-21T14:04:40.887 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-21T14:04:40.889 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-21T14:04:40.969 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-21T14:04:40.977 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-21T14:04:40.977 DEBUG:teuthology.orchestra.run.smithi097:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-21T14:04:40.980 DEBUG:teuthology.orchestra.run.smithi144:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-21T14:04:40.992 INFO:teuthology.orchestra.run.smithi097.stderr:bash: line 1: ntpq: command not found 2024-03-21T14:04:40.993 INFO:teuthology.orchestra.run.smithi144.stderr:bash: line 1: ntpq: command not found 2024-03-21T14:04:41.000 INFO:teuthology.orchestra.run.smithi097.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-21T14:04:41.000 INFO:teuthology.orchestra.run.smithi097.stdout:=============================================================================== 2024-03-21T14:04:41.000 INFO:teuthology.orchestra.run.smithi097.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 29 +3293us[+3289us] +/- 140ms 2024-03-21T14:04:41.000 INFO:teuthology.orchestra.run.smithi097.stdout:^* hv02.front.sepia.ceph.com 3 6 377 26 -1399us[-1403us] +/- 80ms 2024-03-21T14:04:41.000 INFO:teuthology.orchestra.run.smithi097.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 30 -74us[ -78us] +/- 67ms 2024-03-21T14:04:41.000 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T14:04:41.001 INFO:teuthology.orchestra.run.smithi144.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-21T14:04:41.001 INFO:teuthology.orchestra.run.smithi144.stdout:=============================================================================== 2024-03-21T14:04:41.001 INFO:teuthology.orchestra.run.smithi144.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 27 +3201us[+3211us] +/- 140ms 2024-03-21T14:04:41.001 INFO:teuthology.orchestra.run.smithi144.stdout:^* hv02.front.sepia.ceph.com 3 6 377 23 -1504us[-1494us] +/- 80ms 2024-03-21T14:04:41.001 INFO:teuthology.orchestra.run.smithi144.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 25 -210us[ -200us] +/- 67ms 2024-03-21T14:04:41.001 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T14:04:41.002 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-21T14:04:41.011 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-21T14:04:41.011 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-21T14:04:41.020 DEBUG:teuthology.orchestra.run.smithi097:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-21T14:04:41.121 DEBUG:teuthology.orchestra.run.smithi144:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-21T14:04:41.195 DEBUG:teuthology.orchestra.run.smithi097:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-03-21T14:04:41.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T14:04:41.226 DEBUG:teuthology.orchestra.run.smithi144:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-03-21T14:04:41.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T14:04:41.256 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-21T14:04:41.265 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-21T14:04:41.319 INFO:teuthology.task.internal:Duration was 3137.471340 seconds 2024-03-21T14:04:41.319 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-21T14:04:41.328 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-21T14:04:41.328 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-21T14:04:41.331 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-21T14:04:41.371 INFO:teuthology.orchestra.run.smithi097.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-21T14:04:41.374 INFO:teuthology.orchestra.run.smithi144.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-21T14:04:41.668 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-21T14:04:41.668 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi097.front.sepia.ceph.com 2024-03-21T14:04:41.668 DEBUG:teuthology.orchestra.run.smithi097:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-21T14:04:41.687 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi144.front.sepia.ceph.com 2024-03-21T14:04:41.687 DEBUG:teuthology.orchestra.run.smithi144:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-21T14:04:41.730 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-21T14:04:41.730 DEBUG:teuthology.orchestra.run.smithi097:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-21T14:04:41.732 DEBUG:teuthology.orchestra.run.smithi144:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-21T14:04:41.798 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-21T14:04:41.798 DEBUG:teuthology.orchestra.run.smithi097:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-21T14:04:41.801 DEBUG:teuthology.orchestra.run.smithi144:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-21T14:04:42.232 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-21T14:04:42.241 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-21T14:04:42.242 DEBUG:teuthology.orchestra.run.smithi097:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-21T14:04:42.268 DEBUG:teuthology.orchestra.run.smithi144:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-21T14:04:42.295 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-21T14:04:42.307 DEBUG:teuthology.orchestra.run.smithi097:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-21T14:04:42.311 DEBUG:teuthology.orchestra.run.smithi144:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-21T14:04:42.334 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = core 2024-03-21T14:04:42.360 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern = core 2024-03-21T14:04:42.384 DEBUG:teuthology.orchestra.run.smithi097:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-21T14:04:42.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T14:04:42.403 DEBUG:teuthology.orchestra.run.smithi144:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-21T14:04:42.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T14:04:42.439 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-21T14:04:42.449 INFO:teuthology.task.internal:Transferring archived files... 2024-03-21T14:04:42.450 DEBUG:teuthology.misc:Transferring archived files from smithi097:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-03-21_12:04:17-orch:cephadm-wip-guits-testing-2024-03-21-0818-distro-default-smithi/7614989/remote/smithi097 2024-03-21T14:04:42.450 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-21T14:04:42.514 DEBUG:teuthology.misc:Transferring archived files from smithi144:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-03-21_12:04:17-orch:cephadm-wip-guits-testing-2024-03-21-0818-distro-default-smithi/7614989/remote/smithi144 2024-03-21T14:04:42.514 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-21T14:04:42.554 INFO:teuthology.task.internal:Removing archive directory... 2024-03-21T14:04:42.554 DEBUG:teuthology.orchestra.run.smithi097:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-21T14:04:42.556 DEBUG:teuthology.orchestra.run.smithi144:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-21T14:04:42.603 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-21T14:04:42.616 INFO:teuthology.task.internal:Not uploading archives. 2024-03-21T14:04:42.616 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-21T14:04:42.625 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-21T14:04:42.625 DEBUG:teuthology.orchestra.run.smithi097:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-21T14:04:42.627 DEBUG:teuthology.orchestra.run.smithi144:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-21T14:04:42.639 INFO:teuthology.orchestra.run.smithi097.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 21 14:04 /home/ubuntu/cephtest 2024-03-21T14:04:42.657 INFO:teuthology.orchestra.run.smithi144.stdout: 266042 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 21 14:04 /home/ubuntu/cephtest 2024-03-21T14:04:42.658 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-21T14:04:42.668 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-21T14:04:42.688 INFO:teuthology.nuke:Checking targets against current locks 2024-03-21T14:04:42.716 DEBUG:teuthology.nuke:shortname: smithi097 2024-03-21T14:04:42.716 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-21T14:04:42.743 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi097.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-03-21_12:04:17-orch:cephadm-wip-guits-testing-2024-03-21-0818-distro-default-smithi/7614989', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-21 12:58:11.414860', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN/oDPzQhVaF9hfNE3tZavEfktdnClngzT1j2vsNv2ywHYOQiuzoOiVPq/+1imK/u71NhX4vTPxergJdOjhjgZs='} 2024-03-21T14:04:42.747 DEBUG:teuthology.nuke:shortname: smithi144 2024-03-21T14:04:42.747 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-21T14:04:42.767 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi144.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-03-21_12:04:17-orch:cephadm-wip-guits-testing-2024-03-21-0818-distro-default-smithi/7614989', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-21 12:58:11.416562', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNsHy2xxQJP5PcBlwnWObdXJc8jXGCO4Z5IAjWxkBkRtQIelMLzm/vvLeNagsA7HWY6wQ7+KeEYbg12jhJDWUYY='} 2024-03-21T14:04:42.798 INFO:teuthology.orchestra.console.smithi097:Power off 2024-03-21T14:04:42.798 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-21T14:04:42.822 DEBUG:teuthology.orchestra.console.smithi097:power off output: Chassis Power Control: Down/Off 2024-03-21T14:04:42.823 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T14:04:42.835 INFO:teuthology.orchestra.console.smithi144:Power off 2024-03-21T14:04:42.836 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-21T14:04:42.845 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is on 2024-03-21T14:04:42.858 DEBUG:teuthology.orchestra.console.smithi144:power off output: Chassis Power Control: Down/Off 2024-03-21T14:04:42.858 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T14:04:42.883 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is on 2024-03-21T14:04:46.846 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T14:04:46.884 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T14:04:46.956 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is on 2024-03-21T14:04:46.993 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is on 2024-03-21T14:04:50.959 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T14:04:50.994 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T14:04:51.068 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is off 2024-03-21T14:04:51.104 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is off 2024-03-21T14:04:51.170 INFO:teuthology.orchestra.console.smithi097:Power off completed 2024-03-21T14:04:51.205 INFO:teuthology.orchestra.console.smithi144:Power off completed 2024-03-21T14:04:51.309 INFO:teuthology.run:Summary data: description: orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} duration: 3137.4713401794434 failure_reason: '"2024-03-21T13:29:31.728276+0000 mon.a (mon.0) 1076 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log' flavor: default owner: scheduled_gabrioux@teuthology success: false 2024-03-21T14:04:51.310 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T14:04:51.394 INFO:teuthology.run:FAIL